Search Results for "data scraper website" - Page 3
Sort By:
Showing 76 open source projects for "data scraper website"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Gemini 3 and 200+ AI Models on One PlatformBuild generative AI apps with Vertex AI. Switch between models without switching platforms.
-
1
Trape
OSINT tool for tracking users and analyzing browser data online
Trape is an open source OSINT analysis and research tool designed to track and analyze users on the internet in real time. The project focuses on demonstrating how web browsers can reveal sensitive information about users while interacting with websites and online services. It provides researchers, security professionals, and organizations with a platform for studying how attackers could gather intelligence through social engineering techniques. The tool can clone websites and monitor... -
2
Fav-up
Look up IP addresses using favicon hashes via Shodan
fav-up is an open source reconnaissance utility designed to identify servers and websites by analyzing favicon hashes and querying the Shodan search engine. It calculates the hash of a favicon image and then uses that value to search Shodan for other hosts that use the same icon. This technique is commonly used in security research and OSINT investigations to discover related infrastructure or services that may belong to the same organization. fav-up can retrieve favicon data from several... -
3
ECommerceCrawlers
Collection of Python ecommerce and website crawler examples projects
ECommerceCrawlers is a collection of practical Python web crawler projects designed to gather data from a variety of ecommerce platforms, websites, and online services. It aggregates many independent crawler examples created by contributors and organized into separate subprojects that target specific sites or data sources. These examples demonstrate how to build and operate web scrapers capable of collecting structured information such as product listings, news content, job postings, social... -
4
django-dynamic-scraper
Creating Scrapy scrapers via the Django admin interface
...Since it simplifies things DDS is not usable for all kinds of scrapers, but it is well suited for the relatively common case of regularly scraping a website with a list of updated items (e.g. news, events, etc.) and then dig into the detail page to scrape some more infos for each item. Django Dynamic Scraper tries to keep its data structure in the database as separated as possible from the models in your app, so it comes with its own Django model classes for defining scrapers, runtime information related to your scraper runs and classes. -
Enterprise-grade ITSM, for every businessFreshservice is an intuitive, AI-powered platform that helps IT, operations, and business teams deliver exceptional service without the usual complexity. Automate repetitive tasks, resolve issues faster, and provide seamless support across the organization. From managing incidents and assets to driving smarter decisions, Freshservice makes it easy to stay efficient and scale with confidence.
-
5
ShadowSocksShare
Python ShadowSocks framework
This project obtains the shared ss(r) account from the ss(r) shared website crawler, redistributes the account and generates a subscription link by parsing and verifying the account connectivity. Since Google plus will be closed on April 2, 2019, almost all the available accounts crawled before come from Google plus. So if you are building your own website, please keep an eye on the updates of this project and redeploy using the latest source code. -
6
mzitu
Python crawler that downloads image galleries and analyzes titles
...Using text segmentation and frequency analysis, the project can create a word cloud representing common keywords found in the dataset. This makes the repository both a scraping example and a small data analysis experiment built around the collected content. Overall, mzitu serves as a learning-oriented implementation of Python web scraping, data processing, and visualization techniques. -
7
BtcGold2Xml
Convert Gold and/or Bitcoin Chart from Website into XML Output
This python script turns gold chart and bitcoin chart html website input into an xml file. -
8
Basis Retriever
Download your data from mybasis.com, store it on your hard drive.
...Within the app directory, try deleting the cookie file (basis_retr.cookie), then rerun BasisRetriever. I will work on a permanent fix. -Rich email me at rg1 /at/ rawds.com if you have any additional problems. What it does: Retrieve the biometric data you recorded on your Basis Peak or B1 fitness watch (which Basis automatically uploads to their servers at app.mybasis.com). This application downloads the data from the Basis website, converts it to csv, and stores it on your hard drive. BasisRetriever is a portable application. Just download extract the zip file to wherever you please. -
9
ReorJS
Distributed Computing with JavaScript
Create your own distributed computer that can distributed javascript based applications to any computer with a web browser, headless browser or node.js installation. For more information and updates please see our website - http://reorjs.com. -
Full-stack observability with actually useful AI | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
10
Protospacer
Rapid gRNA design and validation for CRISPR
...From here, users are able to evaluate potential off targeting, to export results in FASTA format or tabulated (e.g. for Microsoft Excel), and to connect to the Integrative Genomics Viewer (IGVref) and view target-sites in the context of genomic annotations, amino acid translations, and HT-Seq data (for off-target validation). We provide online video tutorials and a toy database from our main website. -
11
.... ######################## ROOTPWA is a toolkit for partial-wave analysis of multi-particle final states produced in high-energy particle reactions. It is used to determine hadron spectra from experimental data. NOTE: The code is only accessible through the git repository in the "Code" section of this website. Please refer to the README file on how to get and install ROOTPWA and on how to contribute to the project. In case you are using ROOTPWA on a regular basis, please consider signing up for our low-traffic user's mailing list. ...
-
12
openlsm - Web based open source, Linux Server Management(LSM) -LAMP ,FTP,Jabber, Samba,NFS,LDAP and MySQL based virtual hosting, Squid,Backup,Quota,Mail ,spam-assassin, mailable,Asterisk, web hosting, Load balancing, nagios and ganglia monitoring.
-
13
SE Auditor
Free SEO audit software.
SE Auditor is a program for analyzing web pages for search engines. SE Auditor is application that you can use to view statistical data about your website, in order to improve its position within the Web search results. SE Auditor is addressed to SEO professionals, website designers, developers, website testers and owners. SE Auditor enables you to check meta description, keywords, sitemap, the number of links and keyword consistency, the text/HTML ratio and many more ranking / usability / social factors. ... -
14
awb combines simple but powerful AsciiDoc markup with templates, blog and image gallery generation, and sitemap.xml generation to allow you to easily maintain and update a website.
-
15
IP Proxy Scraper - Linux
Extracts multiple proxies from a list of websites
Lightweigh and easy to use tool to extract multiple proxies from a list of websites. IP Proxy Scraper is also available for windows, check it out here: https://sourceforge.net/projects/ipproxyscraper/ -
16
Open-source weather database system for storage of weather/water observations, forecasts, analysis fields, and hindcasts. includes loading program for GRIB files. Visit project website (http://wdb.met.no) for further information. Please note: this project has now moved to http://wdb.met.no
-
17
RAWR - Rapid Assessment of Web Resources
A web interface enumeration tool for simplifying red team reporting.
Introducing RAWR (Rapid Assessment of Web Resources). There's a lot packed in this tool that will help you get a better grasp of the threat landscape that is your client's web resources. It has been tested from extremely large network environments, down to 5 node networks. It has been fine-tuned to promote fast, accurate, and applicable results in usable formats. RAWR will make the mapping phase of your next web assessment efficient and get you producing positive results faster! -
18
PyLibrary
Libraries for Python developers.
Development in Python (be it website or an App development or implementation of an automation framework) always involves certain operations like handling db queries, operations on web, development of data structures, windows operations (handing services, registries), logging and many more... What, if you have these libraries handy with you all the time? Just import and start using them.. -
19
Web Crawler Security Tool
A web crawler oriented to information security.
Last update on tue mar 26 16:25 UTC 2012 The Web Crawler Security is a python based tool to automatically crawl a web site. It is a web crawler oriented to help in penetration testing tasks. The main task of this tool is to search and list all the links (pages and files) in a web site. The crawler has been completely rewritten in v1.0 bringing a lot of improvements: improved the data visualization, interactive option to download files, increased speed in crawling, exports list of... -
20
Data mines the voting record and other actions of Members of the UK Parliament. Extracts information from the parliament website and stores it in a database. Provides tools to analyse the information, producing statistics and tables about the MPs.
-
21
A HTML scraper that uses machine learning frameworks to extract labelled fields from raw HTML. The project also involves the development of a tool to display the semi structured data generated by the scraper component.
-
22
gpsdb automatically generates websites from your gps track files, enabling you to quickly and easily share your gps data.
-
23
LegionWeb is a Python (CGI) website engine. Initially designed simply to allow a site to be templated, it is now developing more mature features. Data is stored in HTML and XML files, although MySQL support is planned.
-
24
Provide a command line client to automate the download of and password removal from market data archives available on the McLagan website 'https://www.McLaganCompResults.com'.
-
25
Green Cloud Online portal
Green cloud: A versatile sensor based precision agriculture system, in
...The environment data is collected via sensors is uploaded to the website
