Search Results for "data scraper website"
Sort By:
Showing 135 open source projects for "data scraper website"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Forever Free Full-Stack Observability | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
1
Linkedin Scraper
A library that scrapes Linkedin for user data
Linkedin Scraper is a library that scrapes Linkedin for user data. Version 2.0.0 and before is called linkedin_user_scraper and can be installed via pip3 install --user linkedin_user_scraper. The reason is that LinkedIn has recently blocked people from viewing certain profiles without having previously signed in. So by setting scrape=False, it doesn't automatically scrape the profile, but Chrome will open the linkedin page anyways. -
2
dude uncomplicated data extraction
dude uncomplicated data extraction: A simple framework
Dude is a very simple framework for writing web scrapers using Python decorators. The design, inspired by Flask, was to easily build a web scraper in just a few lines of code. Dude has an easy-to-learn syntax. Dude is currently in Pre-Alpha. Please expect breaking changes. You can run your scraper from terminal/shell/command-line by supplying URLs, the output filename of your choice and the paths to your python scripts to dude scrape command. -
3
JobFunnel
Scrape job websites into a single spreadsheet with no duplicates.
...Run funnel with your settings YAML to populate your master CSV file with jobs from available providers. JobFunnel can be easily automated to run nightly with crontab. If you have a job website you'd like to write a scraper for, you are welcome to implement it, Review the Base Scraper for implementation details. JobFunnel supports scraping jobs from the same job website across locales & domains. If you are interested in adding support, you may only need to define session headers and domain strings, Review the Base Scraper for further implementation details. -
4
CyberScraper 2077
A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama
CyberScraper 2077 is not just another web scraping tool – it's a glimpse into the future of data extraction. Born from the neon-lit streets of a cyberpunk world, this AI-powered scraper uses OpenAI, Gemini and LocalLLM Models to slice through the web's defenses, extracting the data you need with unparalleled precision and style. -
AI-generated apps that pass security reviewRetool lets you generate dashboards, admin panels, and workflows directly on your data. Type something like “Build me a revenue dashboard on my Stripe data” and get a working app with security, permissions, and compliance built in from day one. Whether on our cloud or self-hosted, create the internal software your team needs without compromising enterprise standards or control.
-
5
Spider
High-performance Rust web crawler and scraper for large-scale data
Spider is a high-performance web crawler and web scraping library written in Rust that enables developers to crawl and index websites efficiently. It focuses on speed, concurrency, and reliability by using asynchronous and multi-threaded processing to handle large volumes of web pages. It can rapidly crawl websites to collect links, retrieve page content, and extract structured information from HTML documents. Spider can operate concurrently across many pages, allowing it to gather large... -
6
ScrapeGraphAI
Python scraper based on AI
Extracting content from websites and local documents using LLM. ScrapeGraphAI is a web scraping python library that uses LLM and direct graph logic to create scraping pipelines for websites and local documents (XML, HTML, JSON, Markdown, etc.). Just say which information you want to extract and the library will do it for you. -
7
Automa
A chrome extension for automating your browser by connecting blocks
Automa is a browser extension for browser automation. From auto-fill forms, doing a repetitive task, taking a screenshot, to scraping data of the website, it's up to you what you want to do with this extension. Automa has provided various kinds of blocks that will help you do automation, and all you need to do is connect them. Want your workflow to run every day or every time you visit a specific website? You can set the workflow trigger on the trigger block. Try a workflow from the marketplace. ... -
8
Matomo
Alternative to Google Analytics that gives you full control over data
Google Analytics alternative that protects your data and your customers' privacy. Take back control with Matomo – a powerful web analytics platform that gives you 100% data ownership. You could lose your customers’ trust and risk damaging your reputation if people learn their data is used for Google’s “own purposes”. By choosing the ethical alternative, Matomo, you won’t make privacy sacrifices or compromise your site. -
9
watercrawl
AI-ready web crawler that extracts and structures website content
WaterCrawl is an open source web crawling and data extraction platform designed to transform website content into structured data suitable for machine learning and AI workflows. It enables developers and researchers to crawl web pages, extract meaningful information, and convert it into formats that are easier to process and analyze. It provides a modern crawling system that can automatically navigate links, control crawl depth, and collect content from targeted sections of a website. ... -
Enterprise-grade ITSM, for every businessFreshservice is an intuitive, AI-powered platform that helps IT, operations, and business teams deliver exceptional service without the usual complexity. Automate repetitive tasks, resolve issues faster, and provide seamless support across the organization. From managing incidents and assets to driving smarter decisions, Freshservice makes it easy to stay efficient and scale with confidence.
-
10
Crawl4AI
Open-source LLM Friendly Web Crawler & Scraper
Crawl4AI is a high-performance, AI‑ready web crawler tailored for LLM data ingestion and RAG pipelines. It supports adaptive crawling heuristics (stopping when enough info is gathered), structured markdown output, and high-speed parallel execution. Designed to operate at scale with optional Docker deployment and framework integrations. -
11
Firecrawl
Turn entire websites into LLM-ready markdown or structured data
Crawl and convert any website into LLM-ready markdown or structured data. Built by Mendable.ai and the Firecrawl community. Includes powerful scraping, crawling, and data extraction capabilities. Firecrawl is an API service that takes a URL, crawls it, and converts it into clean markdown or structured data. We crawl all accessible subpages and give you clean data for each. -
12
crwlr
Library for Rapid (Web) Crawler and Scraper Development
This library provides kind of a framework and a lot of ready-to-use, so-called steps, that you can use as building blocks, to build your own crawlers and scrapers with. Before diving into the library, let's have a look at the terms crawling and scraping. For most real-world use cases, those two things go hand in hand, which is why this library helps with and combines both. A (web) crawler is a program that (down)loads documents and follows the links in it to load them as well. A crawler... -
13
Ghostery
Ghostery Browser Extension for Firefox, Chrome, Opera and Edge
Ghostery helps you browse smarter by giving you control over ads and tracking technologies to speed up page loads, eliminate clutter, and protect your data. This is the unified code repository for the Ghostery browser extensions in Chrome, Firefox, Opera and Edge. Browse the web safer, faster & with less annoying ads. Equipped with award-winning AI anti-tracking technology to browse the websafe and quickly. Ghostery helps you stay informed about what companies are tracking you by listing the trackers on each website you visit. ... -
14
Laravel Sharp
Laravel 10+ Content management framework
Sharp is a content management framework, a toolset that provides help to build a CMS section in a website, with some rules in mind. The public website should not have any knowledge of the CMS, the CMS is a part of the system, not the center of it. In fact, removing the CMS should not have any effect on the project. Content administrators should work with their data and terminology, not CMS terms. I mean, if the project is about spaceships, space travels, and pilots, why would the CMS talk about articles, categories, and tags? ... -
15
SkyCrypt
A Hypixel skyblock stats website
SkyCrypt is a web-based application that allows players of Hypixel SkyBlock to view and share detailed information about their in-game profiles through a visually rich interface. It aggregates data from the Hypixel API and presents it in an organized format, including player statistics, skills, equipment, and inventory details. The project is built with a Node.js-based stack and integrates additional technologies such as MongoDB and Redis to handle data storage and caching. SkyCrypt enhances... -
16
MDCx
Movie metadata scraper and organizer for media libraries and NFO
MDCx is an open source media metadata scraping and organization tool designed to automate the process of collecting detailed information for movie files. It retrieves metadata from multiple online sources and applies it to local media collections, helping users maintain structured and well-organized libraries. MDCx can download information such as titles, cast data, artwork, and other metadata, then generate standardized NFO files compatible with media management systems. It also supports... -
17
I Still Don't Care About Cookies
Debloated fork of the extension "I don't care about cookies"
Debloated fork of the extension "I don't care about cookies". Get rid of cookie warnings from almost all websites! This extension has been acquired by Avast and simply I don't trust Avast with my data. Also having it on Github allows us to improve the code & add support for websites faster. The EU regulations require that any website using tracking cookies must get user's permission before installing them. These warnings appear on most websites until the visitor agrees with the website's terms and conditions. Imagine how irritating that becomes when you surf anonymously or if you delete cookies automatically every time you close the browser. ... -
18
ZetaJS
JS wrapper for ZetaOffice in the browser
The zeta.js library provides the facilities to run an instance of ZetaOffice integrated into your web site, allowing you to control it with JavaScript code via the LibreOffice UNO technology. Use cases range from an in-browser office suite that looks and feels just like its desktop counterpart, to fine-tuned custom text editing and spreadsheet capabilities embedded in your website, to a headless zetajs instance that does document conversion in the background. -
19
finvizfinance
Finviz analysis python library
finvizfinance is a package that collects financial information from FinViz website. Stock charts, fundamental & technical information, insider information and stock news. Forex charts and performance. Crypto charts and performance. Screener and Group provide data frames for comparing stocks according to different filters and trading signals. Getting information (fundament, description, outer rating, stock news, inside trader) of an individual stock. -
20
Surmon.me
Personal website and blog
Surmon.me is a full-featured personal website and blog platform built with Vue and designed as part of a larger ecosystem of interconnected applications and services. The project functions as a server-side rendered (SSR) web application that delivers content dynamically while maintaining performance and SEO optimization. It is powered by a dedicated backend service called NodePress, which provides RESTful APIs for content management, data retrieval, and system operations. ... -
21
Umami
A simple, fast, website analytics alternative to Google Analytics
Umami is a simple, easy to use, self-hosted web analytics solution. The goal is to provide you with a friendlier, privacy-focused alternative to Google Analytics and a free, open-sourced alternative to paid solutions. Umami collects only the metrics you care about and everything fits on a single page. You can view a live demo here. Umami measures just the important metrics that you care about: pageviews, devices used, and where your visitors are coming from. Everything is displayed on a... -
22
cat-catch
Cat scratch chrome resource sniffing extension
...Maozha is open source. Anyone can download and modify it and put it on the app store. There are already many fake Maozha with the ad code added. Please pay attention to your own data security. All installation addresses are subject to github and user documentation. Chromium kernel version 93 or later is required after version 1.0.17. If it is lower than 93, please use version 1.0.16. The extension is a general sniffing tool and has the same function as the browser DevTools. It does not decrypt any website. Any content downloaded by the user has nothing to do with the extension. ... -
23
Z-BlogPHP
Z-BlogPHP blog program
...Our goal is to immerse users in writing and record life, without paying attention to cumbersome settings, etc., and let users focus on creation. For users, it is simple and easy to use, small in size, fast in speed, and large in support data. For developers, it also has a powerful customizable, rich plug interface and a beautiful theme template. We have been working hard to make Z-BlogPHP a highly playable, Lego bricks website program! -
24
Scrapling
An adaptive Web Scraping framework
Scrapling is an adaptive web scraping framework designed to handle everything from a single HTTP request to large-scale, concurrent crawls. Built for modern websites, it intelligently adapts to structural changes by automatically relocating elements when page layouts update. The framework includes advanced fetchers capable of bypassing anti-bot protections such as Cloudflare Turnstile using stealth and browser automation techniques. Its powerful spider system supports multi-session crawling,... -
25
Typecho Blogging Platform
A PHP blogging platform, simple and powerful
...Choose a reliable hosting provider who can provide support services for the installation process. Download the version of the program that suits you and install it. The program will run in a second. Read the documentation, to make a unique website, and wait for your friends' reaction.
