Search Results for "scraping"
Sort By:
Showing 14 open source projects for "scraping"
View related business solutions-
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
1
learn_python3_spider
Python crawler tutorial series, learning Python crawlers from 0 to 1
...Data can be stored with CSV, MySQL, or MongoDB examples. More advanced material explores anti-scraping defenses, JavaScript analysis, WebSocket data, and distributed crawlers. The repository includes practical source-code examples for scraping several real-world websites and services. -
2
Zendriver
A blazing fast, async-first, undetectable webscraping
Zendriver is a modern Python web automation and scraping framework that leverages the Chrome DevTools Protocol to provide fast, asynchronous control over real browser instances. Unlike traditional tools that rely on Selenium or WebDriver, Zendriver communicates directly with the browser through CDP, enabling higher performance and more precise control over browser behavior. The framework is designed to be difficult to detect by anti-bot systems, making it suitable for advanced scraping and automation use cases where stealth is important. ... -
3
Parsera
Lightweight library for scraping web-sites with LLMs
Scrape data from any website with only a link and column descriptions. Parsera is a tool designed to scrape web content, specifically handling poorly structured or messy websites. -
4
Python for OSINT
In this repository you will find sample code files for each day
...It accompanies a free course PDF and provides sample code for lessons organized across three weeks. Topics progress from basic syntax and command-line tools to HTTP requests, APIs, JSON, CSV, databases, scraping, regular expressions, proxies, and file handling. Later lessons cover domain research, document generation, charts, maps, web archives, dates, and simple web applications. The course is designed for investigators and researchers rather than people seeking comprehensive software-engineering training. Exercises emphasize modifying working examples and applying them to practical research tasks. ... -
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
5
Scrapy
A fast, high-level web crawling and web scraping framework
Scrapy is a fast, open source, high-level framework for crawling websites and extracting structured data from these websites. Portable and written in Python, it can run on Windows, Linux, macOS and BSD. Scrapy is powerful, fast and simple, and also easily extensible. Simply write the rules to extract the data, and add new functionality if you wish without having to touch the core. Scrapy does the rest, and can be used in a number of applications. It can be used for data mining, monitoring... -
6
Helium
Lighter web automation with Python
...It replaces verbose boilerplate code with natural language-like API calls such as click("Login") or write("hello", into="Name"). Helium manages browser setup, waits, and teardown, enabling quick development of scripts for testing, scraping, or task automation without requiring deep Selenium knowledge. -
7
DrissionPage
Python based web automation tool. Powerful and elegant
DrissionPage is a Python-based automation framework that blends the capabilities of Selenium for browser automation with Requests-HTML for fast, headless web data extraction. It enables seamless switching between browser-controlled and headless HTTP sessions within the same interface. Ideal for web scraping, testing, and automation, DrissionPage is lightweight and highly efficient, offering more flexibility than standard Selenium or Requests usage alone. -
8
My Python Eggs
Python Examples
...Rather than being a single cohesive application, it functions as a repository of utilities that demonstrate how Python can be used to solve everyday problems and automate repetitive tasks. The scripts cover a wide range of topics, including file management, networking, system monitoring, web scraping, and even simple games, making it a versatile learning resource. Many of the programs are designed to reduce manual workload by automating tasks such as renaming files, scanning directories, or checking system information. The repository also includes examples of more advanced concepts like multithreading, API interaction, and GUI development, providing a gradual learning curve for beginners. -
9
Grab Framework Project
Web Scraping Framework
Grab is a python framework for building web scrapers. With Grab you can build web scrapers of various complexity, from simple 5-line scripts to complex asynchronous website crawlers processing millions of web pages. Grab provides an API for performing network requests and for handling the received content e.g. interacting with DOM tree of the HTML document. The single request/response API that allows you to build network request, perform it and work with the received content. The API is... -
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
10
Python-Spider
Python3 web crawler practice
...As part of the author’s public learning-path repositories, python-spider likely includes examples of HTTP requests, HTML parsing, maybe concurrency or scheduling to crawl multiple pages, and techniques to handle common web-scraping issues. For people wanting to get hands-on with building scrapers, collecting data, or learning how to navigate web programming in Python, this repository acts as a didactic reference or starting point. Because it’s published publicly under an open license, users are free to fork and adapt the code. -
11
GitFive
Track down GitHub users
...It can generate possible email combinations and test them for matches. GitFive also identifies potential secondary accounts and can extract public SSH keys associated with a target. Its implementation uses asynchronous scraping, multiprocessing, and low API consumption to improve investigation speed while remaining within GitHub rate limits. -
12
Twint
An advanced Twitter scraping & OSINT tool written in Python
Twint is an advanced open-source Twitter scraping and OSINT tool written in Python that extracts tweets, user data, followers, likes, and more—without relying on Twitter’s API—making it highly useful for researchers, analysts, and hobbyists who want to bypass rate limits and access public Twitter data. -
13
django-dynamic-scraper
Creating Scrapy scrapers via the Django admin interface
Django Dynamic Scraper (DDS) is an app for Django build on top of the scraping framework Scrapy. While preserving many of the features of Scrapy it lets you dynamically create and manage spiders via the Django admin interface. With Django Dynamic Scraper (DDS) you can define your Scrapy scrapers dynamically via the Django admin interface and save your scraped items in the database you defined for your Django project. -
14
HTML DOM Parser
HTML parser which can be used for screen-scraping applications
htmldom parses the HTML file and provides methods for iterating and searching the parse tree in a similar way as Jquery. To report bugs please mail me at bhimsen.pes@gmail.com
- Previous
- You're on page 1
- Next
