Search Results for "python web crawler"
Sort By:
1106 projects for "python web crawler" with 1 filter applied:
-
Level Up Your Cyber Defense with External Threat ManagementMove beyond alerts. Gain full visibility, context, and control over your external attack surface to stay ahead of every threat.
-
Simple, Secure Domain RegistrationRegister or renew your domain and pay only what we pay. No markups, hidden fees, or surprise add-ons. Choose from over 400 TLDs (.com, .ai, .dev). Every domain is integrated with Cloudflare's industry-leading DNS, CDN, and free SSL to make your site faster and more secure. Simple, secure, at-cost domain registration.
-
1
Spatie Crawler
An easy to use, powerful crawler implemented in PHP
Spatie Crawler is a PHP library that allows developers to crawl websites and extract information efficiently. It can be used for web scraping, link checking, or automated testing of web pages. The library is simple to use and supports customizable crawling strategies, including controlling crawl depth and handling redirects. It’s suitable for building crawlers that navigate large or dynamically generated websites. -
2
FastAPI Python
FastAPI framework, high performance, easy to learn, fast to code
FastAPI framework, high performance, easy to learn, fast to code, ready for production. FastAPI is a modern, fast (high-performance), web framework for building APIs with Python based on standard Python type hints. -
3
Tabby Web
An SSH/Telnet/Serial client in your browser
Tabby Web brings a modern terminal experience to the browser by pairing a web UI with a backend gateway that brokers TCP connections over WebSockets. It aims to deliver an experience similar to the desktop Tabby terminal—sessions, profiles, and rich configuration—while being accessible anywhere through a login. The architecture splits concerns: a Django-based control plane manages users, auth, and configuration, while a gateway service handles network transport so browser clients can reach... -
4
Heritrix
Internet Archive's open-source, web-scale, web crawler project
Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project. Heritrix (sometimes spelled heretrix, or misspelled or missaid as heratrix/heritix/heretix/heratix) is an archaic word for heiress (woman who inherits). Since our crawler seeks to collect and preserve the digital artifacts of our culture for the benefit of future researchers and generations, this name seemed apt. -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
5
OpenAI Quickstart Python
Python example app from the OpenAI API quickstart tutorial
...The examples folder includes small, self-contained projects showcasing common use cases like chat completions, tool usage, and interactive interfaces. Each example is designed to be easily runnable with minimal setup—requiring only Python, a virtual environment, and an API key. The repository also includes environment setup guides and example scripts, such as a simple Flask web app for chat interactions, allowing developers to test OpenAI API integrations locally. Overall, openai-quickstart-python serves as an essential starting point for developers looking to prototype and experiment with OpenAI-powered apps. -
6
Python 100 Days
Python - From Novice to Master in 100 Days
Python-100-Days is a comprehensive, practice-first learning roadmap by Luo Hao that spans 100 days from absolute Python basics to professional, production-grade skills. It starts with foundational syntax, control flow, data structures, and functions, then advances through object-oriented programming, file I/O, exceptions, and modules. The middle sections focus on real-world Python applications, including working with CSV, Excel, Word, PowerPoint, PDFs, images, email/SMS, and regular expressions. ... -
7
Complete-Python-3-Bootcamp
Course Files for Complete Python 3 Bootcamp Course on Udemy
...In addition, it includes applied exercises in areas such as web scraping, working with APIs, and using Python libraries like NumPy, pandas, Matplotlib, and Seaborn for data analysis and visualization. Learners can progress from beginner-friendly basics to more advanced programming skills while reinforcing their knowledge with practice problems and projects. Because it mirrors the course content, this repository is widely used by students taking the Udemy course. -
8
X-Crawl
Flexible Node.js AI-assisted crawler library
A high-performance web crawling and scraping framework for Node.js, designed for large-scale data extraction. -
9
Python Core 50 Courses
Structured learning path that organizes Python fundamentals
Python-Core-50-Courses is a structured learning path that organizes Python fundamentals into 50 digestible lessons designed for steady, incremental progress. The curriculum starts with the basics—syntax, variables, data types, and control flow—then advances to functions, modules, object-oriented programming, and common standard-library utilities. -
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
10
NiceGUI
Create web-based user interfaces with Python
NiceGUI is a Python-based UI framework that enables developers to create interactive web applications using only Python code. It abstracts away the complexities of HTML, CSS, and JavaScript, allowing for rapid development of web interfaces directly from Python scripts. NiceGUI is suitable for building dashboards, control panels, and other web-based tools, especially in contexts like robotics and data visualization. -
11
PyScript is a framework that allows users to create rich Python applications in the browser using HTML's interface and the power of Pyodide, MicroPython and WASM, and modern web technologies. PyScript is a meta project that aims to combine multiple open technologies into a framework that allows users to create sophisticated browser applications with Python. It integrates seamlessly with the way the DOM works in the browser and allows users to add Python logic in a way that feels natural both to web and Python developers.
-
12
Trame
Weave various components and technologies into a Web App
Developed by Kitware, trame is a Python-based framework that allows developers to create web applications with desktop-like functionality. It enables the integration of various components and technologies, such as VTK and ParaView, into web applications written entirely in Python. With best-in-class platforms at its core, trame provides complete control of 3D visualizations and data processing. -
13
Reflex Dev
Web apps in pure Python
Reflex is a Python framework for building full-stack web apps entirely in Python—without writing JavaScript for the frontend. It provides fast live reloads, built-in state management, deployment tooling, and optional AI-powered scaffolding to accelerate development. -
14
Frappe
Low code web framework for real world applications
Frappe is a full-stack, low-code web framework written in Python and JavaScript, used to build scalable and modular enterprise applications. It powers ERPNext and includes tools for REST APIs, user management, document modeling, workflows, and real-time updates. Frappe uses a "model-view-controller" approach with its own ORM and frontend system, enabling rapid development without sacrificing control or performance. -
15
Rio
WebApps in pure Python. No JavaScript, HTML and CSS needed
Rio is a Python framework designed to build web applications without the need for HTML, CSS, or JavaScript. Inspired by frameworks like Flutter and React, Rio offers a declarative interface and reusable components, enabling developers to create dynamic web apps entirely in Python. It streamlines the development process by managing both frontend and backend seamlessly. -
16
DrissionPage
Python based web automation tool. Powerful and elegant
DrissionPage is a Python-based automation framework that blends the capabilities of Selenium for browser automation with Requests-HTML for fast, headless web data extraction. It enables seamless switching between browser-controlled and headless HTTP sessions within the same interface. Ideal for web scraping, testing, and automation, DrissionPage is lightweight and highly efficient, offering more flexibility than standard Selenium or Requests usage alone. -
17
Bottle
bottle.py is a fast and simple micro-framework for python applications
Bottle is a minimalist web framework for building small web applications and APIs in Python. It is distributed as a single file with no external dependencies, making it perfect for rapid development, prototyping, or embedded use. Despite its small size, Bottle supports routing, templates, request handling, and plugin support, offering a full-featured toolkit in an extremely compact package. -
18
PyMySQL
MySQL client library for Python
PyMySQL is a 100% Python implementation of the MySQL client protocol, allowing Python applications to connect to MySQL and MariaDB databases without requiring binary extensions. It supports standard DB‑API 2.0 features, such as cursors, transactions, and parameterized queries. PyMySQL is versatile for web applications, scripts, and tools, offering compatibility with ORMs like SQLAlchemy and frameworks like Django. -
19
F·W·K
3D game engine/framework in C, with Luajit and Python bindings now
3D game framework in C, with Luajit and Python bindings now. -
20
dude uncomplicated data extraction
dude uncomplicated data extraction: A simple framework
Dude is a very simple framework for writing web scrapers using Python decorators. The design, inspired by Flask, was to easily build a web scraper in just a few lines of code. Dude has an easy-to-learn syntax. Dude is currently in Pre-Alpha. Please expect breaking changes. You can run your scraper from terminal/shell/command-line by supplying URLs, the output filename of your choice and the paths to your python scripts to dude scrape command. -
21
WTForms
A flexible forms validation and rendering library for Python
WTForms is a flexible forms validation and rendering library for Python web development. It can work with whatever web framework and template engine you choose. It supports data validation, CSRF protection, internationalization (I18N), and more. There are various community libraries that provide closer integration with popular frameworks. WTForms is designed to work with any web framework and template engine. -
22
Parsera
Lightweight library for scraping web-sites with LLMs
Scrape data from any website with only a link and column descriptions. Parsera is a tool designed to scrape web content, specifically handling poorly structured or messy websites. -
23
img2dataset
Easily turn large sets of image urls to an image dataset
Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine. Also supports saving captions for url+caption datasets. Opt-out directives: Websites can pass the http headers X-Robots-Tag: noai, X-Robots-Tag: noindex , X-Robots-Tag: noimageai and X-Robots-Tag: noimageindex By default img2dataset will ignore images with such headers. -
24
Starlette
The little ASGI framework that shines
Starlette is a lightweight ASGI framework/toolkit, which is ideal for building async web services in Python. It is production-ready and gives you a lightweight, low-complexity HTTP web framework. WebSocket support. In-process background tasks. Startup and shutdown events. Test client built on httpx. CORS, GZip, Static Files, streaming responses. Session and Cookie support. 100% test coverage. 100% type annotated codebase. -
25
ReactPy
It's React, but in Python
ReactPy provides a React-style component model for Python developers to build web frontends without JavaScript. Components and hooks create declarative, reactive UIs, working across frameworks like Flask, Django, Jupyter, and more. With WebAssembly support for performance, ReactPy aims to empower Python-native frontend development.
