Showing 669 open source projects for "python web crawler"

View related business solutions
  • Simple, Secure Domain Registration Icon
    Simple, Secure Domain Registration

    Get your domain at wholesale price. Cloudflare offers simple, secure registration with no markups, plus free DNS, CDN, and SSL integration.

    Register or renew your domain and pay only what we pay. No markups, hidden fees, or surprise add-ons. Choose from over 400 TLDs (.com, .ai, .dev). Every domain is integrated with Cloudflare's industry-leading DNS, CDN, and free SSL to make your site faster and more secure. Simple, secure, at-cost domain registration.
    Sign up for free
  • Our Free Plans just got better! | Auth0 Icon
    Our Free Plans just got better! | Auth0

    With up to 25k MAUs and unlimited Okta connections, our Free Plan lets you focus on what you do best—building great apps.

    You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
    Try free now
  • 1
    AWS X-Ray SDK for Python

    AWS X-Ray SDK for Python

    AWS X-Ray SDK for the Python programming language

    AWS X-Ray SDK for the Python programming language. The AWS X-Ray SDK for Python is compatible with Python 2.7, 3.4, 3.5, 3.6, 3.7, 3.8, and 3.9. X-Ray Python SDK will by default generate no-op trace and entity id for unsampled requests and secure random trace and entity id for sampled requests. If customer wants to enable generating secure random trace and entity id for all the (sampled/unsampled) requests (this is applicable for trace id injection into logs use case) then they should set the AWS_XRAY_NOOP_ID environment variable as False. ...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 2
    Trame

    Trame

    Weave various components and technologies into a Web App

    Developed by Kitware, trame is a Python-based framework that allows developers to create web applications with desktop-like functionality. It enables the integration of various components and technologies, such as VTK and ParaView, into web applications written entirely in Python. With best-in-class platforms at its core, trame provides complete control of 3D visualizations and data processing.
    Downloads: 1 This Week
    Last Update:
    See Project
  • 3
    BlackSheep

    BlackSheep

    Fast ASGI web framework for Python

    BlackSheep is an asynchronous web framework to build event-based web applications with Python. A rich code API, based on dependency injection and inspired by Flask and ASP.NET Core. A typing-friendly codebase, which enables a comfortable development experience thanks to hints when coding with IDEs. Built-in generation of OpenAPI Documentation, supporting version 3, YAML, and JSON.
    Downloads: 1 This Week
    Last Update:
    See Project
  • 4
    Textual

    Textual

    Textual is a TUI (Text User Interface) framework for Python

    Textual is a Python framework for creating interactive applications that run in your terminal. Textual adds interactivity to Rich with a Python API inspired by modern web development. On modern terminal software (installed by default on most systems), Textual apps can use 16.7 million colors with mouse support and smooth flicker-free animation. A powerful layout engine and re-usable components makes it possible to build apps that rival the desktop and web experience. ...
    Downloads: 5 This Week
    Last Update:
    See Project
  • Gen AI apps are built with MongoDB Atlas Icon
    Gen AI apps are built with MongoDB Atlas

    Build gen AI apps with an all-in-one modern database: MongoDB Atlas

    MongoDB Atlas provides built-in vector search and a flexible document model so developers can build, scale, and run gen AI apps without stitching together multiple databases. From LLM integration to semantic search, Atlas simplifies your AI architecture—and it’s free to get started.
    Start Free
  • 5
    Scrapy

    Scrapy

    A fast, high-level web crawling and web scraping framework

    Scrapy is a fast, open source, high-level framework for crawling websites and extracting structured data from these websites. Portable and written in Python, it can run on Windows, Linux, macOS and BSD. Scrapy is powerful, fast and simple, and also easily extensible. Simply write the rules to extract the data, and add new functionality if you wish without having to touch the core. Scrapy does the rest, and can be used in a number of applications. It can be used for data mining, monitoring...
    Downloads: 17 This Week
    Last Update:
    See Project
  • 6
    Eel

    Eel

    A Python library for making simple Electron-like HTML/JS GUI apps

    ...If you are familiar with Python and web development, probably just jump to this example which picks random file names out of the given folder (something that is impossible from a browser). There are several options for making GUI apps in Python, but if you want to use HTML/JS (in order to use jQueryUI or Bootstrap, for example) then you generally have to write a lot of boilerplate code to communicate from the Client (Javascript) side to the Server (Python) side.
    Downloads: 5 This Week
    Last Update:
    See Project
  • 7
    F·W·K

    F·W·K

    3D game engine/framework in C, with Luajit and Python bindings now

    3D game framework in C, with Luajit and Python bindings now.
    Downloads: 43 This Week
    Last Update:
    See Project
  • 8
    I hate money

    I hate money

    A simple shared budget manager web application

    I hate money is a web application made to ease shared budget management. It keeps track of who bought what, when, and for whom; and helps to settle the bills. I hate money is written in python, using the flask framework. It’s developed with ease of use in mind and is trying to keep things simple. Hope you (will) like it! The code is distributed under a BSD beerware derivative: if you meet the people in person and you want to pay them a craft beer, you are highly encouraged to do so.
    Downloads: 3 This Week
    Last Update:
    See Project
  • 9
    Locust

    Locust

    Scalable open source load testing tool

    Locust is an open source user load testing tool written in Python. The idea behind Locust is to swarm your web site or other systems with attacks from simulated users during a test, with each user behavior defined by you using Python code. This swarming process is then monitored from a web UI in real-time, and will help identify any bottlenecks in your code before real users can come in. As it is completely event-based, Locust can have thousands or even millions of simultaneous users distributed over multiple machines swarming your system. ...
    Downloads: 4 This Week
    Last Update:
    See Project
  • Connect every part of your business to one bank account Icon
    Connect every part of your business to one bank account

    North One is a business banking app that integrates cash flow, payments, and budgeting to turn your North One Account into one Connected Bank Account

    North One is proudly built for small businesses, startups and freelancers across America. Make payments easily, keep tabs on your money and put your finances on autopilot through smart integrations with the tools you’re already using. North One was built to make managing money easy so you can focus on running your business. No more branches. No more lines. No more paperwork. Get complete access to your North One Account from your phone or computer, wherever your business takes you. Create Envelopes for taxes, payroll, rent, and anything else automatically.
    Get started for free.
  • 10
    Flet

    Flet

    Flet enables developers to easily build realtime web and mobile apps

    ...With Flet you just write a monolith stateful app in Python only and get a multi-user, real-time Single-Page Application (SPA). To start developing with Flet, you just need your favorite IDE or text editor. With no SDKs, no thousands of dependencies, no complex tooling, Flet has a built-in web server with assets hosting and desktop clients.
    Downloads: 10 This Week
    Last Update:
    See Project
  • 11
    Dash

    Dash

    Build beautiful web-based analytic apps, no JavaScript required

    Dash is a Python framework for building beautiful analytical web applications without any JavaScript. Built on top of Plotly.js, React and Flask, Dash easily achieves what an entire team of designers and engineers normally would. It ties modern UI controls and displays such as dropdown menus, sliders and graphs directly to your analytical Python code, and creates exceptional, interactive analytics apps.
    Downloads: 5 This Week
    Last Update:
    See Project
  • 12
    DrissionPage

    DrissionPage

    Python based web automation tool. Powerful and elegant

    DrissionPage is a Python-based automation framework that blends the capabilities of Selenium for browser automation with Requests-HTML for fast, headless web data extraction. It enables seamless switching between browser-controlled and headless HTTP sessions within the same interface. Ideal for web scraping, testing, and automation, DrissionPage is lightweight and highly efficient, offering more flexibility than standard Selenium or Requests usage alone.
    Downloads: 2 This Week
    Last Update:
    See Project
  • 13
    Sanic

    Sanic

    Async Python 3.6+ web server/framework

    Build fast, run fast with Sanic! Sanic is a Python 3.6+ web server and web framework designed to go fast. It provides a way to get a highly performant HTTP server up and running fast, while also making it easy to build, expand, and eventually scale. Sanic aspires to be as simple as possible while delivering the performance that you require. It allows the usage of the async/await syntax added in Python 3.5, so your code is guaranteed to be non-blocking and speedy. ...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 14
    Reflex Dev

    Reflex Dev

    Web apps in pure Python

    Reflex is a Python framework for building full-stack web apps entirely in Python—without writing JavaScript for the frontend. It provides fast live reloads, built-in state management, deployment tooling, and optional AI-powered scaffolding to accelerate development.
    Downloads: 1 This Week
    Last Update:
    See Project
  • 15
    Rio

    Rio

    WebApps in pure Python. No JavaScript, HTML and CSS needed

    Rio is a Python framework designed to build web applications without the need for HTML, CSS, or JavaScript. Inspired by frameworks like Flutter and React, Rio offers a declarative interface and reusable components, enabling developers to create dynamic web apps entirely in Python. It streamlines the development process by managing both frontend and backend seamlessly.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 16
    Bottle

    Bottle

    bottle.py is a fast and simple micro-framework for python applications

    Bottle is a minimalist web framework for building small web applications and APIs in Python. It is distributed as a single file with no external dependencies, making it perfect for rapid development, prototyping, or embedded use. Despite its small size, Bottle supports routing, templates, request handling, and plugin support, offering a full-featured toolkit in an extremely compact package.
    Downloads: 2 This Week
    Last Update:
    See Project
  • 17
    PyMySQL

    PyMySQL

    MySQL client library for Python

    PyMySQL is a 100% Python implementation of the MySQL client protocol, allowing Python applications to connect to MySQL and MariaDB databases without requiring binary extensions. It supports standard DB‑API 2.0 features, such as cursors, transactions, and parameterized queries. PyMySQL is versatile for web applications, scripts, and tools, offering compatibility with ORMs like SQLAlchemy and frameworks like Django.
    Downloads: 3 This Week
    Last Update:
    See Project
  • 18
    Wifipumpkin3

    Wifipumpkin3

    Powerful framework for rogue access point attack

    wifipumpkin3 is powerful framework for rogue access point attack, written in Python, that allow and offer to security researchers, red teamers and reverse engineers to mount a wireless network to conduct a man-in-the-middle attack.
    Downloads: 14 This Week
    Last Update:
    See Project
  • 19
    Grab Framework Project

    Grab Framework Project

    Web Scraping Framework

    Grab is a python framework for building web scrapers. With Grab you can build web scrapers of various complexity, from simple 5-line scripts to complex asynchronous website crawlers processing millions of web pages. Grab provides an API for performing network requests and for handling the received content e.g. interacting with DOM tree of the HTML document.
    Downloads: 1 This Week
    Last Update:
    See Project
  • 20
    SiteOne Crawler (desktop app)

    SiteOne Crawler (desktop app)

    A free, feature-rich web analyzer and exporter/cloner you will love!

    A free in-depth website analyzer providing audits of security, performance, SEO, accessibility and other technical aspects. Available as a desktop application for Windows/macOS/Linux and as a CLI tool for advanced users and CI/CD processes. It also includes an offline web page exporter (website clone, mirror).
    Downloads: 4 This Week
    Last Update:
    See Project
  • 21
    Frappe

    Frappe

    Low code web framework for real world applications

    Frappe is a full-stack, low-code web framework written in Python and JavaScript, used to build scalable and modular enterprise applications. It powers ERPNext and includes tools for REST APIs, user management, document modeling, workflows, and real-time updates. Frappe uses a "model-view-controller" approach with its own ORM and frontend system, enabling rapid development without sacrificing control or performance.
    Downloads: 2 This Week
    Last Update:
    See Project
  • 22
    Starlette

    Starlette

    The little ASGI framework that shines

    Starlette is a lightweight ASGI framework/toolkit, which is ideal for building async web services in Python. It is production-ready and gives you a lightweight, low-complexity HTTP web framework. WebSocket support. In-process background tasks. Startup and shutdown events. Test client built on httpx. CORS, GZip, Static Files, streaming responses. Session and Cookie support. 100% test coverage. 100% type annotated codebase.
    Downloads: 2 This Week
    Last Update:
    See Project
  • 23
    miniblink49

    miniblink49

    Lighter, faster browser kernel of blink to integrate HTML UI in apps

    ...After turning off the cross-domain switch, you can use various cross-domain functions (support cross-domain). Headless mode, which greatly saves resources and is suitable for crawlers (headless mode, be suitable for Web Crawler).
    Downloads: 3 This Week
    Last Update:
    See Project
  • 24
    WeasyPrint

    WeasyPrint

    The awesome document factory

    WeasyPrint is a smart solution helping people to create PDF documents. You can generate gorgeous statistical reports, invoices, tickets, and anything you want as long as you have some webdesign skills! Design your documents just as you design your websites! WeasyPrint follows the widely used HTML and CSS specifications from the W3C. You can use your usual web tools, languages and frameworks, but for print. Creating high-quality digital documents requires features that you love to use as...
    Downloads: 22 This Week
    Last Update:
    See Project
  • 25
    django-helpdesk

    django-helpdesk

    A Django application to manage tickets for an internal helpdesk

    A Django application to manage tickets for an internal helpdesk. Formerly known as Jutda Helpdesk. django-helpdesk was formerly known as Jutda Helpdesk, named after the company which originally created it. As of January 2011 the name has been changed to reflect what it really is: a Django-powered ticket tracker with contributors reaching far beyond Jutda. django-helpdesk includes a basic demo Django project so that you may easily get started with testing or developing django-helpdesk. The...
    Downloads: 16 This Week
    Last Update:
    See Project