Search Results for "python web crawler" - Page 8
Sort By:
Showing 2674 open source projects for "python web crawler"
View related business solutions-
Gen AI apps are built with MongoDB AtlasMongoDB Atlas provides built-in vector search and a flexible document model so developers can build, scale, and run gen AI apps without stitching together multiple databases. From LLM integration to semantic search, Atlas simplifies your AI architecture—and it’s free to get started.
-
Get the most trusted enterprise browserDefend against security incidents with Chrome Enterprise. Create customizable controls, manage extensions and set proactive alerts to keep your data and employees protected without slowing down productivity.
-
1
Django Ninja
Fast, Async-ready, Openapi, type hints based framework
Django Ninja is a web framework for building APIs with Django and Python 3.6+ type hints. Designed to be easy to use and intuitive. Very high performance thanks to Pydantic and async support. Type hints and automatic docs lets you focus only on business logic. Based on the open standards for APIs: OpenAPI (previously known as Swagger) and JSON Schema. Django friendly (obviously) has good integration with the Django core and ORM. Used by multiple companies on live projects. -
2
Tarsier
Vision utilities for web interaction agents
At Reworkd, we iterated on all these problems across tens of thousands of real web tasks to build a powerful perception system for web agents... Tarsier! In the video below, we use Tarsier to provide webpage perception for a minimalistic GPT-4 LangChain web agent. Tarsier visually tags interactable elements on a page via brackets + an ID e.g. [23]. In doing this, we provide a mapping between elements and IDs for an LLM to take actions upon (e.g. CLICK [23]). We define interactable elements... -
3
img2dataset
Easily turn large sets of image urls to an image dataset
Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine. Also supports saving captions for url+caption datasets. Opt-out directives: Websites can pass the http headers X-Robots-Tag: noai, X-Robots-Tag: noindex , X-Robots-Tag: noimageai and X-Robots-Tag: noimageindex By default img2dataset will ignore images with such headers. -
4
dude uncomplicated data extraction
dude uncomplicated data extraction: A simple framework
Dude is a very simple framework for writing web scrapers using Python decorators. The design, inspired by Flask, was to easily build a web scraper in just a few lines of code. Dude has an easy-to-learn syntax. Dude is currently in Pre-Alpha. Please expect breaking changes. You can run your scraper from terminal/shell/command-line by supplying URLs, the output filename of your choice and the paths to your python scripts to dude scrape command. -
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
5
Django OAuth Toolkit
OAuth2 goodies for the Djangonauts!
... Python Web framework that encourages rapid development and clean, pragmatic design. Built by experienced developers, it takes care of much of the hassle of Web development, so you can focus on writing your app without needing to reinvent the wheel. Your Django app exposes a web API you want to protect with OAuth2 authentication. You need to implement an OAuth2 authorization server to provide tokens management for your infrastructure. -
6
Mopidy
Mopidy is an extensible music server written in Python
Mopidy plays music from local disk, Spotify, SoundCloud, TuneIn, and more. You can edit the playlist from any phone, tablet, or computer using a variety of MPD and web clients. Vanilla Mopidy only plays music from files and radio streams. Through extensions, Mopidy can play music from cloud services like Spotify, SoundCloud, and TuneIn. With Mopidy's extension support, you can easily add backends for new music sources. Mopidy is a Python application that runs in a terminal or in the background... -
7
Helium
Lighter web automation with Python
Helium is a Python library built on top of Selenium to make browser automation more intuitive and human-friendly. It replaces verbose boilerplate code with natural language-like API calls such as click("Login") or write("hello", into="Name"). Helium manages browser setup, waits, and teardown, enabling quick development of scripts for testing, scraping, or task automation without requiring deep Selenium knowledge. -
8
Tortoise ORM
Familiar asyncio ORM for python, built with relations in mind
Tortoise ORM is an easy-to-use asyncio ORM (Object Relational Mapper) for Python, inspired by Django's ORM. It is designed to work with asynchronous frameworks, providing a simple and familiar API for interacting with databases. Tortoise ORM supports various relational databases and is suitable for building high-performance web applications. -
9
OAuthLib
A generic, spec-compliant, thorough implementation of the OAuth
A generic, spec-compliant, thorough implementation of the OAuth request-signing logic for Python 3.8+. OAuthLib is a framework which implements the logic of OAuth1 or OAuth2 without assuming a specific HTTP request object or web framework. Use it to graft OAuth client support onto your favorite HTTP library, or provide support onto your favourite web framework. If you're a maintainer of such a library, write a thin veneer on top of OAuthLib and get OAuth support for very little effort. -
Secure remote access solution to your private network, in the cloud or on-prem.OpenVPN is here to bring simple, flexible, and cost-effective secure remote access to companies of all sizes, regardless of where their resources are located.
-
10
Taipy
Turns Data and AI algorithms into production-ready web applications
From simple pilots to production-ready web applications in no time. No more compromise on performance, customization, and scalability. Taipy enhances performance with caching control of graphical events, optimizing rendering by selectively updating graphical components only upon interaction. Effortlessly manage massive datasets with Taipy's built-in decimator for charts, intelligently reducing the number of data points to save time and memory without losing the essence of your data's shape... -
11
Mara Pipelines
A lightweight opinionated ETL framework, halfway between plain scripts
This package contains a lightweight data transformation framework with a focus on transparency and complexity reduction. Data integration pipelines as code: pipelines, tasks and commands are created using declarative Python code. PostgreSQL as a data processing engine. Extensive web ui. The web browser as the main tool for inspecting, running and debugging pipelines. GNU make semantics. Nodes depend on the completion of upstream nodes. No data dependencies or data flows. No in-app data... -
12
Basketball Reference
NBA Stats API via Basketball Reference
Basketball Reference is a great site (especially for a basketball stats nut like me), and hopefully, they don't get too pissed off at me for creating this. I initially wrote this library as an exercise for creating my first PyPi package, hope you find it valuable! This library was created for another Python project where I was trying to estimate an NBA player's productivity. A lot of sports-related APIs are expensive - luckily, Basketball Reference provides a free service which can be scraped... -
13
Auto Bangumi
Automated Bangumi episode downloader and organizer with Web UI
Auto_Bangumi is a fully automated tool for downloading, organizing, and tracking anime (Bangumi) episodes using RSS feeds and download clients like qBittorrent. It offers a modern Web UI for managing subscriptions, custom filtering rules, automatic file renaming, and subtitle matching. Designed for anime fans, it streamlines the process of staying up-to-date with seasonal shows by integrating feed parsing, downloading, and library organization. -
14
AGiXT
AGiXT is a dynamic AI Automation Platform
AGiXT is a dynamic Artificial Intelligence Automation Platform engineered to orchestrate efficient AI instruction management and task execution across a multitude of providers. Our solution infuses adaptive memory handling with a broad spectrum of commands to enhance AI's understanding and responsiveness, leading to improved task completion. The platform's smart features, like Smart Instruct and Smart Chat, seamlessly integrate web search, planning strategies, and conversation continuity... -
15
TurboGears
Python web framework with full-stack layer
TurboGears is a hybrid web framework able to act both as a Full Stack framework or as a Microframework. TurboGears helps you get going fast and gets out of your way when you want it! TurboGears can be used both as a full stack framework or as a microframework in single-file mode. TurboGears 2 is built on top of the experience of several next-generation web frameworks including TurboGears 1 (of course), Django, and Rails. All of these frameworks had limitations that frustrated us, and TG2... -
16
WTForms
A flexible forms validation and rendering library for Python
WTForms is a flexible forms validation and rendering library for Python web development. It can work with whatever web framework and template engine you choose. It supports data validation, CSRF protection, internationalization (I18N), and more. There are various community libraries that provide closer integration with popular frameworks. WTForms is designed to work with any web framework and template engine. There are a number of community-provided libraries that make integrating... -
17
Flask-JWT-Extended
An open source Flask extension that provides JWT support
Flask-JWT-Extended not only adds support for using JSON Web Tokens (JWT) to Flask for protecting routes, but also many helpful (and optional) features built in to make working with JSON Web Tokens easier. Adding custom claims to JSON Web Tokens. Automatic user loading (current_user). Custom claims validation on received tokens. Refresh tokens, first-class support for fresh tokens for making sensitive changes. Token revoking/blocklisting. Storing tokens in cookies and CSRF protection. Adding... -
18
FastHX
FastAPI server-side rendering with built-in HTMX support.
FastHX is a high-performance HTTP and WebSocket server framework designed for Haxe, enabling fast and scalable web application development. -
19
langrocks
Tools like web browser, computer access and code runner for LLMs
Langrocks is a programming language experimentation toolkit that enables developers to create, test, and optimize custom programming languages. -
20
Miracode
A sharp, readable, vector-y version of Monocraft
A web-based code editor with support for multiple programming languages, designed for collaborative projects. -
21
Connexion
Swagger/OpenAPI First framework for Python on top of Flask
Connexion is a framework on top of Flask that automagically handles HTTP requests defined using OpenAPI (formerly known as Swagger), supporting both v2.0 and v3.0 of the specification. Connexion allows you to write these specifications, then maps the endpoints to your Python functions. This is what makes it unique from other tools that generate the specification based on your Python code. You are free to describe your REST API with as much detail as you want and then Connexion guarantees... -
22
Masonite
The Modern And Developer Centric Python Web Framework
Stop using old frameworks with just a few confusing features. Masonite is the developer-focused dev tool with all the features you need for the rapid development you deserve. Masonite is perfect for beginners getting their first web app deployed or advanced developers and businesses that need to reach for the full fleet of features available. Mail support for sending emails quickly. Queue support to speed your application up by sending jobs to run on a queue or asynchronously. Notifications... -
23
Peewee-async
Asynchronous interface for peewee ORM powered by asyncio
peewee-async is an async extension for the Peewee ORM, enabling non-blocking database access in asyncio-powered Python applications. It allows you to use familiar Peewee models while benefiting from asynchronous I/O, making it ideal for web applications or services that require concurrency. -
24
Piccolo
A fast, user friendly ORM and query builder which supports asyncio
Piccolo is a modern, fast, and type-safe ORM for Python, designed with developer ergonomics in mind. It provides a clean syntax for defining schemas and building queries while supporting both sync and async execution. With built-in admin tools and rich introspection, Piccolo is suitable for web apps, APIs, and small-to-medium scale backends that prioritize clarity and speed. -
25
Awesome Free ChatGPT
List of free ChatGPT mirror sites, continuously updated
This is a curated directory of freely accessible ChatGPT-style services and mirror sites that offer AI chatbot interfaces without login or payment requirements. Resources often support multiple models like GPT-4, Claude, Gemini, and more. Data collected from multiple independent sites with descriptions and tags. Includes services with image upload and drawing capabilities. Aggregates free, no-login-required ChatGPT-like web services. Continually updated mirror list to maintain availability.
