Search Results for "python web crawler" - Page 3
Sort By:
Showing 3141 open source projects for "python web crawler"
View related business solutions-
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
MongoDB Atlas | Run databases anywhereMongoDB Atlas allows you to build and run modern applications across 125+ cloud regions, spanning AWS, Azure, and Google Cloud. Its multi-cloud clusters enable seamless data distribution and automated failover between cloud providers, ensuring high availability and flexibility without added complexity.
-
1
Flask-SQLAlchemy
Adds SQLAlchemy support to Flask
Flask-SQLAlchemy is an extension for Flask that adds support for SQLAlchemy to your application. It simplifies using SQLAlchemy with Flask by setting up common objects and patterns for using those objects, such as a session tied to each web request, models, and engines. Flask-SQLAlchemy does not change how SQLAlchemy works or is used. See the SQLAlchemy documentation to learn how to work with the ORM in depth. The documentation here will only cover setting up the extension, not how to use... -
2
AWS Toolkit for Visual Studio Code
Local Lambda debug, CodeWhisperer, SAM/CFN syntax, etc.
The AWS Toolkit extension for Visual Studio Code enables you to interact with Amazon Web Services (AWS). Try the AWS Code Sample Catalog to start coding with the AWS SDK. The AWS Explorer provides access to the AWS services that you can work with when using the Toolkit. To see the AWS Explorer, choose the AWS icon in the Activity bar. The Developer Tools panel is a section for developer-focused tooling curated for working in an IDE. The Developer Tools panel can be found underneath the AWS... -
3
Flet
Flet enables developers to easily build realtime web and mobile apps
Flet enables developers to easily build real-time web, mobile and desktop apps in Python. No front-end experience is required. An internal tool or a dashboard for your team, weekend project, data entry form, kiosk app or high-fidelity prototype - Flet is an ideal framework to quickly hack great-looking interactive apps to serve a group of users. No more complex architecture with JavaScript frontend, REST API backend, database, cache, etc. With Flet you just write a monolith stateful app... -
4
Scrapy
A fast, high-level web crawling and web scraping framework
Scrapy is a fast, open source, high-level framework for crawling websites and extracting structured data from these websites. Portable and written in Python, it can run on Windows, Linux, macOS and BSD. Scrapy is powerful, fast and simple, and also easily extensible. Simply write the rules to extract the data, and add new functionality if you wish without having to touch the core. Scrapy does the rest, and can be used in a number of applications. It can be used for data mining, monitoring... -
Picsart Enterprise Background Removal API for Stunning eCommerce VisualsWith our Remove Background API tool, you can access the transformative capabilities of automation , which will allow you to turn any photo asset into compelling product imagery. With elevated visuals quality on your digital platforms, you can captivate your audience, and therefore achieve higher engagement and sales.
-
5
MoneyPrinterTurbo
Generate short videos with one click using AI LLM
MoneyPrinterTurbo is an AI-driven tool that enables users to generate high-definition short videos with minimal input. By providing a topic or keyword, the system automatically creates video scripts, sources relevant media assets, adds subtitles, and incorporates background music, resulting in a polished video ready for distribution. -
6
Crawlab
Distributed web crawler admin platform for spiders management
Golang-based distributed web crawler management platform, supporting various languages including Python, NodeJS, Go, Java, PHP and various web crawler frameworks including Scrapy, Puppeteer, Selenium. Please use docker-compose to one-click to start up. By doing so, you don't even have to configure MongoDB database. The frontend app interacts with the master node, which communicates with other components such as MongoDB, SeaweedFS and worker nodes. Master node and worker nodes communicate... -
7
pyTelegramBotAPI
Python Telegram bot api.
TeleBot is the synchronous and asynchronous implementation of Telegram Bot API. -
8
miniblink49
Lighter, faster browser kernel of blink to integrate HTML UI in apps
... electron). Customize as you wish, simulate another browser environment. Perfect HTML5 support, friendly to various front-end libraries (support HTML5, and friendly to front framework). After turning off the cross-domain switch, you can use various cross-domain functions (support cross-domain). Headless mode, which greatly saves resources and is suitable for crawlers (headless mode, be suitable for Web Crawler). -
9
Agent Development Kit (ADK)
Open-source, code-first Python toolkit for building, evaluating, etc.
ADK (Android Device Key) Python is a reference implementation by Google for working with Android attestation keys in Python. It facilitates the integration of Android attestation features into backends or systems that require verification of device identity and integrity. This is especially important in high-security applications where verifying that a device is genuine and uncompromised is critical. ADK Python helps developers verify hardware-backed keys, work with JSON Web Tokens (JWT... -
No-Nonsense Code-to-Cloud Security for Devs | AikidoAikido provides a unified security platform for developers, combining 12 powerful scans like SAST, DAST, and CSPM. AI-driven AutoFix and AutoTriage streamline vulnerability management, while runtime protection blocks attacks.
-
10
Browser Use
Make websites accessible for AI agents
Browser-Use is a framework that makes websites accessible for AI agents, enabling automated interactions and data extraction from web pages. -
11
QR Code generator library
High-quality QR Code generator library in Java, TypeScript/JavaScript
... to TypeScript, Python, Rust, C++, and C. It is open source under the MIT License. For each language, the codebase is roughly 1000 lines of code and has no dependencies other than the respective language’s standard library. -
12
Pyxel
A retro game engine for Python
A retro game engine for Python. Thanks to its simple specifications inspired by retro gaming consoles, such as only 16 colors can be displayed and only 4 sounds can be played back at the same time, you can feel free to enjoy making pixel art style games. The motivation for the development of Pyxel is the feedback from users. Please give Pyxel a star on GitHub! Pyxel's specifications and APIs are inspired by PICO-8 and TIC-80. Pyxel is open source and free to use. Let's start making a retro game... -
13
OpenHands
Open-source autonomous AI software engineer
... in the open on GitHub, under the MIT license. Our agents can do anything a human developer can: they write code, run commands, and use the web. We're partnering with AI safety experts like Invariant Labs to balance innovation with security. -
14
Wifipumpkin3
Powerful framework for rogue access point attack
wifipumpkin3 is powerful framework for rogue access point attack, written in Python, that allow and offer to security researchers, red teamers and reverse engineers to mount a wireless network to conduct a man-in-the-middle attack. -
15
Frappe
Low code web framework for real world applications
Frappe is a full-stack, low-code web framework written in Python and JavaScript, used to build scalable and modular enterprise applications. It powers ERPNext and includes tools for REST APIs, user management, document modeling, workflows, and real-time updates. Frappe uses a "model-view-controller" approach with its own ORM and frontend system, enabling rapid development without sacrificing control or performance. -
16
Mail-in-a-Box
Take back control of your email with this mail server in a box
... a good mail server easy, promote decentralization, innovation, and privacy on the web, have automated, auditable, and idempotent configuration, not make a totally unhackable, NSA-proof server, and not make something customizable by power users. Mail-in-a-Box turns a fresh Ubuntu 18.04 LTS 64-bit machine into a working mail server by installing and configuring various components. It is a one-click email appliance. There are no user-configurable setup options. It "just works." -
17
Snoop Project
This is the most powerful software taking into account CIS location
Snoop is an open data intelligence tool (OSINT world). Snoop Project is one of the most promising OSINT tools for finding nicknames. This is the most powerful software taking into account the CIS location. Is your life slideshow? Ask Snoop. Snoop project is developed without taking into account the opinions of the NSA and their friends, that is, it is available to the average user. Snoop is a research work (own database / closed bugbounty) in the field of searching and processing public data... -
18
HAXE
The cross-platform toolkit
Haxe is an open source high-level strictly-typed programming language with a fast optimizing cross-compiler. Haxe can build cross-platform applications targeting JavaScript, C++, C#, Java, JVM, Python, Lua, PHP, Flash, and allows access to each platform's native capabilities. Haxe has its own VMs (HashLink and NekoVM) but can also run in interpreted mode. Haxe is useful in a wide variety of domains; games, web, mobile, desktop, command-line and cross-platform APIs. Take a look at who is using... -
19
Stable Diffusion WebUI
Web interface for generating images using Stable Diffusion models
This project provides a powerful web-based interface for running Stable Diffusion, a text-to-image generation model. Developed by AUTOMATIC1111, it supports numerous features like model customization, prompt history, image upscaling, inpainting, and batch processing. The WebUI is beginner-friendly yet powerful enough for advanced users, becoming one of the most popular community-run UIs for AI image generation. -
20
WebMagic
A scalable web crawler framework for Java
WebMagic is a scalable crawler framework. It covers the whole lifecycle of crawler, downloading, url management, content extraction and persistent. It can simplify the development of a specific crawler. WebMagic is a simple but scalable crawler framework. You can develop a crawler easily based on it. WebMagic has a simple core with high flexibility, a simple API for html extracting. It also provides annotation with POJO to customize a crawler, and no configuration is needed. Some other features... -
21
Code App
Building a full-fledged code editor for iPad
...) Local Python Runtime. Local Clang compiler. Git Version Control. Package manager support (Pip and NPM) and Remote connection support (Files and terminal). While we want to make the editing experience as close as a desktop offers, Code App is still bounded by iOS's limitations. For example, you cannot download arbitrary commands or modules with native components. Spawning subprocesses is also not possible. -
22
FreeTAKServer
Situational Awareness Server compatible with TAK clients
FTS is a Python3 implementation of a TAK Server for devices like ATAK, WinTAK, and ITAK, it is cross-platform and runs from a multi-node installation on AWS down to the Android edition. It's free and open source (released under the Eclipse Public License. FTS allows you to connect ATAK clients to share geo-information, to chat with all the connected clients, exchange files and more. It intends to support all the major use cases of the original TAK server. -
23
Locust
Scalable open source load testing tool
Locust is an open source user load testing tool written in Python. The idea behind Locust is to swarm your web site or other systems with attacks from simulated users during a test, with each user behavior defined by you using Python code. This swarming process is then monitored from a web UI in real-time, and will help identify any bottlenecks in your code before real users can come in. As it is completely event-based, Locust can have thousands or even millions of simultaneous users... -
24
PyMySQL
MySQL client library for Python
PyMySQL is a 100% Python implementation of the MySQL client protocol, allowing Python applications to connect to MySQL and MariaDB databases without requiring binary extensions. It supports standard DB‑API 2.0 features, such as cursors, transactions, and parameterized queries. PyMySQL is versatile for web applications, scripts, and tools, offering compatibility with ORMs like SQLAlchemy and frameworks like Django. -
25
Glances
An eye on your system
Glances is an open source, cross-platform monitoring tool that aims to provide a significant amount of monitoring information through a curses or Web-based interface. Depending on the size of the user interface, this information can then dynamically adapt. Glances can work in client/server mode, and is also capable of remote monitoring. All systems statistics can be exported to files or external time/value databases. Glances gets information from your system through various libraries...
