Search Results for "python web crawler" - Page 9
Sort By:
Showing 3076 open source projects for "python web crawler"
View related business solutions-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
99.99% Uptime for MySQL and PostgreSQL DatabasesCloud SQL Enterprise Plus delivers near-zero downtime with 35 days of point-in-time recovery. Supports MySQL, PostgreSQL, and SQL Server.
-
1
Transcrypt
Python 3.7 to JavaScript compiler
...Also runs on top of node.js. Extensive documentation with many code examples. Apache 2.0 license. Pip-install and go! Seamless integration with the universe of high-quality web-oriented JavaScript libraries, rather than the desktop-oriented Python ones. -
2
Douyin TikTok Download API
Douyin TikTok Download API
Use the official interface to capture Douyin|TikTok data, support API calls, Web portals, and batch analysis. Fast, asynchronous, free, open source, ad-free, long-term maintenance. This project is based on PyWebIO , FastAPI , HTTPX , a fast and asynchronous Douyin / TikTok data crawling tool, and realizes online batch parsing and downloading of watermark-free videos or atlases through the web, data crawling API, and iOS shortcut instructions for watermark-free download and other functions.... -
3
oMLX
LLM inference server with continuous batching & SSD caching
oMLX is a local LLM inference server optimized for Apple Silicon and managed from the macOS menu bar or command line. It serves text models, vision-language models, OCR models, embeddings, and rerankers through OpenAI- and Anthropic-compatible APIs. Continuous batching allows concurrent requests, while a tiered KV cache keeps active data in RAM and moves colder blocks to SSD for reuse. Models can be pinned, unloaded manually, or evicted automatically when memory runs low. A web dashboard... -
4
Index
The SOTA Open-Source Browser Agent
Index is an open-source browser automation agent designed to autonomously perform complex tasks across websites by transforming web interfaces into programmable APIs. The system enables developers to instruct an AI agent to interact with web pages using natural language rather than traditional automation scripts. Instead of writing detailed browser automation code, users can describe the desired task and allow the agent to interpret the page structure, interact with elements, and complete... -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
5
Kitty-Tools
Kitty-Tools is a Kahoot Cheating Client for all devices and interfaces
Kitty-Tools is a multi-platform client designed to interact with Kahoot games by providing automation and enhanced control features that go beyond the standard user experience. It is built to run across a wide range of environments including desktop systems, web interfaces, and even embedded or microcontroller-based setups, emphasizing accessibility and ease of use. The project focuses on simplifying interaction with Kahoot sessions by offering tools that can automate participation,... -
6
Motor
The async Python driver for MongoDB and Tornado or asyncio
Motor is an asynchronous Python driver for MongoDB that enables developers to work with MongoDB using non-blocking I/O patterns, making it ideal for high-performance and scalable applications. Built on top of Python’s Tornado and asyncio frameworks, Motor lets you issue database operations without blocking the event loop, enabling concurrency in web servers, real-time systems, and microservices. -
7
Unsloth Studio
Unified web UI for training and running open models locally
Unsloth Studio is a web-based interface for running and training AI models locally with a unified and user-friendly experience. It allows users to work with a wide range of models for text, audio, vision, embeddings, and more without relying heavily on cloud infrastructure. Built on top of the Unsloth framework, it focuses on high-performance training with reduced VRAM usage and faster speeds compared to traditional methods. The platform supports fine-tuning, pretraining, and reinforcement... -
8
dirsearch
Web path scanner
An advanced command-line tool designed to brute force directories and files in webservers, AKA web path scanner. Wordlist is a text file, each line is a path. About extensions, unlike other tools, dirsearch only replaces the %EXT% keyword with extensions from -e flag. For wordlists without %EXT% (like SecLists), -f | --force-extensions switch is required to append extensions to every word in wordlist, as well as the /. To use multiple wordlists, you can separate your wordlists with commas.... -
9
Flask RESTX
Fully featured framework for fast, easy and documented API development
Fork of Flask-RESTPlus fully featured framework for fast, easy and documented API development with Flask. Flask-RESTX is an extension for Flask that adds support for quickly building REST APIs. Flask-RESTX encourages best practices with minimal setup. If you are familiar with Flask, Flask-RESTX should be easy to pick up. It provides a coherent collection of decorators and tools to describe your API and expose its documentation properly using Swagger. With Flask-RESTX, you only import the api... -
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
10
Agent Reach
Give your AI agent eyes to see the entire internet
Agent Reach is a command-line tool that gives AI agents access to public internet content across multiple platforms. It is designed for agent workflows that need to read and search sources like Twitter, Reddit, YouTube, GitHub, Bilibili, and XiaoHongShu without relying on paid platform APIs. The project focuses on giving AI tools broader visibility into social, video, code, and community data through one interface. It can be useful for research agents, automation workflows, competitive... -
11
Nautobot
Network Source of Truth & Network Automation Platform
Nautobot is an open-source network source of truth and automation platform designed to manage network infrastructure data effectively. Initially built as a fork of NetBox, Nautobot extends its capabilities by offering flexible data modeling, powerful REST and GraphQL APIs, and built-in automation tools. It enables network engineers and operators to store, query, and integrate network infrastructure data with external systems, making it a key component in modern network automation workflows.... -
12
OpenHands
Open-source autonomous AI software engineer
Welcome to OpenHands (formerly OpenDevin), an open-source autonomous AI software engineer who is capable of executing complex engineering tasks and collaborating actively with users on software development projects. Use AI to tackle the toil in your backlog, so you can focus on what matters: hard problems, creative challenges, and over-engineering your dotfiles We believe agentic technology is too important to be controlled by a few corporations. So we're building all our agents in the... -
13
Glances
An eye on your system
Glances is an open source, cross-platform monitoring tool that aims to provide a significant amount of monitoring information through a curses or Web-based interface. Depending on the size of the user interface, this information can then dynamically adapt. Glances can work in client/server mode, and is also capable of remote monitoring. All systems statistics can be exported to files or external time/value databases. Glances gets information from your system through various libraries,... -
14
OWASP Maryam
Modular OSINT framework for automated open-source intelligence gatheri
Maryam is an open source intelligence (OSINT) framework designed to automate the process of gathering and analyzing publicly available information from the internet. It provides a modular environment that enables users to collect data from search engines, open data sources, and various online services for reconnaissance and investigative purposes. Written in Python, Maryam is built to provide a flexible and extensible framework for harvesting information quickly and efficiently from open... -
15
Sinas
Open-source platform for building AI agents and serverless automation
Sinas is an open-source platform for building AI agents and serverless automation with fine-grained access control. It provides a self-hosted backend where developers can configure agents, connect LLM providers, write Python functions, and trigger workflows through webhooks or schedules. The platform supports isolated container execution for functions, which helps separate automation logic from the rest of the system. It also includes reusable skills, state stores, document collections, database connections, and embeddable UI components. Sinas can be managed through a web console or declarative YAML configuration, making it suitable for both interactive administration and GitOps-style workflows. ... -
16
UI UX Pro Max
AI SKILL that provide design intelligence
UI UX Pro Max is an open-source AI skill designed to provide intelligent design assistance for professional user interfaces and user experiences across web, mobile, and cross-platform frameworks. It uses an AI reasoning engine to generate complete design systems tailored to project requirements, recommending layouts, typography, colors, spacing, and component structures automatically based on natural language prompts. Users can ask for specific UI/UX tasks or design patterns, and the skill... -
17
Django Ninja
Fast, Async-ready, Openapi, type hints based framework
Django Ninja is a web framework for building APIs with Django and Python 3.6+ type hints. Designed to be easy to use and intuitive. Very high performance thanks to Pydantic and async support. Type hints and automatic docs lets you focus only on business logic. Based on the open standards for APIs: OpenAPI (previously known as Swagger) and JSON Schema. -
18
Local-NotebookLM
Googles NotebookLM but local
Local-NotebookLM is a local AI tool for turning PDF documents into generated audio content. It works like a self-hosted alternative to NotebookLM-style document-to-audio workflows. The system extracts and processes PDF text, sends the content through an LLM, and converts the result into speech with configurable voices. Users can generate podcasts, summaries, interviews, lectures, debates, tutorials, news reports, executive briefs, and other formats. It supports multiple LLM providers,... -
19
Open Notebook
An Open Source implementation of Notebook LM with more flexibility
Open Notebook is an open-source, privacy-focused alternative to Google’s Notebook LM that gives users full control over their research and AI workflows. Designed to be self-hosted, it ensures complete data sovereignty by keeping your content local or within your own infrastructure. The platform supports 16+ AI providers—including OpenAI, Anthropic, Ollama, Google, and LM Studio—allowing flexible model choice and cost optimization. Open Notebook enables users to organize and analyze... -
20
Roxy-WI
Web interface for managing Haproxy, Nginx, Apache and Keepalived
For those who need a convenient interface for managing all services in one place. Roxy-WI was created for people who want to have a fault-tolerant infrastructure, but do not want to plunge deep into the details of setting up and creating a cluster based on HAProxy, NGINX, Apache, and Keepalived. Use Roxy-WI to build a high available cluster for a couple of clicks: install HAProxy, NGINX, Apache, Keepalived, and its exporters, and carry out the initial configuration for the services. Collect... -
21
AimiliVPN
aimili-vpngate is a proxy tool that allows Linux to use clean IPs
AimiliVPN is a Python-based VPNGate proxy gateway for Linux servers. It uses the official VPNGate open protocol to fetch and connect through public VPN nodes. The project is written with the Python standard library and does not require external runtime dependencies. It includes a responsive web management interface, concurrent speed testing, route selection modes, outbound proxy behavior, and real-time logs. -
22
Style-Bert-VITS2
Style-Bert-VITS2: Bert-VITS2 with more controllable voice styles
...It takes the original Bert-VITS2 v2.1 and its Japanese-Extra variant and extends them so you can control emotion and speaking style with fine-grained intensity, not just choose a generic tone. The project targets both power users and beginners: Windows users without Git or Python can install and run it using bundled .bat scripts, while advanced users can work with virtual environments, uv, and Python tooling. It includes a full GUI editor to script dialogue, set different styles per line, edit dictionaries, and save/load projects, plus a separate web UI and Colab notebooks for training and experimentation. ... -
23
Seeker
Accurately Locate Smartphones using Social Engineering
Seeker is an open source project that demonstrates how to obtain precise location information from devices using social engineering and web-based techniques. The tool sets up a phishing page that asks for location permissions, allowing GPS and other device data to be shared if the user consents. It can capture latitude, longitude, accuracy, altitude, direction, and even speed, with results displayed in a terminal. The project supports both manual deployment and tunneling services like Ngrok... -
24
Ulauncher
Feature rich application Launcher for Linux
Type in an application name without worrying about spelling. Ulauncher will figure out what you meant. It also remembers your previous choices and automatically selects the best option for you. Ulauncher provides 4 themes built in. But if you need something different you can always create a custom color theme. Improve your workflow with customizable shortcuts and extensions. Create a shortcut for web search or your scripts or install a 3rd party extension. -
25
openvpn-monitor
openvpn-monitor is a web based OpenVPN monitor
openvpn-monitor is a simple Python program to generate HTML that displays the status of an OpenVPN server, including all current connections. It uses the OpenVPN management console. It typically runs on the same host as the OpenVPN server, however, it does not necessarily need to. OpenVPN-monitor is a web-based OpenVPN monitor, that shows current connection information, such as users, location, and data transferred.
