Search Results for "python web crawler"
Sort By:
Showing 158 open source projects for "python web crawler"
View related business solutions-
Endpoint Management with Remote Control Built InSplashtop AEM handles patching, alerts, scripts, and inventory, and unattended remote control is included. When something needs hands-on work, you’re one click away instead of switching tools.
-
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
1
sqlite-web
Web-based SQLite database browser written in Python
sqlite-web is a web-based SQLite database browser written in Python. It lets users open existing SQLite databases or create new ones through a browser interface. The tool can browse table data, inspect database structure, and run arbitrary SQL queries from a dedicated query tab. It also supports common editing actions, including inserting, updating, and deleting rows. -
2
Motor
The async Python driver for MongoDB and Tornado or asyncio
Motor is an asynchronous Python driver for MongoDB that enables developers to work with MongoDB using non-blocking I/O patterns, making it ideal for high-performance and scalable applications. Built on top of Python’s Tornado and asyncio frameworks, Motor lets you issue database operations without blocking the event loop, enabling concurrency in web servers, real-time systems, and microservices. -
3
pagodo
Automate Google Hacking Database scraping and searching
pagodo automates Google searching for potentially vulnerable web pages and applications on the Internet. It replaces manually performing Google dork searches with a web GUI browser. There are 2 parts. The first is ghdb_scraper.py that retrieves the latest Google dorks and the second portion is pagodo.py that leverages the information gathered by ghdb_scraper.py. This version of pagodo also supports native HTTP(S) and SOCKS5 application support, so no more wrapping it in a tool like... -
4
Advanced Alchemy
A carefully crafted, thoroughly tested, optimized companion library
advanced-alchemy is an opinionated ORM toolkit built on SQLAlchemy and designed for integration with the Litestar web framework. It simplifies common ORM patterns such as CRUD, pagination, and async support while providing extensibility and best practices out of the box. It’s designed to accelerate backend development for modern Python web apps. -
Save Up to 91% on Cloud Compute With Spot VMsRun batch jobs at 60-91% off with Spot VMs. Long-running workloads get automatic discounts with sustained use.
-
5
Cinemagoer
Python package useful to retrieve and manage the data of IMDb
Cinemagoer (previously known as IMDbPY) is a Python package for retrieving and managing the data of the IMDb movie database about movies and people. You can use the search_movie method of the access object to search for movies with a given (or similar) title. Similarly, you can search for people and companies using the search_person and the search_company methods. Movie, person, and company objects have id attributes which -when fetched through the IMDb web server- store the IMDb id of the object. ... -
6
Grafana
Leading open-source visualization and observability platform
Grafana OSS is the leading open-source platform for visualization and observability. It enables teams to query, visualize, alert on, and explore telemetry data from multiple sources in a single interface. With support for 100+ data source plugins—including Prometheus, Loki, Elasticsearch, InfluxDB, SQL/NoSQL databases, and OpenTelemetry—Grafana helps teams correlate metrics, logs, and traces across applications and infrastructure. Users can build interactive dashboards with rich... -
7
CodeChecker
CodeChecker is an analyzer tooling, defect database
CodeChecker is a static analysis infrastructure built on the LLVM/Clang Static Analyzer toolchain, replacing scan-build in a Linux or macOS (OS X) development environment. Executes Clang-Tidy and Clang Static Analyzer with Cross-Translation Unit analysis, Statistical Analysis (when checkers are available). Creates the JSON compilation database by wiretapping any build process (e.g., CodeChecker log -b "make"). Automatically analyzes GCC cross-compiled projects: detecting GCC or Clang... -
8
ddia
Designing Data-Intensive Application
ddia is a Chinese translation and online reading project for “Designing Data-Intensive Applications.” It covers both the first edition and ongoing second-edition translation work. The repository organizes the book content into chapters about data systems, storage, encoding, replication, partitioning, transactions, distributed systems, consistency, and derived data. It is built for readers who want deep systems design material in Chinese with a structured web reading experience. The project... -
9
OSV.dev
Open source vulnerability DB and triage service
osv.dev (Open Source Vulnerabilities) is Google’s open source platform and API for aggregating, managing, and analyzing vulnerability data across multiple ecosystems. It powers the osv.dev website, providing a unified, queryable database of vulnerabilities that map directly to open source packages and versions. The system hosts vulnerability data for ecosystems such as PyPI, npm, Go, Maven, and Debian, among others. The platform includes a web UI, API, and a Go-based dependency scanner... -
Host LLMs in Production With On-Demand GPUsDeploy your model, get an endpoint, pay only for compute time. No GPU provisioning or infrastructure management required.
-
10
TeleDrive
Unlimited Telegram cloud drive across 5 platforms & 11 languages.
TeleDrive is a 100% open-source virtual cloud filesystem that turns private Telegram storage channels into an unthrottled, unlimited personal cloud drive with zero monthly subscription fees. Connect directly to Telegram MTProto 2.0 datacenters with zero relay servers and zero cloud storage bills. Key Highlights: • Zero Monthly Rent: Unlimited file storage via private Telegram channels. • 5 Native Platforms: Builds for macOS (DMG), Windows (EXE/MSI), Linux (AppImage/DEB/RPM), Android... -
11
isfdb is a set of web-based bibliographic tools for displaying author, award, and publication bibliographies. It uses MySQL plus Python and contains a set of editing applications, as well as a moderated approval system. Used at www.isfdb.org.
-
12
Cinemagoer
Python package to retrieve and manage data of the IMDb
Cinemagoer is a Python package useful to retrieve and manage the data of the IMDb movie database about movies, people, characters and companies. Platform-independent, it can retrieve data from both the IMDb's web server and a local copy of the whole db. -
13
Audax Data Manager
The database manager that stays out of your way.
Audax Data Manager is the SQL Editor that's fast enough for a one-off query, and powerful enough for backups, performance tuning, and database design. Supports PostgreSQL, SQLite, Microsoft SQL Server, MySQL, MariaDB, and Oracle. Works on Linux, Windows and Mac. Explore and edit your data, write SQL queries with smart autocomplete, monitor database performance, run background backups and more! -
14
aiopg
aiopg is a library for accessing a PostgreSQL database
aiopg is an asynchronous PostgreSQL database driver for Python built on top of asyncio and psycopg2. It provides support for asynchronous query execution and connection pooling, enabling efficient, non-blocking database access in Python applications. aiopg is well-suited for web services and microservices that require concurrency without threads. -
15
Pity
API test platform
Pity is a powerful and user-friendly open-source API test platform that supports automated test case creation, test scheduling, and result visualization. Designed for developers and QA teams, it provides a web-based interface for writing, organizing, and executing HTTP tests, making it easy to maintain robust API quality in CI/CD pipelines. -
16
PHP mini vulnerability suite
Multiple server/webapp vulnerability scanner
github: https://github.com/samedog/phpmvs -
17
IntrospectDB
File System Relational / Object Oriented DBMS/CMS
Introspect is a File System oriented Database Management System And CMS built on UNIX as a Python and HTML webapp. For the latest version use the Bitbucket repository: https://bitbucket.org/mieszkowski/introspect/src/master/ -
18
labman2
A web database for experimental results of research
A database with a web interface for storing the experimental results of research, aimed at ease of adding new types of data. Goal is to make adding as simple as one day for an unexperienced in programming scientist to add a new type of data similar to an existing type, by preparing two - three files analogues to the existing ones. An experienced programmer should need only one day to one week for adding a totally new kind of data like arrays with more dimensions that was available before. -
19
Tera
An Internet-scale database
...Multi-type table support (RAMDISK/SSD/DISK table). Easy to use C++/Java/Python/REST-ful API. Column-oriented storage and locality group support. Ranged and hashed sharding strategies. -
20
Smart-M3 is a functional platform that provides a cross domain search extent for triple based information. Smart-M3 enables smart cross domain applications that rely on information level interoperability.
-
21
QAL
Query Abstraction Layer
...Of course custom SQL:s are also supported. It is currently distributed as a Python 3 Library (pip3 install python3-qal) and Debian .deb package. It is related the Optimal BPM project, see its Optimal Sync application for usage examples. The text of this page is released under the Creative Commons Zero Waiver 1.0 (CC0). -
22
Scalaris
Distributed, transactional key-value store
Scalaris is a scalable, transactional, distributed key-value store. It was the first NoSQL database, that supported the ACID properties for multi-key transactions. It can be used for building scalable Web 2.0 services. Scalaris uses a structured overlay with a non-blocking Paxos commit protocol for transaction processing with strong consistency over replicas. Scalaris is implemented in Erlang. -
23
NoSQLMap
Automated NoSQL database enumeration and web application exploitation
A security tool for detecting and exploiting vulnerabilities in NoSQL databases, similar to SQLMap for traditional databases. -
24
PyLucid is a Open Source web content management system written in Python using the Django Framework. fully customizable output, expandable with plugins, i18n, L10n, Revision controled, support many database engines, WSGI conform and many more...
-
25
Collective Mind Technology
plugin-based framework for systematic and reproducible experimentation
New version moved to http://github.com/ctuning/ck Collective Mind framework (cM) is an open-source plugin-based schema-free repository and infrastructure for collaborative, systematic and reproducible research and experimentation. This 3rd version (started in 2006) helps to implement, preserve, share and reproduce the whole experimental setup as connected modules and data. cM uses crowdsourcing to leverage knowledge and computational resources of multiple users. For example, it...
