Search Results for "python web crawler" - Page 17
Sort By:
Showing 3076 open source projects for "python web crawler"
View related business solutions-
Veeam Data Platform v13.1Try Veeam Data Platform today. Experience the unified platform that's secure by design, portable by default, and proven to recover clean, fast, and anywhere.
-
Go from Code to Production URL in SecondsSkip the Kubernetes configs. Cloud Run handles HTTPS, scaling, and infrastructure automatically. Two million requests free per month.
-
1
docext
An on-premises, OCR-free unstructured data extraction
docext is a document intelligence toolkit that uses vision-language models to extract structured information from documents such as PDFs, forms, and scanned images. The system is designed to operate entirely on-premises, allowing organizations to process sensitive documents without relying on external cloud services. Unlike traditional document processing pipelines that rely heavily on optical character recognition, docext leverages multimodal AI models capable of understanding both visual... -
2
IntelOwl
Centralized platform for automated threat intelligence analysis
IntelOwl is an open source platform designed to manage and enrich threat intelligence data at scale. It provides a centralized environment where security analysts can gather information about suspicious files and observables such as IP addresses, domains, URLs, or hashes using a single API request. The platform integrates numerous online intelligence sources and advanced malware analysis tools, enabling users to obtain comprehensive threat intelligence without manually querying multiple... -
3
Archivematica
Free and open-source digital preservation system
Archivematica is a web- and standards-based, open-source application which allows your institution to preserve long-term access to trustworthy, authentic, and reliable digital content. Our target users are archivists, librarians, and anyone working to preserve digital objects. You are free to copy, modify, and distribute Archivematica with attribution under the terms of the AGPLv3 license. Archivematica is an open-source application based on recognized standards that makes it possible to... -
4
PyTorch Geometric Temporal
Spatiotemporal Signal Processing with Neural Machine Learning Models
The library consists of various dynamic and temporal geometric deep learning, embedding, and Spatio-temporal regression methods from a variety of published research papers. Moreover, it comes with an easy-to-use dataset loader, train-test splitter and temporal snaphot iterator for dynamic and temporal graphs. The framework naturally provides GPU support. It also comes with a number of benchmark datasets from the epidemiological forecasting, sharing economy, energy production and web traffic... -
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
5
ClearML
Streamline your ML workflow
...It is designed as an end-to-end MLOps suite allowing you to focus on developing your ML code & automation, while ClearML ensures your work is reproducible and scalable. The ClearML Python Package for integrating ClearML into your existing scripts by adding just two lines of code, and optionally extending your experiments and other workflows with ClearML powerful and versatile set of classes and methods. The ClearML Server storing experiment, model, and workflow data, and supports the Web UI experiment manager, and ML-Ops automation for reproducibility and tuning. ... -
6
Graph Notebook
Library extending Jupyter notebooks to integrate with Apache TinkerPop
The graph notebook provides an easy way to interact with graph databases using Jupyter notebooks. Using this open-source Python package, you can connect to any graph database that supports the Apache TinkerPop, openCypher or the RDF SPARQL graph models. These databases could be running locally on your desktop or in the cloud. Graph databases can be used to explore a variety of use cases including knowledge graphs and identity graphs. This project includes many examples of Jupyter notebooks.... -
7
X4G
Fast and modern gateway for VLESS tunneling on WebSocket and XHTTP
X4G is a self-hosted gateway for operating VLESS tunnels over WebSocket and two XHTTP transport modes. It also includes an internal HTTP proxy for clients that need a conventional proxy endpoint. A web dashboard displays service statistics, hourly traffic charts, live connections, activity records, and error logs. Administrators can create unlimited connection links with individual traffic quotas, bandwidth limits, expiration dates, ports, and simultaneous IP restrictions. Subscription... -
8
Lagent
A lightweight framework for building LLM-based agents
Lagent is a lightweight open-source framework designed to help developers build autonomous agents powered by large language models. The framework provides tools and abstractions that allow language models to interact with external tools, execute tasks, and perform multi-step reasoning processes. Instead of using LLMs only for text generation, Lagent enables developers to transform models into agents capable of performing actions such as retrieving data, executing code, or interacting with... -
9
screenshot-to-code
Drop in a screenshot and convert it to clean code
screenshot-to-code converts UI screenshots or design images into working front-end code, accelerating the path from concept to prototype. It uses modern vision-capable or code-generating models to infer layout structure, typography, and components, then outputs clean HTML/CSS (often Tailwind) or framework code. A web interface lets you upload images, tune options, and preview generated results, while a backend service orchestrates the model calls and post-processing. The tool focuses on... -
99.99% Uptime for MySQL and PostgreSQL DatabasesCloud SQL Enterprise Plus delivers near-zero downtime with 35 days of point-in-time recovery. Supports MySQL, PostgreSQL, and SQL Server.
-
10
OSV.dev
Open source vulnerability DB and triage service
osv.dev (Open Source Vulnerabilities) is Google’s open source platform and API for aggregating, managing, and analyzing vulnerability data across multiple ecosystems. It powers the osv.dev website, providing a unified, queryable database of vulnerabilities that map directly to open source packages and versions. The system hosts vulnerability data for ecosystems such as PyPI, npm, Go, Maven, and Debian, among others. The platform includes a web UI, API, and a Go-based dependency scanner... -
11
Marvin
A batteries-included library for building AI-powered software
Meet Marvin: a batteries-included library for building AI-powered software. Marvin's job is to integrate AI directly into your codebase by making it look and feel like any other function. Marvin introduces a new concept called AI Functions. These functions differ from conventional ones in that they don’t rely on source code, but instead generate their outputs on-demand through AI. With AI functions, you don't have to write complex code for tasks like extracting entities from web pages,... -
12
Apprise
Apprise - Push Notifications that work with just about every platform!
Take advantage of Apprise through your network with a user-friendly API. Apprise API was designed to easily fit into existing (and new) eco-systems that are looking for a simple notification solution. There is a small built-in Configuration Manager that can be optionally accessed through your web browser allowing you to create and save as many configurations as you'd like. Each configuration is differentiated by a unique {KEY} that you decide on. Once you've saved your configuration, you'll... -
13
Rhino
On-device Speech-to-Intent engine powered by deep learning
Rhino is Picovoice's Speech-to-Intent engine. It directly infers intent from spoken commands within a given context of interest, in real-time. The end-to-end platform for embedding private voice AI into any software in a few lines of code. Design with no limits on top of a modular platform. Create use-case-specific voice AI models in seconds. Develop voice features with a few lines of code using intuitive and cross-platform SDKs. Deliver voice AI everywhere: on-device, mobile, web browsers,... -
14
django-helpdesk
A Django application to manage tickets for an internal helpdesk
A Django application to manage tickets for an internal helpdesk. Formerly known as Jutda Helpdesk. django-helpdesk was formerly known as Jutda Helpdesk, named after the company which originally created it. As of January 2011 the name has been changed to reflect what it really is: a Django-powered ticket tracker with contributors reaching far beyond Jutda. django-helpdesk includes a basic demo Django project so that you may easily get started with testing or developing django-helpdesk. The... -
15
Tencent Cloud Code Analysis
Static code analysis
Tencent Cloud Code Analysis (TCA for short, used internally by the R&D code CodeDog ) is a cloud-native, distributed, high-performance comprehensive code analysis and tracking platform that integrates many analysis tools, including server, web and client The three components have integrated a number of self-developed tools, and also support the dynamic integration of analysis tools of various programming languages in the industry. Obtain the Tencent Cloud code analysis platform by... -
16
CoPaw
Your Personal AI Assistant; easy to install, deploy on local or coud
CoPaw is a personal AI assistant designed to run on your own machine or in the cloud, giving you full control over memory, models, and data. Built by the AgentScope team, it connects to multiple chat platforms—including DingTalk, Feishu, QQ, Discord, iMessage, and more—through a single unified assistant. CoPaw supports both cloud-based LLM providers and fully local models such as llama.cpp, MLX, and Ollama, allowing you to operate without API keys if preferred. It includes a browser-based... -
17
FAY
Framework for building AI-powered interactive digital humans and agent
Fay is an open source framework designed to build and deploy interactive digital humans powered by large language models. It acts as a middleware layer that connects digital character technologies with conversational AI systems and business applications. Fay supports various types of digital humans, including 2.5D and 3D avatars, and can be integrated with applications running on mobile devices, PCs, web platforms, and embedded systems. Its architecture allows developers to combine different... -
18
GTFOBins
GTFOBins is a curated list of Unix binaries
GTFOBins is a curated catalog of Unix / POSIX system binaries and how they can be misused to bypass restrictions, escalate privileges, exfiltrate data, spawn shells, or otherwise act as “living off the land” tools in a compromised environment. It collects documented techniques for how everyday binaries (e.g. awk, bash, tar, scp) can be abused under constrained conditions. Indexed list of Unix binaries and documented misuse techniques. Examples of command invocations to exploit... -
19
Raccoon
High-performance reconnaissance and vulnerability scanning tool
Raccoon is a high-performance offensive security tool designed to assist with reconnaissance and vulnerability scanning during penetration testing and security assessments. It automates several common reconnaissance tasks, allowing security professionals to quickly gather information about a target system or web application. The tool combines multiple scanning techniques into a single workflow, helping users identify potential weaknesses, exposed services, and accessible resources on a... -
20
/last30days
Claude Code skill that researches any topic across Reddit + X
/last30days is a specialized Claude Code skill designed to research current trends and practices across Reddit, X, and the wider web from the last 30 days, synthesize that data, and produce copy-paste-ready prompts or summaries that reflect what the community is actually talking about now. Rather than returning generic model responses, it intelligently analyzes social media and community discussions to identify what’s genuinely trending or working in practice across topics ranging from... -
21
Kinto
A generic JSON document store with sharing and synchronisation options
Kinto is a minimalist JSON storage service with synchronization and sharing abilities. It is meant to be easy to use and easy to self-host. Kinto is used at Mozilla and released under the Apache v2 license. It’s hard for frontend developers to respect users' privacy when building applications that work offline, store data remotely and synchronize across devices. Existing solutions either rely on big corporations that crave user data or require a non-trivial amount of time and expertise to... -
22
HTTPie Desktop
Cross-platform API testing client for humans
HTTPie Desktop is a graphical API client built on top of the popular HTTPie terminal tool, offering a user-friendly interface for testing and interacting with APIs. It combines the simplicity of HTTPie’s CLI with a modern desktop and web UI for a more visual workflow. Developers can easily build, send, and preview HTTP requests without needing to memorize commands or write scripts. The platform supports organizing work into spaces, collections, and tabs, making it ideal for managing multiple... -
23
nanochat
The best ChatGPT that $100 can buy
nanochat is a from-scratch, end-to-end “mini ChatGPT” that shows the entire path from raw text to a chatty web app in one small, dependency-lean codebase. The repository stitches together every stage of the lifecycle: tokenizer training, pretraining a Transformer on a large web corpus, mid-training on dialogue and multiple-choice tasks, supervised fine-tuning, optional reinforcement learning for alignment, and finally efficient inference with caching. Its north star is approachability and... -
24
NGINX Admin’s Handbook
How to improve NGINX performance, security, and other important things
nginx-admins-handbook is a practical, in-depth guide for configuring, securing, and operating NGINX across real-world deployments. It distills years of research, notes, and field experience into a single handbook that complements the official docs with concrete rules, explanations, and curated external references. The handbook spans fundamentals and advanced topics alike, from HTTP and SSL/TLS basics to reverse proxy patterns, performance tuning, debugging workflows, and hardening... -
25
Stable Diffusion Version 2
High-Resolution Image Synthesis with Latent Diffusion Models
Stable Diffusion (the stablediffusion repo by Stability-AI) is an open-source implementation and reference codebase for high-resolution latent diffusion image models that power many text-to-image systems. The repository provides code for training and running Stable Diffusion-style models, instructions for installing dependencies (with notes about performance libraries like xformers), and guidance on hardware/driver requirements for efficient GPU inference and training. It’s organized as a...
