Search Results for "python web crawler"
Sort By:
Showing 313 open source projects for "python web crawler"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
1
Lightnovel Crawler
Generate and download e-books from online sources
Lightnovel Crawler is a tool for downloading web novels and similar fiction from supported online reading sites. It can save stories as offline e-books so users can read them on phones, tablets, computers, or e-readers. The project supports command-line use, standalone executables, Docker deployment, and a browser-based web interface. Users can paste a novel URL, choose chapter ranges, track download progress, and open results in a built-in reader. -
2
Web Spider, Web Crawler, Email Extractor
Free Extracts Emails, Phones and custom text from Web using JAVA Regex
In Files there is WebCrawlerMySQL.jar which supports MySql Connection Free Web Spider & Crawler. Extracts Information from Web by parsing millions of pages. Store data into Derby Database and data are not being lost after force closing the spider. - Free Web Spider , Parser, Extractor, Crawler - Extraction of Emails , Phones and Custom Text from Web - Export to Excel File - Data Saved into Derby and MySQL Database - Written in Java Cross Platform Also See Free email Sender : https://sourceforge.net/projects/gitst-free-email-ender/ Please install Microsoft OpenJDK to start the application https://www.microsoft.com/openjdk -
3
Auto Bangumi
Automated Bangumi episode downloader and organizer with Web UI
Auto_Bangumi is a fully automated tool for downloading, organizing, and tracking anime (Bangumi) episodes using RSS feeds and download clients like qBittorrent. It offers a modern Web UI for managing subscriptions, custom filtering rules, automatic file renaming, and subtitle matching. Designed for anime fans, it streamlines the process of staying up-to-date with seasonal shows by integrating feed parsing, downloading, and library organization. -
4
Modoboa
Mail hosting made simple
Modoboa is a mail hosting and management platform including a modern and simplified Web User Interface. It provides useful components such as an administration panel and webmail. Modoboa integrates with well known software such as Postfix or Dovecot. A SQL database (MySQL, PostgreSQL or SQLite) is used as a central point of communication between all components. Modoboa is developed with modularity in mind, expanding it is really easy. -
Host LLMs in Production With On-Demand GPUsDeploy your model, get an endpoint, pay only for compute time. No GPU provisioning or infrastructure management required.
-
5
reg-factory
Google Mail registration
reg-factory is an automation-oriented account registration pipeline aimed at research, testing, and controlled platform workflow experiments. It combines browser automation, environment-based configuration, and a local web control panel to run multi-step registration scripts. The project includes modules for email registration, platform sign-up flows, token handling, cookie export, and account workflow management. It uses local scripts and external tool integrations to coordinate browser... -
6
Synapse
Matrix reference homeserver
Matrix is an ambitious new ecosystem for open federated Instant Messaging and VoIP. Everything in Matrix happens in a room. Rooms are distributed and do not exist on any single server. Rooms can be located using convenience aliases like #matrix:matrix.org or #test:localhost:8448. Synapse is currently in rapid development, but as of version 0.5 we believe it is sufficiently stable to be run as an internet-facing service for real usage! Create and manage fully distributed chat rooms with no... -
7
Kirara AI
DIY multimodal AI chatbot
Kirara AI is a chatbot framework designed to connect large language models with mainstream chat platforms. It supports multi-turn conversation, persona presets, keyword-triggered replies, administrator commands, and conditional automation. The project can work with model providers such as OpenAI, DeepSeek, Claude, Gemini, Qwen, Mistral, Kimi, Minimax, and others. It also supports image generation models, voice replies, cross-platform message sending, and custom workflows. A WebUI helps users... -
8
INTERCEPT
Unites the best signal intelligence tools
iNTERCEPT is a web-based interface that brings multiple software-defined radio and signal-intelligence style tools under one consistent dashboard, making complex workflows more approachable. Rather than requiring you to learn a different UI and setup process for each underlying utility, it provides a single place to start modes, view results, and monitor activity from a browser. The project’s goal is accessibility: lowering the skill and setup barrier so learners and authorized testers can... -
9
DiscordPHP
An API to interact with the popular messaging app Discord
...It is built around ReactPHP components, so developers need to understand event loops, promises, and long-running CLI processes. The library is not designed to run like a traditional web request inside Apache or Nginx. Instead, it runs as a persistent PHP process that connects to Discord, listens for events, and reacts through handlers. It supports Composer installation, intents, examples, documentation, community tutorials, and framework-agnostic bot development. DiscordPHP is best suited for PHP developers who want to build real-time Discord applications without switching to JavaScript, Python, or Go. -
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
10
Kinto
A generic JSON document store with sharing and synchronisation options
Kinto is a minimalist JSON storage service with synchronization and sharing abilities. It is meant to be easy to use and easy to self-host. Kinto is used at Mozilla and released under the Apache v2 license. It’s hard for frontend developers to respect users' privacy when building applications that work offline, store data remotely and synchronize across devices. Existing solutions either rely on big corporations that crave user data or require a non-trivial amount of time and expertise to... -
11
PULSAR
Distributed pub-sub messaging system
Apache Pulsar is a cloud-native, distributed messaging and streaming platform originally created at Yahoo! and now a top-level Apache Software Foundation project. Easy to deploy, lightweight compute process, developer-friendly APIs, no need to run your own stream processing engine. Run in production at Yahoo! scale for over 5 years, with millions of messages per second across millions of topics. Expand capacity seamlessly to hundreds of nodes. Low publish latency (< 5ms) at scale with strong... -
12
ipwb
A distributed and persistent archive replay system using IPFS
InterPlanetary Wayback (ipwb) facilitates permanence and collaboration in web archives by disseminating the contents of WARC files into the IPFS network. IPFS is a peer-to-peer content-addressable file system that inherently allows deduplication and facilitates opt-in replication. ipwb splits the header and payload of WARC response records before disseminating into IPFS to leverage the deduplication, builds a CDXJ index with references to the IPFS hashes returned, and combines the header and... -
13
Mail-in-a-Box
Take back control of your email with this mail server in a box
Mail-in-a-Box helps individuals take back control of their email by defining a one-click, easy-to-deploy SMTP+everything else server, a mail server in a box. Mail-in-a-Box lets you become your own mail service provider in a few easy steps. It’s sort of like making your own gmail, but one you control from top to bottom. Technically, Mail-in-a-Box turns a fresh cloud computer into a working mail server. But you don’t need to be a technology expert to set it up. Our goals are to make deploying... -
14
Tock
Tock, the open source conversational AI toolkit
Complete and autonomous NLU solution leveraging opensource libs, such as OpenNLP, Stanford, Duckling and more. Web, mobile, social networks, smart speakers and more. Create your bot once, connect it progressively to multiple channels as you need them. Simple graphical interfaces to build stories and models, manage multilingual and multichannel bots, better understand users with analytics. Program complex stories using Kotlin, Python or Node.js provided components, or integrate with any language by leveraging Tock APIs. ... -
15
qBittorrent
A free and reliable P2P BitTorrent client
An advanced and multi-platform BitTorrent client with a nice Qt user interface as well as a Web UI for remote control and an integrated search engine. qBittorrent aims to meet the needs of most users while using as little CPU and memory as possible. -
16
Roundup Issue Tracker
Simple-to-use/-install issue-tracking system: web, REST, email, CLI
Roundup is a simple-to-use and -install issue-tracking system with command-line, web, REST, XML-RPC, and e-mail interfaces. It runs on python 2.7 or 3.6 or newer. Roundup has been deployed for: * bug tracking and TODO list management (the classic installation) * customer help desk support (with a wizard for the phone answerers, linking to networking, system and development issue trackers) * issue management for IETF working groups * sales lead tracking ...and so on. ... -
17
ZEG / Zero-Effort-Groupware
SOGo Zero-Effort-Groupware
The ZEG (Zero Effort Groupware) edition of SOGo is intended to provide a complete out-of-the-box testing environment of SOGo, the Open Source messaging and calendaring software. -
18
SALERATOR
Acelerador de venda para autopeças via WhatsApp. Alerta personalizado
SALERATOR é o Acelerador de Vendas definitivo para balcões de autopeças, motopeças, oficinas mecânicas e distribuidores que vendem pelo WhatsApp. Recursos do SALERATOR: 1. Alerta Sonoro Imediato: Assim que a mensagem chega, o computador do balcão notifica com clareza “Tem Cliente no WhatsApp!” Nenhuma cotação passa despercebida, mesmo com o balcão lotado. 2. Fila Organizada: Separa o que está em aberto, em atendimento e fechado, acabando com a bagunça de chats perdidos no celular... -
19
Network Security Toolkit (NST)
A network security analysis and monitoring toolkit Linux distribution.
Network Security Toolkit (NST) is a bootable ISO image (Live USB Flash Drive) based on Fedora 44 providing easy access to best-of-breed Open Source Network Security Applications and should run on most x86_64 systems. The main intent of developing this toolkit was to provide the security professional and network administrator with a comprehensive set of Open Source Network Security Tools. The majority of tools published in the article: Top 125 Security Tools by INSECURE.ORG are available... -
20
ufonet
UFONet - Denial of Service Toolkit
UFONet - Is a set of hacktivist tools that allow launching coordinated DDoS and DoS attacks and combine both in a single offensive. It also works as an encrypted DarkNET to publish and receive content by creating a global client/server network based on a direct-connect P2P architecture. + FAQ: https://ufonet.03c8.net/FAQ.html -------------------------------------------- -> UFONet-v2.0 [R3D] "R3DST4R!" (.zip) -> md5 = [98fcd2bd3bba3f1d52738113522d3bc9] -> UFONet-v2.0 [R3D]... -
21
elFinder is a file manager for web similar to that you use on your computer. Written in JavaScript using jQuery UI, it just work's in any modern browser. Its creation is inspired by simplicity and convenience of Finder.app program used in Mac OS X.
-
22
inTox is a lightweight, peer-to-peer (P2P) messaging application built with Python, PyQt6, and Flask-SocketIO. It combines a modern web-powered GUI with the privacy of local execution and decentralized communication.
-
23
MailCleaner
Anti Spam SMTP Gateway
MailCleaner Open Source Edition is now discontinued but will continue under the spamtagger project https://github.com/SpamTagger [antispam] MailCleaner is an anti-spam / anti-virus filter SMTP gateway with user and admin web interfaces, quarantine, multi-domains, multi-templates, multi-languages. Using Bayes, RBLs, Spamassassin, MailScanner, ClamAV. Based on Debian. Enterprise ready. MailCleaner is an anti spam gateway installed between your mail infrastructure and the Internet. It... -
24
MOFO Linux
A live Linux environment for computing without censorship barriers.
MOFO Linux is a USB pluggable live Linux environment you boot on PC hardware. It gives you the power to unblock any media, at your discretion, clearing the way for you to read, write, watch, listen to, debate, or collaborate anywhere - beyond the reach of Big Brother. In other words, you jump the barrier, find media, and interact with people. MOFO Linux is designed for easy usage on home PCs, laptop computers, or workstations, whether installed in internet cafes anywhere the world or on... -
25
Oblivion-Inverse
Open source e-mail tracker made with Flask, setup & run in 10 minutes
Oblivion-Inverse is a simple, free & open source e-mail tracking solution which based on the usage of web beacons or tracking pixels. Build with Flask and setup under 10 minutes on your own production environment. Designed to provide information about the email read status, time, IP address of the recipient's device or proxy, as well as request headers such as the user-agent, which can reveal details about the recipient's browser, operating system, and device.
