Search Results for "python web crawler"
Sort By:
Showing 483 open source projects for "python web crawler"
View related business solutions-
Veeam Data Platform v13.1Try Veeam Data Platform today. Experience the unified platform that's secure by design, portable by default, and proven to recover clean, fast, and anywhere.
-
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
1
MDServer Web
Simple Linux Panel
MDServer-Web is an open-source, web-based control panel for managing web servers and hosting environments. It supports popular web servers like Nginx and Apache, along with databases such as MySQL and Redis. The panel provides a user-friendly interface to manage websites, databases, SSL certificates, and more, making server administration accessible even to those with limited technical knowledge. -
2
Zappa - Serverless Python
Serverless Python
Zappa makes it super easy to build and deploy server-less, event-driven Python applications (including, but not limited to, WSGI web apps) on AWS Lambda + API Gateway. Think of it as "serverless" web hosting for your Python apps. That means infinite scaling, zero downtime, zero maintenance - and at a fraction of the cost of your current deployments! With a traditional HTTP server, the server is online 24/7, processing requests one by one as they come in. ... -
3
BotCity Framework Core Python
BotCity Framework - Python
Recognize and interact with UI elements using state-of-art computer vision module. Operate any UI interface independent of the technology or platform (desktop, web, terminal). BotCity is a platform to develop, deploy, manage and maintain automation. Automation can be developed in Python or Java using open-source libraries that are market standard. Develop, deploy, manage and scale your Automation Ops using All in One platform that provides task queue, runtime environment management, reports, alerts, logs and much more. -
4
Pyodide
Pyodide is a Python distribution for the browser and Node.js
Pyodide brings the Python runtime to the browser by compiling Python and its scientific libraries to WebAssembly. It allows developers to run Python code directly in web browsers without a server, supporting packages like NumPy, Pandas, and Matplotlib. Pyodide opens up new possibilities for interactive data analysis, scientific computing, and educational tools in web environments, all while integrating seamlessly with JavaScript. -
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
5
pyLoad
The free and open-source Download Manager written in pure Python
pyLoad is a lightweight, open-source download manager written entirely in Python that automates downloading files from file hosting services, cloud drives, and other online sources. The software is designed to run efficiently on a wide range of systems, including desktops, servers, routers, and NAS devices, making it suitable for both personal and self-hosted environments. It uses a plugin-driven architecture that supports hundreds of hosters, link decrypters, and extensions that extend its capabilities. pyLoad includes a modern web-based interface that allows users to remotely manage downloads from a browser, enabling full control over queues, links, and download settings. ... -
6
NetBox
The premiere source of truth powering network automation
...By combining the traditional disciplines of IP address management (IPAM) and datacenter infrastructure management (DCIM) with powerful APIs and extensions, NetBox provides the ideal "source of truth" to power network automation. Available as open source software under the Apache 2.0 license, NetBox is employed by thousands of organizations around the world. Netbox is written in Python and uses the Django web framework. It is a web-based application that can be used to manage IP addresses and the devices and cables connected to them, as well as providing a data center infrastructure management (DCIM) tool. It supports virtualization, inventory management, and cable management. It has a web-based user interface and RESTful API, to easily integrate with other tools and automate tasks. -
7
Locust
Scalable open source load testing tool
Locust is an open source user load testing tool written in Python. The idea behind Locust is to swarm your web site or other systems with attacks from simulated users during a test, with each user behavior defined by you using Python code. This swarming process is then monitored from a web UI in real-time, and will help identify any bottlenecks in your code before real users can come in. As it is completely event-based, Locust can have thousands or even millions of simultaneous users distributed over multiple machines swarming your system. ... -
8
Remove MS Edge
Uninstall Microsoft Edge with an executable or batch script
Remove MS Edge is a Windows-focused project that packages methods to uninstall Microsoft Edge using either an executable approach or batch scripts, depending on how automated or hands-on you want the process to be. It targets users who prefer not to keep Edge installed and want a repeatable removal path rather than manual digging through system folders and app packages. The repository highlights practical compatibility considerations, including that certain apps and features may rely on... -
9
Shelfmark
Web interface for searching and downloading books and audiobooks
Shelfmark is a self-hosted web application that serves as a unified interface for searching, discovering, and downloading ebooks and audiobooks from multiple sources into a personal digital library. Formerly known as Calibre Web Automated Book Downloader, the platform aggregates content from web archives, torrents, Usenet, IRC, and metadata providers to create a centralized acquisition hub. It provides both direct search and metadata-driven discovery modes, enabling richer results and... -
Cut Data Warehouse Costs by 54%BigQuery delivers 54% lower TCO with exabyte scale and flexible pricing. Free migration tools handle the SQL translation automatically.
-
10
Flower
Real-time monitor and web admin for Celery distributed task queue
Flower is an open-source web application that provides real-time monitoring and administrative control for Celery distributed task queues. It exposes detailed visibility into worker status, task execution history, and queue metrics through an interactive web dashboard. Developers and operators can remotely manage workers by restarting instances, adjusting pool sizes, revoking tasks, or applying rate limits without direct server access. -
11
Glances
An eye on your system
Glances is an open source, cross-platform monitoring tool that aims to provide a significant amount of monitoring information through a curses or Web-based interface. Depending on the size of the user interface, this information can then dynamically adapt. Glances can work in client/server mode, and is also capable of remote monitoring. All systems statistics can be exported to files or external time/value databases. Glances gets information from your system through various libraries,... -
12
Ulauncher
Feature rich application Launcher for Linux
Type in an application name without worrying about spelling. Ulauncher will figure out what you meant. It also remembers your previous choices and automatically selects the best option for you. Ulauncher provides 4 themes built in. But if you need something different you can always create a custom color theme. Improve your workflow with customizable shortcuts and extensions. Create a shortcut for web search or your scripts or install a 3rd party extension. -
13
dirsearch
Web path scanner
An advanced command-line tool designed to brute force directories and files in webservers, AKA web path scanner. Wordlist is a text file, each line is a path. About extensions, unlike other tools, dirsearch only replaces the %EXT% keyword with extensions from -e flag. For wordlists without %EXT% (like SecLists), -f | --force-extensions switch is required to append extensions to every word in wordlist, as well as the /. To use multiple wordlists, you can separate your wordlists with commas.... -
14
Nautobot
Network Source of Truth & Network Automation Platform
Nautobot is an open-source network source of truth and automation platform designed to manage network infrastructure data effectively. Initially built as a fork of NetBox, Nautobot extends its capabilities by offering flexible data modeling, powerful REST and GraphQL APIs, and built-in automation tools. It enables network engineers and operators to store, query, and integrate network infrastructure data with external systems, making it a key component in modern network automation workflows.... -
15
Meli-Action
Download files from direct links, YouTube, Telegram, Google Play
Meli-Action is a GitHub Actions-based downloading and web archiving project designed for heavily restricted internet environments. It uses GitHub-hosted workflow runners to fetch content from direct links, YouTube, Telegram, Google Play, SoundCloud, and web pages. Downloaded files are committed into the user’s repository so they can be retrieved later through normal GitHub access. The project includes workflows for different content sources, plus a Python script that can render and save web pages as MHTML archives using a headless browser. ... -
16
AnySearch Skill
Unified real-time search engine skill for AI agents
AnySearch Skill is a real-time search engine skill for AI agents. It gives agents a structured way to search the web, run vertical searches, perform parallel batch searches, and extract full-page content. The project is packaged as a skill rather than a standalone search application, so it is meant to be installed into compatible AI-agent environments. It supports multiple domain-specific search categories, making it useful when general web search is too broad. The skill can also fetch page... -
17
ArchiveBox
Open source self-hosted web archiving
ArchiveBox is a powerful, self-hosted internet archiving solution to collect, save, and view websites offline. Without active preservation effort, everything on the internet eventually disappears or degrades. Archive.org does a great job as a centralized service, but saved URLs have to be public, and they can't save every type of content. ArchiveBox is an open source tool that lets organizations & individuals archive both public & private web content while retaining control over their data.... -
18
Roxy-WI
Web interface for managing Haproxy, Nginx, Apache and Keepalived
For those who need a convenient interface for managing all services in one place. Roxy-WI was created for people who want to have a fault-tolerant infrastructure, but do not want to plunge deep into the details of setting up and creating a cluster based on HAProxy, NGINX, Apache, and Keepalived. Use Roxy-WI to build a high available cluster for a couple of clicks: install HAProxy, NGINX, Apache, Keepalived, and its exporters, and carry out the initial configuration for the services. Collect... -
19
WebODM
User-friendly, commercial-grade software for processing aerial imagery
WebODM is an open-source, user-friendly application for processing aerial imagery into mapping and 3D products. It can generate georeferenced maps, point clouds, elevation models, textured 3D models, and Gaussian splats. Its interface is designed to make photogrammetry workflows accessible without requiring direct command-line operation. Processing can be handled by several supported engines, including ODX, MicMac, and LGT. The current project has formally separated from OpenDroneMap and no... -
20
Glazier
A tool for automating the installation of Windows OS
...Its extensibility makes it easy for administrators to create custom actions using Python or PowerShell, enabling tailored automation for diverse enterprise environments. Designed for engineers, Glazier emphasizes repeatability, maintainability, and transparency in Windows. -
21
Wagtail
A Django content management system focused on flexibility & UX
Wagtail is a powerful, open source content management system that’s focused on flexibility and user experience. Built on Django, Wagtail offers precise control and flexibility for designers, developers and editors. Designed by developers for developers, Wagtail plays nicely with everything else in your tech stack so you can do more and focus on perfecting your site. Designers will find Wagtail’s simple templating system ideal for building beautiful websites just the way they want, without... -
22
ClipCascade
Utility that automatically syncs the clipboard across devices
ClipCascade is a free, open-source clipboard synchronization application designed for sharing copied content between devices. It automatically transfers clipboard updates without requiring keyboard shortcuts or manual synchronization. Users can share text, images, and files across Windows, macOS, Linux, and Android. The software supports both centralized server synchronization and direct peer-to-peer connections. End-to-end encryption protects clipboard contents during transmission. Users... -
23
Synapse
Matrix reference homeserver
Matrix is an ambitious new ecosystem for open federated Instant Messaging and VoIP. Everything in Matrix happens in a room. Rooms are distributed and do not exist on any single server. Rooms can be located using convenience aliases like #matrix:matrix.org or #test:localhost:8448. Synapse is currently in rapid development, but as of version 0.5 we believe it is sufficiently stable to be run as an internet-facing service for real usage! Create and manage fully distributed chat rooms with no... -
24
Wizarr
User invitation and management system for Jellyfin, Plex, Emby etc.
Wizarr is an open-source system focused on simplifying user invitation, onboarding, and management for personal media servers like Jellyfin, Plex, and Emby, and it aims to evolve into a more comprehensive server administration tool. Initially conceived to enable administrators to create unique invite links that automatically register new users on their media servers, Wizarr abstracts many of the manual account-creation tasks typical of media server setups. It features a web interface and... -
25
Mailu
Insular email distribution - mail server as Docker images
Mailu is a simple yet full-featured mail server as a set of Docker images. It is free software (both as in free beer and as in free speech), open to suggestions and external contributions. The project aims at providing people with an easily setup, easily maintained and full-featured mail server while not shipping proprietary software nor unrelated features often found in popular groupware. Security, enforced TLS, DANE, MTA-STS, Letsencrypt!, outgoing DKIM, anti-virus scanner, Snuffleupagus,...
