Search Results for "python web crawler"
Sort By:
Showing 474 open source projects for "python web crawler"
View related business solutions-
Error to trace to log to deploy. One click. No SSH.AppSignal links every error to the trace, the trace to the log, the log to the deploy that shipped it.
-
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
1
MDServer Web
Simple Linux Panel
MDServer-Web is an open-source, web-based control panel for managing web servers and hosting environments. It supports popular web servers like Nginx and Apache, along with databases such as MySQL and Redis. The panel provides a user-friendly interface to manage websites, databases, SSL certificates, and more, making server administration accessible even to those with limited technical knowledge. -
2
Zappa - Serverless Python
Serverless Python
Zappa makes it super easy to build and deploy server-less, event-driven Python applications (including, but not limited to, WSGI web apps) on AWS Lambda + API Gateway. Think of it as "serverless" web hosting for your Python apps. That means infinite scaling, zero downtime, zero maintenance - and at a fraction of the cost of your current deployments! With a traditional HTTP server, the server is online 24/7, processing requests one by one as they come in. ... -
3
BotCity Framework Core Python
BotCity Framework - Python
Recognize and interact with UI elements using state-of-art computer vision module. Operate any UI interface independent of the technology or platform (desktop, web, terminal). BotCity is a platform to develop, deploy, manage and maintain automation. Automation can be developed in Python or Java using open-source libraries that are market standard. Develop, deploy, manage and scale your Automation Ops using All in One platform that provides task queue, runtime environment management, reports, alerts, logs and much more. -
4
Pyodide
Pyodide is a Python distribution for the browser and Node.js
Pyodide brings the Python runtime to the browser by compiling Python and its scientific libraries to WebAssembly. It allows developers to run Python code directly in web browsers without a server, supporting packages like NumPy, Pandas, and Matplotlib. Pyodide opens up new possibilities for interactive data analysis, scientific computing, and educational tools in web environments, all while integrating seamlessly with JavaScript. -
Build Securely on AWS with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
5
NetBox
The premiere source of truth powering network automation
...By combining the traditional disciplines of IP address management (IPAM) and datacenter infrastructure management (DCIM) with powerful APIs and extensions, NetBox provides the ideal "source of truth" to power network automation. Available as open source software under the Apache 2.0 license, NetBox is employed by thousands of organizations around the world. Netbox is written in Python and uses the Django web framework. It is a web-based application that can be used to manage IP addresses and the devices and cables connected to them, as well as providing a data center infrastructure management (DCIM) tool. It supports virtualization, inventory management, and cable management. It has a web-based user interface and RESTful API, to easily integrate with other tools and automate tasks. -
6
Locust
Scalable open source load testing tool
Locust is an open source user load testing tool written in Python. The idea behind Locust is to swarm your web site or other systems with attacks from simulated users during a test, with each user behavior defined by you using Python code. This swarming process is then monitored from a web UI in real-time, and will help identify any bottlenecks in your code before real users can come in. As it is completely event-based, Locust can have thousands or even millions of simultaneous users distributed over multiple machines swarming your system. ... -
7
Shelfmark
Web interface for searching and downloading books and audiobooks
Shelfmark is a self-hosted web application that serves as a unified interface for searching, discovering, and downloading ebooks and audiobooks from multiple sources into a personal digital library. Formerly known as Calibre Web Automated Book Downloader, the platform aggregates content from web archives, torrents, Usenet, IRC, and metadata providers to create a centralized acquisition hub. It provides both direct search and metadata-driven discovery modes, enabling richer results and... -
8
pyLoad
The free and open-source Download Manager written in pure Python
pyLoad is a lightweight, open-source download manager written entirely in Python that automates downloading files from file hosting services, cloud drives, and other online sources. The software is designed to run efficiently on a wide range of systems, including desktops, servers, routers, and NAS devices, making it suitable for both personal and self-hosted environments. It uses a plugin-driven architecture that supports hundreds of hosters, link decrypters, and extensions that extend its capabilities. pyLoad includes a modern web-based interface that allows users to remotely manage downloads from a browser, enabling full control over queues, links, and download settings. ... -
9
ArchiveBox
Open source self-hosted web archiving
ArchiveBox is a powerful, self-hosted internet archiving solution to collect, save, and view websites offline. Without active preservation effort, everything on the internet eventually disappears or degrades. Archive.org does a great job as a centralized service, but saved URLs have to be public, and they can't save every type of content. ArchiveBox is an open source tool that lets organizations & individuals archive both public & private web content while retaining control over their data.... -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
10
AnySearch Skill
Unified real-time search engine skill for AI agents
AnySearch Skill is a real-time search engine skill for AI agents. It gives agents a structured way to search the web, run vertical searches, perform parallel batch searches, and extract full-page content. The project is packaged as a skill rather than a standalone search application, so it is meant to be installed into compatible AI-agent environments. It supports multiple domain-specific search categories, making it useful when general web search is too broad. The skill can also fetch page... -
11
Remove MS Edge
Uninstall Microsoft Edge with an executable or batch script
Remove MS Edge is a Windows-focused project that packages methods to uninstall Microsoft Edge using either an executable approach or batch scripts, depending on how automated or hands-on you want the process to be. It targets users who prefer not to keep Edge installed and want a repeatable removal path rather than manual digging through system folders and app packages. The repository highlights practical compatibility considerations, including that certain apps and features may rely on... -
12
Ulauncher
Feature rich application Launcher for Linux
Type in an application name without worrying about spelling. Ulauncher will figure out what you meant. It also remembers your previous choices and automatically selects the best option for you. Ulauncher provides 4 themes built in. But if you need something different you can always create a custom color theme. Improve your workflow with customizable shortcuts and extensions. Create a shortcut for web search or your scripts or install a 3rd party extension. -
13
Glances
An eye on your system
Glances is an open source, cross-platform monitoring tool that aims to provide a significant amount of monitoring information through a curses or Web-based interface. Depending on the size of the user interface, this information can then dynamically adapt. Glances can work in client/server mode, and is also capable of remote monitoring. All systems statistics can be exported to files or external time/value databases. Glances gets information from your system through various libraries,... -
14
Meli-Action
Download files from direct links, YouTube, Telegram, Google Play
Meli-Action is a GitHub Actions-based downloading and web archiving project designed for heavily restricted internet environments. It uses GitHub-hosted workflow runners to fetch content from direct links, YouTube, Telegram, Google Play, SoundCloud, and web pages. Downloaded files are committed into the user’s repository so they can be retrieved later through normal GitHub access. The project includes workflows for different content sources, plus a Python script that can render and save web pages as MHTML archives using a headless browser. ... -
15
Nautobot
Network Source of Truth & Network Automation Platform
Nautobot is an open-source network source of truth and automation platform designed to manage network infrastructure data effectively. Initially built as a fork of NetBox, Nautobot extends its capabilities by offering flexible data modeling, powerful REST and GraphQL APIs, and built-in automation tools. It enables network engineers and operators to store, query, and integrate network infrastructure data with external systems, making it a key component in modern network automation workflows.... -
16
Grafana
Leading open-source visualization and observability platform
Grafana OSS is the leading open-source platform for visualization and observability. It enables teams to query, visualize, alert on, and explore telemetry data from multiple sources in a single interface. With support for 100+ data source plugins—including Prometheus, Loki, Elasticsearch, InfluxDB, SQL/NoSQL databases, and OpenTelemetry—Grafana helps teams correlate metrics, logs, and traces across applications and infrastructure. Users can build interactive dashboards with rich... -
17
Synapse
Matrix reference homeserver
Matrix is an ambitious new ecosystem for open federated Instant Messaging and VoIP. Everything in Matrix happens in a room. Rooms are distributed and do not exist on any single server. Rooms can be located using convenience aliases like #matrix:matrix.org or #test:localhost:8448. Synapse is currently in rapid development, but as of version 0.5 we believe it is sufficiently stable to be run as an internet-facing service for real usage! Create and manage fully distributed chat rooms with no... -
18
rag-search
RAG Search API
rag-search is a lightweight Retrieval-Augmented Generation API service designed to provide structured semantic search and answer generation through a simple FastAPI backend. The project integrates web search, vector embeddings, and reranking logic to retrieve relevant context before passing it to a language model for response generation. It is built to be easily deployable, requiring only environment configuration and dependency installation to run a functional RAG service. The system... -
19
Wagtail
A Django content management system focused on flexibility & UX
Wagtail is a powerful, open source content management system that’s focused on flexibility and user experience. Built on Django, Wagtail offers precise control and flexibility for designers, developers and editors. Designed by developers for developers, Wagtail plays nicely with everything else in your tech stack so you can do more and focus on perfecting your site. Designers will find Wagtail’s simple templating system ideal for building beautiful websites just the way they want, without... -
20
Wizarr
User invitation and management system for Jellyfin, Plex, Emby etc.
Wizarr is an open-source system focused on simplifying user invitation, onboarding, and management for personal media servers like Jellyfin, Plex, and Emby, and it aims to evolve into a more comprehensive server administration tool. Initially conceived to enable administrators to create unique invite links that automatically register new users on their media servers, Wizarr abstracts many of the manual account-creation tasks typical of media server setups. It features a web interface and... -
21
Modoboa
Mail hosting made simple
Modoboa is a mail hosting and management platform including a modern and simplified Web User Interface. It provides useful components such as an administration panel and webmail. Modoboa integrates with well known software such as Postfix or Dovecot. A SQL database (MySQL, PostgreSQL or SQLite) is used as a central point of communication between all components. Modoboa is developed with modularity in mind, expanding it is really easy. -
22
Roxy-WI
Web interface for managing Haproxy, Nginx, Apache and Keepalived
For those who need a convenient interface for managing all services in one place. Roxy-WI was created for people who want to have a fault-tolerant infrastructure, but do not want to plunge deep into the details of setting up and creating a cluster based on HAProxy, NGINX, Apache, and Keepalived. Use Roxy-WI to build a high available cluster for a couple of clicks: install HAProxy, NGINX, Apache, Keepalived, and its exporters, and carry out the initial configuration for the services. Collect... -
23
Mailu
Insular email distribution - mail server as Docker images
Mailu is a simple yet full-featured mail server as a set of Docker images. It is free software (both as in free beer and as in free speech), open to suggestions and external contributions. The project aims at providing people with an easily setup, easily maintained and full-featured mail server while not shipping proprietary software nor unrelated features often found in popular groupware. Security, enforced TLS, DANE, MTA-STS, Letsencrypt!, outgoing DKIM, anti-virus scanner, Snuffleupagus,... -
24
Android Emulator Container Scripts
Minimal scripts to run the emulator in a container for various systems
android-emulator-container-scripts turns the Android Emulator into a cloud-native service you can run in Docker and Kubernetes, so teams can provision ephemeral Android devices on demand. It includes scripts and container images that configure the emulator for headless operation, wire up networking, and expose endpoints for ADB and web access. A built-in WebRTC bridge lets you stream the emulator screen to a browser with interactive input, which is ideal for CI dashboards, remote debugging,... -
25
Crossbar.io
Crossbar.io - WAMP application router
Crossbar.io is an open-source networking platform for distributed and microservice applications. It implements the Web Application Messaging Protocol, which allows application components to communicate through routed remote procedure calls and publish-subscribe messaging. The platform is designed to handle the messaging layer so developers can focus on business logic instead of building custom connection, routing, and event systems. It supports real-time application architectures where...
