Search Results for "python web crawler" - Page 4
Sort By:
Showing 669 open source projects for "python web crawler"
View related business solutions-
Gen AI apps are built with MongoDB AtlasMongoDB Atlas is the developer-friendly database used to build, scale, and run gen AI and LLM-powered apps—without needing a separate vector database. Atlas offers built-in vector search, global availability across 115+ regions, and flexible document modeling. Start building AI apps faster, all in one place.
-
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
1
Promgen
Promgen is a configuration file generator for Prometheus
Promgen is a configuration file generator for Prometheus. Promgen is a web application written with Django and can help you do several jobs. The primary management UI is a Django application and many of the concepts that apply to a typical Django application will apply to Promgen. Configure Prometheus to load the target file from Prometheus and configure AlertManager to send notifications back to Promgen. Arbitrary Django settings can be set for the Promgen web app by adding those under the... -
2
Piccolo
A fast, user friendly ORM and query builder which supports asyncio
Piccolo is a modern, fast, and type-safe ORM for Python, designed with developer ergonomics in mind. It provides a clean syntax for defining schemas and building queries while supporting both sync and async execution. With built-in admin tools and rich introspection, Piccolo is suitable for web apps, APIs, and small-to-medium scale backends that prioritize clarity and speed. -
3
Visual Studio Code
Modern IDE and code editor from Microsoft for Mac, Windows, and Linux
Visual Studio Code combines the simplicity of a code editor with what developers need for their core edit-build-debug cycle. It provides comprehensive code editing, navigation, and understanding support along with lightweight debugging, a rich extensibility model, and lightweight integration with existing tools. Visual Studio Code is a distribution of the Code - OSS repository with Microsoft-specific customizations released under a traditional Microsoft product license. Visual Studio Code is... -
4
Flask-Limiter
Rate Limiting extension for Flask
Flask-Limiter provides rate-limiting features to flask applications. It allows configuring various backends to persist the rate limits, which is provided by the limits library. Sponsored by Zuplo - fully-managed API Gateway with rate limiting, authentication, and more. Add rate limiting to your API in minutes, try it at zuplo.com Test it out. The fast endpoint respects the default rate limit while the slow endpoint uses the decorated one. ping has no rate limit associated with it. By adding... -
Build Securely on AWS with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
5
JumpServer
Manage assets on different clouds at the same time
The JumpServer bastion machine complies with the 4A specification of operation and maintenance security audit. Zero threshold, fast online acquisition and installation. Just a browser, the ultimate Web Terminal experience. Easily support massive concurrent access. One system manages assets on different clouds at the same time. Audit recordings are stored in the cloud and will never be lost. One system, is used by multiple subsidiaries and departments at the same time. Prevent identity fraud... -
6
AWX
A web-based user interface built on top of Ansible
AWX provides a web-based user interface, REST API, and task engine built on top of Ansible. It is one of the upstream projects for Red Hat Ansible Automation Platform. Starting in version 18.0, the AWX Operator is the preferred way to install AWX. AWX can also alternatively be installed and run in Docker, but this install path is only recommended for development/test-oriented deployments, and has no official published release. Uses naming and structure consistent with the AWX HTTP API.... -
7
TurboGears
Python web framework with full-stack layer
TurboGears is a hybrid web framework able to act both as a Full Stack framework or as a Microframework. TurboGears helps you get going fast and gets out of your way when you want it! TurboGears can be used both as a full stack framework or as a microframework in single-file mode. TurboGears 2 is built on top of the experience of several next-generation web frameworks including TurboGears 1 (of course), Django, and Rails. All of these frameworks had limitations that frustrated us, and TG2 was... -
8
Dulwich
Pure-Python Git implementation
Dulwich is a Python implementation of the Git file formats and protocols, which does not depend on Git itself. All functionality is available in pure Python. Optional C extensions can be built for improved performance. Dulwich takes its name from the area in London where the friendly Mr. and Mrs. Git once attended a cocktail party. Supported Python versions are Python 3.5 and later. Versions of Dulwich prior to 0.20 also supported Python 2.7. Supported platforms include Linux, Mac OS X and... -
9
Indico
A feature-rich event management system
The effortless open-source tool for event organization, archival, and collaboration. Event-organization workflow that fits lectures, meetings, workshops, and conferences. A feature-rich event management system, made @ CERN, the place where the Web was born. A powerful and flexible hierarchical content management system for events, a full-blown conference organization workflow with call for Abstracts and abstract reviewing modules; flexible registration form creation and configuration;... -
Build Securely on Azure with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
10
OSRFramework
OSRFramework, the Open Sources Research Framework is a AGPLv3+ project
OSRFramework is a GNU AGPLv3+ set of libraries developed by i3visio to perform Open Source Intelligence collection tasks. They include references to a bunch of different applications related to username checking, DNS lookups, information leaks research, deep web search, regular expressions extraction and many others. At the same time, by means of ad-hoc Maltego transforms, OSRFramework provides a way of making these queries graphically as well as several interfaces to interact with like... -
11
Douyin TikTok Download API
Douyin TikTok Download API
Use the official interface to capture Douyin|TikTok data, support API calls, Web portals, and batch analysis. Fast, asynchronous, free, open source, ad-free, long-term maintenance. This project is based on PyWebIO , FastAPI , HTTPX , a fast and asynchronous Douyin / TikTok data crawling tool, and realizes online batch parsing and downloading of watermark-free videos or atlases through the web, data crawling API, and iOS shortcut instructions for watermark-free download and other functions.... -
12
cheat.sh
The only cheat sheet you need
cheat.sh is a compact, network-accessible cheat-sheet service that serves concise examples and usage notes for hundreds of shell commands, programming languages, and tools via a simple HTTP interface. You can query it from the terminal (for example curl cht.sh/rsync or curl cheat.sh/ls) or browse the web front page; it also supports a shorthand hostname (cht.sh) and provides both online and standalone/local installation modes. The repository contains the server and client code, instructions to run a local standalone instance (including Python virtualenv setup), and tooling to fetch or maintain the upstream cheat-sheet data; installation documentation explains disk-space needs and dependency setup for offline use. ... -
13
Flask-APScheduler
Adds APScheduler support to Flask
Flask-APScheduler is a Flask extension that adds support for the APScheduler. Loads scheduler configuration from Flask configuration. Loads job definitions from Flask configuration. Allows to specify the hostname which the scheduler will run on. Provides a REST API to manage the scheduled jobs. Provides authentication for the REST API. Integrates with Flask Blueprints. -
14
Maltrail
Malicious traffic detection system
Maltrail is a malicious traffic detection system, utilizing publicly available (black)lists containing malicious and/or generally suspicious trails, along with static trails compiled from various AV reports and custom user-defined lists, where trail can be anything from domain name, URL, IP address (e.g. 185.130.5.231 for the known attacker) or HTTP User-Agent header value (e.g. sqlmap for automatic SQL injection and database takeover tool). Also, it uses (optional) advanced heuristic... -
15
Werkzeug
The comprehensive WSGI web application library
Werkzeug is a comprehensive WSGI web application library. It began as a simple collection of various utilities for WSGI applications and has become one of the most advanced WSGI utility libraries. Werkzeug doesn’t enforce any dependencies. It is up to the developer to choose a template engine, database adapter, and even how to handle requests. Includes an interactive debugger that allows inspecting stack traces and source code in the browser with an interactive interpreter for any frame in... -
16
dynaconf
Configuration Management for Python
Inspired by the 12-factor application guide. Settings management (default values, validation, parsing, templating). Protection of sensitive information (passwords/tokens). Multiple file formats toml|yaml|json|ini|py and also customizable loaders. Full support for environment variables to override existing settings (dotenv support included). Optional layered system for multi environments [default, development, testing, production] (also called multi profiles). Built-in support for Hashicorp... -
17
Mezzanine
CMS framework for Django
Mezzanine is a powerful open source content management platform built using the Django framework. In many ways it is like many other content management tools, offering an intuitive interface for managing all of your content. But Mezzanine is different in that it provides most of its functionality by default. While other platforms rely heavily on modules or reusable applications, Mezzanine comes ready with all the functionality you need, making it the more efficient choice. Mezzanine has a... -
18
Posting
The modern API client that lives in your terminal
posting is a lightweight command-line tool that lets users schedule and automate Mastodon posts using Markdown files. It reads a simple folder structure of Markdown drafts and posts them at predefined intervals or manually. Designed for content creators and developers, posting helps maintain consistent and organized Mastodon accounts without depending on web UIs or third-party schedulers. -
19
CTFd
CTFs as you need them
CTFd is a Capture The Flag framework focusing on ease of use and customizability. It comes with everything you need to run a CTF and it's easy to customize with plugins and themes. Create your own challenges, categories, hints, and flags from the Admin Interface. Dynamic Scoring Challenges. Unlockable challenge support. Challenge plugin architecture to create your own custom challenges. Static & Regex-based flags. Custom flag plugins. Unlockable hints. File uploads to the server or an Amazon... -
20
GPT All Star
AI-powered code generation tool for scratch development of web apps
AI-powered code generation tool for scratch development of web applications with a team collaboration of autonomous AI agents. This is a research project, and its primary value is to explore the possibility of autonomous AI agents. -
21
Weblate
Web based localization tool with tight version control integration
Weblate is a copylefted libre software web-based continuous localization system, used by over 2500 libre projects and companies in more than 165 countries. Copylefted libre software, used by over 2,500 libre software projects and companies in over 165 countries. Hosted service and standalone tool with tight version control integration. Simple and clean user interface, propagation of translations across components, quality checks and automatic linking to source files. There is infrastructure... -
22
TorBot
Dark Web OSINT Tool
Contributions to this project are always welcome. To add a new feature fork the dev branch and give a pull request when your new feature is tested and complete. If its a new module, it should be put inside the modules directory. The branch name should be your new feature name in the format <Feature_featurename_version(optional)>. On Linux platforms, you can make an executable for TorBot by using the install.sh script. You will need to give the script the correct permissions using chmod +x... -
23
SickChill
Less rage, more chill
Automatic Video Library Manager for TV shows. It watches for new episodes of your favorite shows, and when they are posted it does its magic. Select the show you want to grab, add it, and let SickChill handle the rest. See what SickChill holds in store for you. SickChill has a nice calendar that allows you to know what you will see next. It watches for new episodes of your favorite shows, and when they are posted it does its magic: automatic torrent/nzb searching, downloading, and processing... -
24
Awesome Free ChatGPT
List of free ChatGPT mirror sites, continuously updated
This is a curated directory of freely accessible ChatGPT-style services and mirror sites that offer AI chatbot interfaces without login or payment requirements. Resources often support multiple models like GPT-4, Claude, Gemini, and more. Data collected from multiple independent sites with descriptions and tags. Includes services with image upload and drawing capabilities. Aggregates free, no-login-required ChatGPT-like web services. Continually updated mirror list to maintain availability. -
25
Attendedsysupgrade Server
An image on demand server for OpenWrt based distributions
The Attended Sysupgrade (ASU) is a service developed by OpenWrt to streamline the firmware upgrade process for devices running OpenWrt or its derivatives. It allows users to generate custom firmware images with a selected set of pre-installed packages, eliminating the need to manually reinstall packages after an upgrade. This service simplifies maintaining up-to-date and customized firmware across various devices.
