Search Results for "python web crawler" - Page 10
Sort By:
Showing 3076 open source projects for "python web crawler"
View related business solutions-
Veeam Data Platform v13.1 - Get Your Free TrialTry Veeam Data Platform today. Experience the unified platform that's secure by design, portable by default, and proven to recover clean, fast, and anywhere.
-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
1
Piccolo
A fast, user friendly ORM and query builder which supports asyncio
Piccolo is a modern, fast, and type-safe ORM for Python, designed with developer ergonomics in mind. It provides a clean syntax for defining schemas and building queries while supporting both sync and async execution. With built-in admin tools and rich introspection, Piccolo is suitable for web apps, APIs, and small-to-medium scale backends that prioritize clarity and speed. -
2
Open Notebook
An Open Source implementation of Notebook LM with more flexibility
Open Notebook is an open-source, privacy-focused alternative to Google’s Notebook LM that gives users full control over their research and AI workflows. Designed to be self-hosted, it ensures complete data sovereignty by keeping your content local or within your own infrastructure. The platform supports 16+ AI providers—including OpenAI, Anthropic, Ollama, Google, and LM Studio—allowing flexible model choice and cost optimization. Open Notebook enables users to organize and analyze... -
3
Portainer Templates
500+ 1-click Portainer app templates
A compiled list of 500+ ready-to-go Portainer App templates. In Portainer, App Templates enable you to easily deploy services with a predetermined configuration, while allowing you to customize options through the web UI. While Portainer ships with some default templates, it's often helpful to have 1-click access to many more apps + stacks, without having to constantly switch template sources. This repo combines app templates from several sources, to create a ready-to-go template file... -
4
Kiro Crew
A persistent workspace for development work that self-improves
Kiro Crew is an open-source persistent development workspace that keeps AI-assisted work running beyond a single chat session. It can operate locally, in Docker, or on a remote machine controlled by the user. Sessions, memory, schedules, checkpoints, and learned preferences survive restarts and can influence later work. Long-running tasks can plan steps, execute them, validate results, retry failures, and resume from saved checkpoints. The system supports subagents, reusable skills, MCP... -
Build Securely on AWS with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
5
FFmpeg Quality Metrics
Calculate quality metrics with FFmpeg (SSIM, PSNR, VMAF, VIF)
...It also includes optional visualization features through an interactive web-based dashboard for analyzing results graphically. Designed for cross-platform use, it integrates seamlessly into automated workflows and supports JSON or CSV output formats for further analysis. This makes it particularly valuable for engineers working on video compression and streaming systems. -
6
Helium
Lighter web automation with Python
Helium is a Python library built on top of Selenium to make browser automation more intuitive and human-friendly. It replaces verbose boilerplate code with natural language-like API calls such as click("Login") or write("hello", into="Name"). Helium manages browser setup, waits, and teardown, enabling quick development of scripts for testing, scraping, or task automation without requiring deep Selenium knowledge. -
7
Tortoise ORM
Familiar asyncio ORM for python, built with relations in mind
Tortoise ORM is an easy-to-use asyncio ORM (Object Relational Mapper) for Python, inspired by Django's ORM. It is designed to work with asynchronous frameworks, providing a simple and familiar API for interacting with databases. Tortoise ORM supports various relational databases and is suitable for building high-performance web applications. -
8
Webifier
A GitHub Action to deploy Notebooks, Markdowns
Webifier is a stand-alone build tool for converting any repository into a deployable jekyll website. You can define your pages via yaml files and provide notebooks, markdown and pdf and other files for Webifier to render. It uses python markdown providing additional control over attributes and other extensive functionalities. It lets you define and direct how your web pages feel and automatically manages your assets, making it a perfect solution for fast static website development and a straightforward tool for creating Github pages as a Github action. Webifier is a good fit for the missing puzzle piece of collaborative content creation on Github and is a great tool for sharing educational material on the web. -
9
Modoboa
Mail hosting made simple
Modoboa is a mail hosting and management platform including a modern and simplified Web User Interface. It provides useful components such as an administration panel and webmail. Modoboa integrates with well known software such as Postfix or Dovecot. A SQL database (MySQL, PostgreSQL or SQLite) is used as a central point of communication between all components. Modoboa is developed with modularity in mind, expanding it is really easy. -
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
10
OAuthLib
A generic, spec-compliant, thorough implementation of the OAuth
A generic, spec-compliant, thorough implementation of the OAuth request-signing logic for Python 3.8+. OAuthLib is a framework which implements the logic of OAuth1 or OAuth2 without assuming a specific HTTP request object or web framework. Use it to graft OAuth client support onto your favorite HTTP library, or provide support onto your favourite web framework. If you're a maintainer of such a library, write a thin veneer on top of OAuthLib and get OAuth support for very little effort. -
11
Weblate
Web based localization tool with tight version control integration
Weblate is a copylefted libre software web-based continuous localization system, used by over 2500 libre projects and companies in more than 165 countries. Copylefted libre software, used by over 2,500 libre software projects and companies in over 165 countries. Hosted service and standalone tool with tight version control integration. Simple and clean user interface, propagation of translations across components, quality checks and automatic linking to source files. There is infrastructure... -
12
Speakr
Speakr is a personal, self-hosted web application
Speakr is an open-source, real-time text-to-speech (TTS) web application that allows users to convert written text into natural-sounding speech in just a few clicks. It provides a clean, user-friendly interface where users can input text, choose a voice style or language, and immediately hear the output, making it ideal for accessibility, content creation, and learning applications. Behind the scenes, Speakr leverages modern TTS engines and streaming audio technologies to deliver smooth and... -
13
newspaper4k
Python library for scraping and analyzing online news articles easily
Newspaper4k is a Python library designed for extracting, processing, and analyzing news articles from websites. It is a continuation and active fork of the original newspaper3k library, which had stopped receiving updates, with the goal of keeping the ecosystem maintained while adding improvements and bug fixes. It provides developers with tools to automatically download web pages, extract the main article content, and collect associated metadata such as titles, authors, images, and publication dates. ... -
14
Meli-Action
Download files from direct links, YouTube, Telegram, Google Play
Meli-Action is a GitHub Actions-based downloading and web archiving project designed for heavily restricted internet environments. It uses GitHub-hosted workflow runners to fetch content from direct links, YouTube, Telegram, Google Play, SoundCloud, and web pages. Downloaded files are committed into the user’s repository so they can be retrieved later through normal GitHub access. The project includes workflows for different content sources, plus a Python script that can render and save web pages as MHTML archives using a headless browser. ... -
15
FastUI
Build better UIs faster
FastUI is a library that lets developers build interactive user interfaces for FastAPI applications using Pydantic models. It automatically generates frontend components based on data schemas and endpoint logic, reducing the need for manual UI development. Designed to be type-safe, reactive, and fast, FastUI streamlines the creation of web dashboards, admin panels, and internal tools within a FastAPI backend. -
16
LuxTTS
A high-quality rapid TTS voice cloning model
...The project supports zero-shot voice cloning, meaning it can adapt to a reference speaker’s voice with minimal example data, enabling realistic and personalized synthetic speech. Intended for developers, hobbyists, and creators, the repository includes installation instructions, usage examples, and Python APIs that make it feasible to integrate the model in local workflows, web demos, or production systems. Its design emphasizes efficiency and practicality, fitting within modest GPU memory footprints. -
17
reg-factory
Google Mail registration
reg-factory is an automation-oriented account registration pipeline aimed at research, testing, and controlled platform workflow experiments. It combines browser automation, environment-based configuration, and a local web control panel to run multi-step registration scripts. The project includes modules for email registration, platform sign-up flows, token handling, cookie export, and account workflow management. It uses local scripts and external tool integrations to coordinate browser... -
18
Websoft9
Applications self-hosting and DevOps platform
Websoft9 is a web-based PaaS and Linux panel for self-hosting open-source applications on a user’s own server. It is designed for running many web apps on a single machine instead of requiring a complex Kubernetes-first architecture. The platform provides a graphical interface for installing, managing, and operating more than 200 application templates. It combines application deployment with server administration tools, so users can manage files, containers, users, proxies, SSL, logs,... -
19
Composio
Composio equip's your AI agents & LLMs
Empower your AI agents with Composio - a platform for managing and integrating tools with LLMs & AI agents using Function Calling. Equip your agent with high-quality tools & integrations without worrying about authentication, accuracy, and reliability in a single line of code. -
20
OpenWPM
A web privacy measurement framework
OpenWPM is a web privacy measurement framework that makes it easy to collect data for privacy studies on a scale of thousands to millions of websites. OpenWPM is built on top of Firefox, with automation provided by Selenium. It includes several hooks for data collection. Check out the instrumentation section below for more details. OpenWPM is tested on Ubuntu 18.04 via TravisCI and is commonly used via the docker container that this repo builds, which is also based on Ubuntu. Although we... -
21
django-viewflow
Reusable workflow library for Django
Viewflow is a lightweight reusable workflow library that helps to organize people collaboration business logic in Django applications. In conjunction with Django-material, they could be used as the framework to build ready-to-use business applications in minutes. Django web framework solves only technical problems related to the client-server interaction on top of the stateless HTTP protocol. Model-View-Template separation pattern helps to maintain simple CRUD-based logic. Viewflow is the... -
22
Snoop Project
This is the most powerful software taking into account CIS location
Snoop is an open data intelligence tool (OSINT world). Snoop Project is one of the most promising OSINT tools for finding nicknames. This is the most powerful software taking into account the CIS location. Is your life slideshow? Ask Snoop. Snoop project is developed without taking into account the opinions of the NSA and their friends, that is, it is available to the average user. Snoop is a research work (own database / closed bugbounty) in the field of searching and processing public data... -
23
Frontend Slides
Create beautiful slides on the web using Claude's frontend skills
Frontend Slides is a lightweight tool that enables users to create visually appealing, animation-rich web presentations without requiring knowledge of CSS or JavaScript by leveraging a guided, interactive workflow. It operates on a “show, don’t tell” philosophy, generating visual previews of styles so users can select their preferred design rather than describing it abstractly. The system produces fully self-contained HTML presentations with inline CSS and JavaScript, eliminating the need... -
24
Gradio
Create UIs for your machine learning model in Python in 3 minutes
Gradio is the fastest way to demo your machine learning model with a friendly web interface so that anyone can use it, anywhere! Gradio can be installed with pip. Creating a Gradio interface only requires adding a couple lines of code to your project. You can choose from a variety of interface types to interface your function. Gradio can be embedded in Python notebooks or presented as a webpage. A Gradio interface can automatically generate a public link you can share with colleagues that lets them interact with the model on your computer remotely from their own devices. ... -
25
Fli
Google Flights MCP and Python Library
Fli is a powerful Python library and command-line tool that provides direct programmatic access to Google Flights data through reverse-engineered API interactions rather than traditional web scraping. This approach enables faster, more reliable, and more stable access to flight information, avoiding the fragility associated with HTML parsing and UI changes.
