Search Results for "python web crawler"
Sort By:
Showing 229 open source projects for "python web crawler"
View related business solutions-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
1
Web RPA
Web Robotics Process Automation Tool
Web RPA is a browser automation framework designed to perform robotic process automation tasks directly within web environments. It enables users to automate repetitive actions such as form filling, data extraction, and workflow execution through programmable scripts. The system focuses on simplicity and flexibility, allowing automation without requiring complex infrastructure. It supports interaction with web elements, navigation flows, and dynamic content handling, making it suitable for... -
2
python-socketio
Python Socket.IO server and client
python-socketio is a robust Python library that implements the Socket.IO protocol, enabling real-time, bidirectional communication between web clients and servers. It works with multiple asynchronous frameworks such as asyncio, eventlet, and gevent, so developers can choose the concurrency model that best fits their application needs while still using a consistent API. -
3
MediaCrawler
Self-media platform crawler
MediaCrawler is an open-source tool for collecting public content from major Chinese social media platforms. It supports Xiaohongshu, Douyin, Kuaishou, Bilibili, Weibo, Tieba, Zhihu, and other services. The project uses Playwright to automate browser login and preserve authenticated sessions. It obtains required request signatures from the active browser context, avoiding complex JavaScript reverse engineering. Users can search by keyword, crawl specific posts, collect nested comments, and... -
4
ReClip
Download videos from almost any website
...Its backend is implemented in a compact Python Flask application, making it easy to deploy and customize. Users can paste multiple URLs at once, select output formats such as MP4 or MP3, and choose quality settings before downloading. The system also includes features like automatic URL deduplication and batch processing to improve usability. -
Paessler: Easy to Use With Enterprise Power. Free TrialYou shouldn't have to choose between a monitoring tool that's easy to use and one that's powerful enough for a complex environment. PRTG's low-code interface lets any admin build dashboards, set alerts and monitor devices without scripting, while custom sensors and full API access are there when your team needs deeper control. One platform, no compromise. Download a free 30-day trial now.
-
5
CadQuery
A python parametric CAD scripting framework based on OCCT
...Provide a non-proprietary, plain text model format that can be edited and executed with only a web browser. The scripts use a standard programming language, Python, and thus can benefit from the associated infrastructure. This includes many standard libraries and IDEs. CadQuery's CAD kernel Open CASCADE Technology (OCCT) is much more powerful than the CGAL used by OpenSCAD. Features supported natively by OCCT include NURBS, splines, surface sewing, STL repair, STEP import/export, and other complex operations. -
6
Blender GIS
Blender addons to make the bridge between Blender and geographic data
Import in Blender most commons GIS data format, Shapefile vector, raster image, geotiff DEM, OpenStreetMap XML. There are a lot of possibilities to create a 3D terrain from geographic data with BlenderGIS, check the Flowchart to have an overview. Display dynamics web maps inside Blender 3d view, requests for OpenStreetMap data (buildings, roads, etc.), get true elevation data from the NASA SRTM mission. Manage georeferencing information of a scene, compute a terrain mesh by Delaunay... -
7
Windrecorder
Windrecorder is a memory search app by records everything
Windrecorder is an open-source personal memory search engine that continuously records on-screen activity in a highly optimized and storage-efficient format. It captures screen content locally and builds a searchable database using OCR and image understanding, allowing users to rewind and rediscover anything they have previously seen. The system indexes only meaningful visual changes, extracting text, browser data, and contextual information to improve search accuracy and reduce storage... -
8
DouK-Downloader
TikTok releases/likes/compilations/live streams/videos/atlases/music
...It supports automated workflows like incremental downloads, duplicate detection, and persistent storage of collected data in formats like CSV, XLSX, and SQLite, making it useful for both content archiving and analytics. The project also includes a Web UI interface and API support, allowing remote access, automation, and integration into larger systems or private servers. -
9
FFmpeg Quality Metrics
Calculate quality metrics with FFmpeg (SSIM, PSNR, VMAF, VIF)
...It also includes optional visualization features through an interactive web-based dashboard for analyzing results graphically. Designed for cross-platform use, it integrates seamlessly into automated workflows and supports JSON or CSV output formats for further analysis. This makes it particularly valuable for engineers working on video compression and streaming systems. -
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
10
Speakr
Speakr is a personal, self-hosted web application
Speakr is an open-source, real-time text-to-speech (TTS) web application that allows users to convert written text into natural-sounding speech in just a few clicks. It provides a clean, user-friendly interface where users can input text, choose a voice style or language, and immediately hear the output, making it ideal for accessibility, content creation, and learning applications. Behind the scenes, Speakr leverages modern TTS engines and streaming audio technologies to deliver smooth and... -
11
EBOOK ETC
A treasure trove of e-book downloads
EBOOK ETC is a large Chinese-language index of ebook download links collected from across the web. It organizes books by subject, author, and commonly used descriptive tags. The collection spans literature, history, science, business, technology, philosophy, art, and many other categories. Entries commonly provide links for EPUB, MOBI, and AZW3 editions so readers can choose formats for different devices. A GitHub Pages interface adds real-time title, author, category, and multi-keyword... -
12
Mopidy
Mopidy is an extensible music server written in Python
Mopidy plays music from local disk, Spotify, SoundCloud, TuneIn, and more. You can edit the playlist from any phone, tablet, or computer using a variety of MPD and web clients. Vanilla Mopidy only plays music from files and radio streams. Through extensions, Mopidy can play music from cloud services like Spotify, SoundCloud, and TuneIn. With Mopidy's extension support, you can easily add backends for new music sources. Mopidy is a Python application that runs in a terminal or in the background on Linux computers or Macs that have network connectivity and audio output. ... -
13
yt-dlp
A youtube-dl fork with additional features and fixes
yt-dlp is a youtube-dl fork based on the now inactive youtube-dlc. The main focus of this project is adding new features and patches while also keeping up to date with the original project -
14
Jellyfin MPV Shim
MPV-Based Jellyfin Client with Offline Sync
Jellyfin MPV Shim is a free, open-source media client that combines Jellyfin server integration with the MPV playback engine. It allows users to browse and play movies, television shows, music, photos, live television, comics, and audiobooks. The application preserves MPV's native rendering pipeline to support HDR, Dolby Vision, and advanced video processing. Users can download media for offline playback and synchronize viewing sessions through Jellyfin SyncPlay. Its casting functionality... -
15
You-Get
Dumb downloader that scrapes the web
You-Get is a small command-line utility for downloading media (video, audio and images) from the Web when there are no other means to do so. It can download video and audio files from such popular web sites as YouTube, Twitter, Niconico, Vimeo, Flickr, Instagram and a whole lot more. You-Get is a great option for when you want to enjoy your favorite videos, audio or images from the internet without having to open any web browsers or get interrupted by ads. It’s also a good choice for... -
16
StreamCap
Multi-Platform Live Stream Automatic Recording Tool
StreamCap is a multi-platform live stream recording client that enables automated capture and management of live broadcasts from over 40 streaming platforms worldwide. Built on FFmpeg and StreamGet, it provides a robust system for monitoring live stream availability and automatically starting recordings when streams go online. The software supports batch recording and scheduled monitoring, allowing users to manage multiple streams simultaneously without manual intervention. It offers a wide... -
17
Bazarr
Bazarr is a companion application to Sonarr and Radarr
Bazarr is a companion application for media automation that works alongside the popular Sonarr and Radarr tools to manage and download subtitles for TV shows and movies based on user-defined preferences. Once you configure the languages and quality rules for your media library, Bazarr continuously monitors new episodes or releases indexed by those applications and automatically finds and fetches matching subtitle files, including external upgrades when better options appear. It offers both... -
18
8mb.local
Ffree local self hosted video compressor webui
8mb.local is a lightweight, self-hosted local network status dashboard and utilities hub designed to help users monitor devices and services connected to their home or small business network. The platform presents a simple yet informative web dashboard that dynamically updates with device statuses, IP addresses, and latency information, giving users a centralized view of network health without complex dashboards. It also includes basic tools such as ping monitoring, service reachability... -
19
asciinema
Open source terminal session recorder
asciinema is a free and open source terminal session recorder. It lets you easily record and play back terminal sessions in the terminal or in a web browser. Forget old screen recording methods and resulting blurry videos. asciinema lets you record your terminal sessions the right way, which is right where you work, in the terminal. Recording is as easy as running one command, and since it’s purely text-based you can copy and paste any content you want, simply pause the recording! You... -
20
Armory
3D Engine with Blender Integration
Armory is an open-source 3D engine focused on portability, minimal footprint and performance. The renderer is fully scriptable with deferred and forward paths supported out of the box. Armory provides a full Blender integration add-on, turning it into a complete game development tool. The result is a unified workflow from start to finish. Powered by Armory engine, ArmorPaint is a stand-alone software designed for physically-based texture painting. Drag & drop your 3D models and start... -
21
Fanqie-novel-Downloader
Tomato Novel Downloader | Supports Windows/macOS/Linux/android
Fanqie Novel Downloader is a free novel downloading and reading application designed for accessing content from the Fanqie platform. It allows users to search by title, author, book ID, or URL. Novels can be downloaded in TXT or EPUB format, with support for selected chapters and batch operations. Its integrated reader offers scrolling, page turning, customizable themes, and automatic reading. Users can organize personal bookshelves and synchronize reading progress with a Fanqie account. The... -
22
cloud-morph
Decentralize, Self-host Cloud Gaming/Application
cloud-morph is a cloud-based media processing service that enables real-time video and audio transformation using FFmpeg in scalable environments. It is designed to run as a backend service that processes media streams or files and applies transformations such as transcoding, filtering, and format conversion. The system supports API-driven workflows, allowing integration into web applications or automation pipelines. It leverages containerization and distributed architecture to handle... -
23
Oil Motion
Create smooth, responsive interactive web animations
Oil Motion is an agent-agnostic skill for creating smooth interactive animations for websites and digital interfaces. Users describe the desired movement, available assets, and the interaction that should control it while the agent manages the production workflow. The system defines key states, generates continuous motion between them, reviews frames, removes artifacts, and prepares optimized assets. Animation progress can then be mapped to scrolling, mouse movement, dragging, touch input,... -
24
screenshot-to-code
Drop in a screenshot and convert it to clean code
screenshot-to-code converts UI screenshots or design images into working front-end code, accelerating the path from concept to prototype. It uses modern vision-capable or code-generating models to infer layout structure, typography, and components, then outputs clean HTML/CSS (often Tailwind) or framework code. A web interface lets you upload images, tune options, and preview generated results, while a backend service orchestrates the model calls and post-processing. The tool focuses on... -
25
Kemono Downloader
Kemono Downloader - A cross-platform Python app built with PyQt6
Welcome to Kemono Downloader, a versatile Python-based desktop application built with PyQt6, designed to download content from Kemono.su. This tool enables users to archive individual posts or entire creator profiles from services like Patreon, Fanbox, and more, supporting a wide range of file types with customizable settings and advanced features.
