Search Results for "python web crawler"
Sort By:
Showing 239 open source projects for "python web crawler"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Stop vibe-debugging.AppSignal's MCP server hands Claude, Cursor, or Zed your real errors, traces, and the deploy that shipped them. AI writes the fix; you review the diff.
-
1
Web RPA
Web Robotics Process Automation Tool
Web RPA is a browser automation framework designed to perform robotic process automation tasks directly within web environments. It enables users to automate repetitive actions such as form filling, data extraction, and workflow execution through programmable scripts. The system focuses on simplicity and flexibility, allowing automation without requiring complex infrastructure. It supports interaction with web elements, navigation flows, and dynamic content handling, making it suitable for... -
2
python-socketio
Python Socket.IO server and client
python-socketio is a robust Python library that implements the Socket.IO protocol, enabling real-time, bidirectional communication between web clients and servers. It works with multiple asynchronous frameworks such as asyncio, eventlet, and gevent, so developers can choose the concurrency model that best fits their application needs while still using a consistent API. -
3
MediaCrawler
Self-media platform crawler
MediaCrawler is an open-source tool for collecting public content from major Chinese social media platforms. It supports Xiaohongshu, Douyin, Kuaishou, Bilibili, Weibo, Tieba, Zhihu, and other services. The project uses Playwright to automate browser login and preserve authenticated sessions. It obtains required request signatures from the active browser context, avoiding complex JavaScript reverse engineering. Users can search by keyword, crawl specific posts, collect nested comments, and... -
4
CadQuery
A python parametric CAD scripting framework based on OCCT
...Provide a non-proprietary, plain text model format that can be edited and executed with only a web browser. The scripts use a standard programming language, Python, and thus can benefit from the associated infrastructure. This includes many standard libraries and IDEs. CadQuery's CAD kernel Open CASCADE Technology (OCCT) is much more powerful than the CGAL used by OpenSCAD. Features supported natively by OCCT include NURBS, splines, surface sewing, STL repair, STEP import/export, and other complex operations. -
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
5
Blender GIS
Blender addons to make the bridge between Blender and geographic data
Import in Blender most commons GIS data format, Shapefile vector, raster image, geotiff DEM, OpenStreetMap XML. There are a lot of possibilities to create a 3D terrain from geographic data with BlenderGIS, check the Flowchart to have an overview. Display dynamics web maps inside Blender 3d view, requests for OpenStreetMap data (buildings, roads, etc.), get true elevation data from the NASA SRTM mission. Manage georeferencing information of a scene, compute a terrain mesh by Delaunay... -
6
ReClip
Download videos from almost any website
...Its backend is implemented in a compact Python Flask application, making it easy to deploy and customize. Users can paste multiple URLs at once, select output formats such as MP4 or MP3, and choose quality settings before downloading. The system also includes features like automatic URL deduplication and batch processing to improve usability. -
7
Windrecorder
Windrecorder is a memory search app by records everything
Windrecorder is an open-source personal memory search engine that continuously records on-screen activity in a highly optimized and storage-efficient format. It captures screen content locally and builds a searchable database using OCR and image understanding, allowing users to rewind and rediscover anything they have previously seen. The system indexes only meaningful visual changes, extracting text, browser data, and contextual information to improve search accuracy and reduce storage... -
8
Mopidy
Mopidy is an extensible music server written in Python
Mopidy plays music from local disk, Spotify, SoundCloud, TuneIn, and more. You can edit the playlist from any phone, tablet, or computer using a variety of MPD and web clients. Vanilla Mopidy only plays music from files and radio streams. Through extensions, Mopidy can play music from cloud services like Spotify, SoundCloud, and TuneIn. With Mopidy's extension support, you can easily add backends for new music sources. Mopidy is a Python application that runs in a terminal or in the background on Linux computers or Macs that have network connectivity and audio output. ... -
9
yt-dlp
A youtube-dl fork with additional features and fixes
yt-dlp is a youtube-dl fork based on the now inactive youtube-dlc. The main focus of this project is adding new features and patches while also keeping up to date with the original project -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
10
You-Get
Dumb downloader that scrapes the web
You-Get is a small command-line utility for downloading media (video, audio and images) from the Web when there are no other means to do so. It can download video and audio files from such popular web sites as YouTube, Twitter, Niconico, Vimeo, Flickr, Instagram and a whole lot more. You-Get is a great option for when you want to enjoy your favorite videos, audio or images from the internet without having to open any web browsers or get interrupted by ads. It’s also a good choice for... -
11
FFmpeg Quality Metrics
Calculate quality metrics with FFmpeg (SSIM, PSNR, VMAF, VIF)
...It also includes optional visualization features through an interactive web-based dashboard for analyzing results graphically. Designed for cross-platform use, it integrates seamlessly into automated workflows and supports JSON or CSV output formats for further analysis. This makes it particularly valuable for engineers working on video compression and streaming systems. -
12
armory
3D Engine with Blender Integration
Armory is an open-source 3D engine focused on portability, minimal footprint and performance. The renderer is fully scriptable with deferred and forward paths supported out of the box. Armory provides a full Blender integration add-on, turning it into a complete game development tool. The result is a unified workflow from start to finish. Powered by Armory engine, ArmorPaint is a stand-alone software designed for physically-based texture painting. Drag & drop your 3D models and start... -
13
DouK-Downloader
TikTok releases/likes/compilations/live streams/videos/atlases/music
...It supports automated workflows like incremental downloads, duplicate detection, and persistent storage of collected data in formats like CSV, XLSX, and SQLite, making it useful for both content archiving and analytics. The project also includes a Web UI interface and API support, allowing remote access, automation, and integration into larger systems or private servers. -
14
Bazarr
Bazarr is a companion application to Sonarr and Radarr
Bazarr is a companion application for media automation that works alongside the popular Sonarr and Radarr tools to manage and download subtitles for TV shows and movies based on user-defined preferences. Once you configure the languages and quality rules for your media library, Bazarr continuously monitors new episodes or releases indexed by those applications and automatically finds and fetches matching subtitle files, including external upgrades when better options appear. It offers both... -
15
StreamCap
Multi-Platform Live Stream Automatic Recording Tool
StreamCap is a multi-platform live stream recording client that enables automated capture and management of live broadcasts from over 40 streaming platforms worldwide. Built on FFmpeg and StreamGet, it provides a robust system for monitoring live stream availability and automatically starting recordings when streams go online. The software supports batch recording and scheduled monitoring, allowing users to manage multiple streams simultaneously without manual intervention. It offers a wide... -
16
asciinema
Open source terminal session recorder
asciinema is a free and open source terminal session recorder. It lets you easily record and play back terminal sessions in the terminal or in a web browser. Forget old screen recording methods and resulting blurry videos. asciinema lets you record your terminal sessions the right way, which is right where you work, in the terminal. Recording is as easy as running one command, and since it’s purely text-based you can copy and paste any content you want, simply pause the recording! You... -
17
Speakr
Speakr is a personal, self-hosted web application
Speakr is an open-source, real-time text-to-speech (TTS) web application that allows users to convert written text into natural-sounding speech in just a few clicks. It provides a clean, user-friendly interface where users can input text, choose a voice style or language, and immediately hear the output, making it ideal for accessibility, content creation, and learning applications. Behind the scenes, Speakr leverages modern TTS engines and streaming audio technologies to deliver smooth and... -
18
screenshot-to-code
Drop in a screenshot and convert it to clean code
screenshot-to-code converts UI screenshots or design images into working front-end code, accelerating the path from concept to prototype. It uses modern vision-capable or code-generating models to infer layout structure, typography, and components, then outputs clean HTML/CSS (often Tailwind) or framework code. A web interface lets you upload images, tune options, and preview generated results, while a backend service orchestrates the model calls and post-processing. The tool focuses on... -
19
Listen 1
One for all free music in china (chrome extension)
...Select "Load unpacked extension..." and select the folder you just unpacked. Download the Windows zip file and choose the 32-bit or 64-bit version according to the system. The original web player, using Python to develop a web server. Can run directly on the server, or use the packaged Windows and Mac versions to run the web server locally. Windows, Mac, Linux desktop. Using Electron framework, based on Listen 1 Chrome plug-in version JS library development. -
20
cloud-morph
Decentralize, Self-host Cloud Gaming/Application
cloud-morph is a cloud-based media processing service that enables real-time video and audio transformation using FFmpeg in scalable environments. It is designed to run as a backend service that processes media streams or files and applies transformations such as transcoding, filtering, and format conversion. The system supports API-driven workflows, allowing integration into web applications or automation pipelines. It leverages containerization and distributed architecture to handle... -
21
8mb.local
Ffree local self hosted video compressor webui
8mb.local is a lightweight, self-hosted local network status dashboard and utilities hub designed to help users monitor devices and services connected to their home or small business network. The platform presents a simple yet informative web dashboard that dynamically updates with device statuses, IP addresses, and latency information, giving users a centralized view of network health without complex dashboards. It also includes basic tools such as ping monitoring, service reachability... -
22
iCn3D
Web-based protein structure viewer and analysis tool
"I see in 3D" (iCn3D) Structure Viewer is not only a web-based 3D viewer, but also a structure analysis tool interactively or in the batch mode using NodeJS scripts based on the npm package icn3d. iCn3D synchronizes the display of 3D structure, 2D interaction, and 1D sequences and annotations. Users' custom displays can be saved in a short URL or a PNG image. The complete package of iCn3D including Three.js and jQuery is in the directory "dist" after you get the source code with the "Code"... -
23
This project has evolved from its original version into Calimero (a new Windows desktop wallpaper manager). Details here: https://sourceforge.net/projects/kalimero/ WallPaper (alias crawlpaper) is a desktop changer (NOT a screensaver) which includes a web crawler for picture download, an audio stream ripper, an audio player, a mini mp3 tag editor,etc. Also included support for .zip and .rar files and an interface to the BerkleyDB code for small databases.
-
24
Kemono Downloader
Kemono Downloader - A cross-platform Python app built with PyQt6
Welcome to Kemono Downloader, a versatile Python-based desktop application built with PyQt6, designed to download content from Kemono.su. This tool enables users to archive individual posts or entire creator profiles from services like Patreon, Fanbox, and more, supporting a wide range of file types with customizable settings and advanced features. -
25
YehDown
Yeahdown: Easy-to-use video downloader for Windows
Yeahdown is a straightforward, user-friendly Windows-based application designed to simplify the process of downloading videos and audio from popular websites like YouTube and Vimeo. Perfect for non-technical users, it offers an intuitive interface and fast, reliable downloads. Key features include improved download speeds, support for multiple major video platforms, and real-time updates for new features. Tested on windows 11.
