Search Results for "python web crawler" - Page 19
Sort By:
Showing 3076 open source projects for "python web crawler"
View related business solutions-
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
Go from Code to Production URL in SecondsSkip the Kubernetes configs. Cloud Run handles HTTPS, scaling, and infrastructure automatically. Two million requests free per month.
-
1
Qwen-2.5-VL
Qwen2.5-VL is the multimodal large language model series
Qwen2.5 is a series of large language models developed by the Qwen team at Alibaba Cloud, designed to enhance natural language understanding and generation across multiple languages. The models are available in various sizes, including 0.5B, 1.5B, 3B, 7B, 14B, 32B, and 72B parameters, catering to diverse computational requirements. Trained on a comprehensive dataset of up to 18 trillion tokens, Qwen2.5 models exhibit significant improvements in instruction following, long-text generation... -
2
VideoChat
Real-time voice interactive digital human
...It supports both pure end-to-end voice solutions based on multimodal large language models (GLM-4-Voice feeding directly into talking-head generation) and a more traditional cascaded pipeline using ASR → LLM → TTS → talking head. It is built as a Gradio Python demo, exposing a web interface where users can talk to an animated avatar that lip-syncs to synthesized speech while responding intelligently. The system is customizable: you can define your own avatar appearance and voice, and it supports voice cloning so you can generate a new voice from a short 3–10 second reference sample. The tech stack integrates FunASR for speech recognition, Qwen for language understanding, multiple TTS engines like GPT-SoVITS, CosyVoice, or edge-tts, and MuseTalk for talking-head generation. -
3
Universal Commerce Protocol (UCP)
The common language for platforms, agents and businesses.
Universal Commerce Protocol (UCP) is an open standard designed to unify how platforms, businesses, and payment providers interact across the modern commerce ecosystem. It provides a common language that eliminates fragmented, custom integrations and enables seamless interoperability between diverse commerce systems. Built for an increasingly agentic web, UCP supports AI-driven platforms that can discover products, manage carts, and complete transactions securely on a user’s behalf. Its... -
4
HunyuanWorld 1.0
Generating Immersive, Explorable, and Interactive 3D Worlds
HunyuanWorld-1.0 is an open-source, simulation-capable 3D world generation model developed by Tencent Hunyuan that creates immersive, explorable, and interactive 3D environments from text or image inputs. It combines the strengths of video-based diversity and 3D-based geometric consistency through a novel framework using panoramic world proxies and semantically layered 3D mesh representations. This approach enables 360° immersive experiences, seamless mesh export for graphics pipelines, and... -
Build Securely on AWS with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
5
reNgine
Automated framework for web application reconnaissance and scanning
reNgine is an automated reconnaissance framework designed to simplify and enhance the process of gathering information about web applications during security assessments. It provides a streamlined workflow for penetration testers, bug bounty hunters, and security teams who need to perform reconnaissance efficiently and at scale. The platform integrates multiple open-source reconnaissance tools into a unified environment with a configurable scanning engine and an intuitive web interface.... -
6
myGPTReader
AI Slack bot for reading, summarizing, and chatting with content
myGPTReader is an AI-powered Slack bot designed to help users read, summarize, and interact with various types of digital content through conversational interfaces. It enables users to quickly understand web pages, documents, and even video content by transforming them into interactive discussions rather than static reading experiences. myGPTReader supports a wide range of file formats, including eBooks, PDFs, and text-based documents, making it flexible for both casual and professional use... -
7
Conversational Health Agents (CHA)
A Personalized LLM-powered Agent Frameworks
CHA, or Conversational Health Agents, is an open-source framework designed to build intelligent healthcare assistants powered by large language models and external data sources. The system enables developers to create personalized AI agents that can interact with users through natural language while performing multi-step reasoning and task execution. It integrates orchestration capabilities that allow the agent to gather information from APIs, knowledge bases, and external services in order... -
8
GitDiagram
AI tool that converts GitHub repositories into interactive diagrams
GitDiagram is an open source web application designed to help developers quickly understand the structure and architecture of GitHub repositories by automatically generating interactive diagrams. It analyzes repository metadata such as the file tree and project documentation to build a visual representation of how different components of a project relate to one another. It uses an AI-powered pipeline to interpret repository structure and transform that information into system design diagrams... -
9
StarVector
StarVector is a foundation model for SVG generation
StarVector is a multimodal foundation model designed for generating Scalable Vector Graphics (SVG) from images or textual descriptions. The system treats vector graphics creation as a code generation problem, producing SVG code that can render detailed vector images. Its architecture combines computer vision techniques with language modeling capabilities so it can understand visual inputs and textual prompts simultaneously. The model converts raster images or text instructions into... -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
10
AWS Agent Skills
AWS Skills for Agents
AWS Agent Skills is a repository that curates AWS-focused agent skills — capability modules that give AI assistants like Claude Code and Codex deep, practical knowledge across key Amazon Web Services domains. Instead of streaming giant documentation sets or relying on episodic web search, this project compresses AWS best practices, usage patterns, edge cases, and real-world engineering guides into pre-structured skill definitions that are token-efficient and tailored for reasoning. The... -
11
HunyuanDiT
Diffusion Transformer with Fine-Grained Chinese Understanding
HunyuanDiT is a high-capability text-to-image diffusion transformer with bilingual (Chinese/English) understanding and multi-turn dialogue capability. It trains a diffusion model in latent space using a transformer backbone and integrates a Multimodal Large Language Model (MLLM) to refine captions and support conversational image generation. It supports adapters like ControlNet, IP-Adapter, LoRA, and can run under constrained VRAM via distillation versions. LoRA, ControlNet (pose, depth,... -
12
JobFunnel
Scrape job websites into a single spreadsheet with no duplicates.
Scrape job websites into a single spreadsheet with no duplicates. Automated tool for scraping job postings into a .csv file. You can search for jobs with YAML configuration files or by passing command arguments. By performing regular scraping and reviewing, you can cut through the noise of even the busiest job markets. Run funnel with your settings YAML to populate your master CSV file with jobs from available providers. JobFunnel can be easily automated to run nightly with crontab. If you... -
13
wger
Self hosted FLOSS fitness/workout, nutrition and weight tracker
wger Workout Manager is a free and open web application that manages your exercises, routines and nutrition. It started out as a personal project to replace my growing collection of spreadsheets but has turned into something that other people may find useful. You can create and manage flexible training routines for whatever goals you have. Select exactly what exercises you are going to do and how many repetitions, time or distance you want to do. You can also combine different workouts in... -
14
NeuTTS Air
NeuTTS model built from small LLM backbones
NeuTTS Air is an open-source collection of on-device text-to-speech speech language models from Neuphonic. It is built for natural-sounding voice generation that can run locally instead of relying on a remote web API. The project emphasizes instant voice cloning, real-time performance, and deployment on smaller devices such as phones, laptops, and Raspberry Pi-class hardware. Its LLM-based architecture is intended to bring more expressive and flexible speech generation to local applications.... -
15
NeuTTS Nano
On-device TTS model by Neuphonic
NeuTTS Nano is an open-source collection of on-device text-to-speech speech language models from Neuphonic. It is built for natural-sounding voice generation that can run locally instead of relying on a remote web API. The project emphasizes instant voice cloning, real-time performance, and deployment on smaller devices such as phones, laptops, and Raspberry Pi-class hardware. Its LLM-based architecture is intended to bring more expressive and flexible speech generation to local... -
16
StreamSpeech
StreamSpeech is a seamless model for offline speech recognition
StreamSpeech is an “all-in-one” speech model designed to perform offline and simultaneous speech recognition, speech translation, and speech synthesis within a single unified architecture. Developed as part of an ACL 2024 paper, it targets streaming and low-latency scenarios where intermediate results and final translations or synthetic speech must be produced continuously as audio is being received. The model supports eight tasks: offline ASR, speech-to-text translation, speech-to-speech... -
17
Android Emulator Container Scripts
Minimal scripts to run the emulator in a container for various systems
android-emulator-container-scripts turns the Android Emulator into a cloud-native service you can run in Docker and Kubernetes, so teams can provision ephemeral Android devices on demand. It includes scripts and container images that configure the emulator for headless operation, wire up networking, and expose endpoints for ADB and web access. A built-in WebRTC bridge lets you stream the emulator screen to a browser with interactive input, which is ideal for CI dashboards, remote debugging,... -
18
MOSS-TTS-Nano
MOSS-TTS-Nano is an open-source multilingual tiny speech generation
MOSS-TTS-Nano is a lightweight text-to-speech model designed for real-time voice generation in resource-constrained environments. It is part of the broader MOSS-TTS family and focuses on delivering high-quality speech synthesis with a compact architecture. The model operates efficiently on CPU-only systems, enabling deployment without specialized hardware. It supports multilingual voice cloning and produces high-fidelity audio with low latency. The system uses an autoregressive audio... -
19
shot-scraper
A command-line utility for taking automated screenshots of websites
shot-scraper is a command-line utility for taking automated screenshots of web pages using a headless browser engine. After installation, a single command can capture a full-page screenshot of a URL and save it to a file, making it ideal for documentation, monitoring, and visual regression tasks. Under the hood it uses a modern browser (installed via a one-time shot-scraper install step) and exposes options for viewport size, full-page versus clipped screenshots, and device emulation. Beyond... -
20
Papis
Powerful and highly extensible command-line based document
Papis is a powerful and highly extensible CLI document and bibliography manager. With Papis, you can search your library for books and papers, add documents and notes, import and export to and from other formats, and much much more. Papis uses a human-readable and easily hackable .yaml file to store each entry's bibliographical data. It strives to be easy to use while providing a wide range of features. And for those who still want more, Papis makes it easy to write scripts that extend its... -
21
cloud-morph
Decentralize, Self-host Cloud Gaming/Application
cloud-morph is a cloud-based media processing service that enables real-time video and audio transformation using FFmpeg in scalable environments. It is designed to run as a backend service that processes media streams or files and applies transformations such as transcoding, filtering, and format conversion. The system supports API-driven workflows, allowing integration into web applications or automation pipelines. It leverages containerization and distributed architecture to handle... -
22
Build Your Own OpenClaw
A step-by-step guide to build your own AI agent
Build Your Own OpenClaw is a step-by-step educational framework that teaches developers how to construct a fully functional AI agent system from scratch, gradually evolving from a simple chat loop into a multi-agent, production-ready architecture. The project is structured into 18 progressive stages, each introducing a new concept such as tool usage, memory persistence, event-driven design, and multi-agent coordination, with each step including both explanatory documentation and runnable... -
23
Pal
A personal context-agent that learns how you work
Pal is an open-source AI personal agent built within the Agno ecosystem that functions as an intelligent digital assistant designed to learn from user activity over time. The system acts as an AI-powered “second brain” capable of capturing, organizing, and retrieving personal knowledge such as notes, bookmarks, research findings, people, and meeting information. Instead of acting as a simple chatbot, Pal continuously builds a structured database of a user’s knowledge and context so it can... -
24
PokeeResearch-7B
Pokee Deep Research Model Open Source Repo
PokeeResearchOSS provides an open-source, agentic “deep research” model centered on a 7B backbone that can browse, read, and synthesize current information from the web. Instead of relying only on static training data, the agent performs searches, visits pages, and extracts evidence before forming answers to complex queries. It is built to operate end-to-end: planning a research strategy, gathering sources, reasoning over conflicting claims, and writing a grounded response. The repository... -
25
Letterboxd Recommendations
Scraping publicly-accessible Letterboxd data for movie recommendations
Scraping publicly-accessible Letterboxd data and creating a movie recommendation model with it that can generate recommendations when provided with a Letterboxd username. A user's "star" ratings are scraped from their Letterboxd profile and assigned numerical ratings from 1 to 10 (accounting for half stars). Their ratings are then combined with a sample of ratings from the top 4000 most active users on the site to create a collaborative filtering recommender model using singular value...
