Search Results for "python web crawler" - Page 2
Sort By:
Showing 144 open source projects for "python web crawler"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Simply solve complex auth. Easy for devs to set up. Easy for non-devs to use.Custom auth drains 25% of dev time and risks 62% more breaches, stalling enterprise deals. Frontegg platform delivers a simple login box, seamless authentication (SSO, MFA, passwordless), robust multi-tenancy, and a customizable Admin Portal. Integrate fast with the React SDK, meet compliance needs, and focus on innovation.
-
1
Notte
Opensource browser using agents
Notte is an open-source browser framework that enables the development and deployment of web-based AI agents. It introduces a perception layer that transforms web pages into structured, navigable maps described in natural language, allowing agents to interact with the internet more effectively. Notte is designed for building scalable and efficient browser-based AI applications. -
2
Klavis AI
Open Source MCP integration for AI applications
Klavis AI is an open-source platform that simplifies the integration of Model Context Protocols (MCPs) into AI applications. It provides hosted, secure MCP servers with built-in OAuth support, eliminating the need for complex authentication management and client-side code. Klavis AI enables developers to connect AI agents to various tools and services efficiently, facilitating scalable and secure AI deployments. -
3
HunyuanDiT
Diffusion Transformer with Fine-Grained Chinese Understanding
HunyuanDiT is a high-capability text-to-image diffusion transformer with bilingual (Chinese/English) understanding and multi-turn dialogue capability. It trains a diffusion model in latent space using a transformer backbone and integrates a Multimodal Large Language Model (MLLM) to refine captions and support conversational image generation. It supports adapters like ControlNet, IP-Adapter, LoRA, and can run under constrained VRAM via distillation versions. -
4
Composio
Composio equip's your AI agents & LLMs
Empower your AI agents with Composio - a platform for managing and integrating tools with LLMs & AI agents using Function Calling. Equip your agent with high-quality tools & integrations without worrying about authentication, accuracy, and reliability in a single line of code. -
The Ultimate Quiz Maker & Engagement PlatformRiddle is an online platform for creating interactive content such as quizzes, surveys, personality tests, prediction games, and leaderboards. Our customers create content on our platform and then embed it on their website. The goal? Increased engagement, lead generation, segmentation, and content monetization - all 100% GDPR compliant.
-
5
h2oGPT
Private chat with local GPT with document, images, video, etc.
h2oGPT is an open-source platform that allows users to interact with local GPT models in a completely private environment. It supports a variety of document types, including PDFs, Word files, images, video frames, and even audio, enabling users to query and analyze their documents or engage in a private chat with AI. The platform is designed to be secure and offline, ensuring that all data remains private and under the user's control. h2oGPT supports several AI models, including oLLaMa and... -
6
LLaMA Efficient Tuning
Easy-to-use LLM fine-tuning framework (LLaMA-2, BLOOM, Falcon
Easy-to-use LLM fine-tuning framework (LLaMA-2, BLOOM, Falcon, Baichuan, Qwen, ChatGLM2) -
7
Omnara
Talk to Your AI Agents from Anywhere
Omnara is an open-source agent control platform that empowers developers to turn autonomous AI tools (e.g., Claude Code, Cursor, GitHub Copilot) into collaborative teammates by offering real-time dashboards, push notifications, and remote guidance across terminals, web, and mobile. -
8
GPT-SoVITS
1 min voice data can also be used to train a good TTS model
GPT‑SoVITS is a state-of-the-art voice conversion and TTS system that enables zero‑shot and few‑shot synthesis based on a short vocal sample (e.g., 5 seconds). It supports cross‑lingual speech synthesis across English, Chinese, Japanese, Korean, Cantonese, and more. It's powered by VITS architecture enhanced for few‑sample adaptation and real‑time usability. -
9
WebArena
Code repo for "WebArena to build Autonomous Agents
WebArena is a realistic web environment designed for building and testing autonomous agents, providing a platform for developing web-based AI agents. -
No-Nonsense Code-to-Cloud Security for Devs | AikidoAikido provides a unified security platform for developers, combining 12 powerful scans like SAST, DAST, and CSPM. AI-driven AutoFix and AutoTriage streamline vulnerability management, while runtime protection blocks attacks.
-
10
Substra
Low-level Python library used to interact with a Substra network
An open-source framework supporting privacy-preserving, traceable federated learning and machine learning orchestration. Offers a Python SDK, high-level FL library (SubstraFL), and web UI to define datasets, models, tasks, and orchestrate secure, auditable collaborations. -
11
Dendrite
Tools to build web AI agents that can authenticate
Dendrite Python SDK is a toolkit for building web AI agents that can authenticate, interact with, and extract data from any website, facilitating web automation tasks. -
12
ScrapeGraphAI
Python scraper based on AI
Extracting content from websites and local documents using LLM. ScrapeGraphAI is a web scraping python library that uses LLM and direct graph logic to create scraping pipelines for websites and local documents (XML, HTML, JSON, Markdown, etc.). Just say which information you want to extract and the library will do it for you. -
13
Browser Use MCP Server
Browse the web, directly from Cursor etc.
A browser automation server implementing the Model Context Protocol, designed to allow AI assistants to browse the web directly from applications like Cursor. It supports natural language commands for web navigation and interaction. -
14
img2dataset
Easily turn large sets of image urls to an image dataset
Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine. Also supports saving captions for url+caption datasets. Opt-out directives: Websites can pass the http headers X-Robots-Tag: noai, X-Robots-Tag: noindex , X-Robots-Tag: noimageai and X-Robots-Tag: noimageindex By default img2dataset will ignore images with such headers. -
15
Taipy
Turns Data and AI algorithms into production-ready web applications
From simple pilots to production-ready web applications in no time. No more compromise on performance, customization, and scalability. Taipy enhances performance with caching control of graphical events, optimizing rendering by selectively updating graphical components only upon interaction. Effortlessly manage massive datasets with Taipy's built-in decimator for charts, intelligently reducing the number of data points to save time and memory without losing the essence of your data's shape... -
16
langrocks
Tools like web browser, computer access and code runner for LLMs
Langrocks is a programming language experimentation toolkit that enables developers to create, test, and optimize custom programming languages. -
17
MagicTime
Time-lapse Video Generation Models as Metamorphic Simulators
This repository is the official implementation of MagicTime, a metamorphic video generation pipeline based on the given prompts. The main idea is to enhance the capacity of video generation models to accurately depict the real world through our proposed methods and dataset. Compared to general videos, metamorphic videos contain physical knowledge, long persistence, and strong variation, making them difficult to generate. -
18
ChatGLM-6B
ChatGLM-6B: An Open Bilingual Dialogue Language Model
ChatGLM-6B is an open bilingual (Chinese + English) conversational language model based on the GLM architecture, with approximately 6.2 billion parameters. The project provides inference code, demos (command line, web, API), quantization support for lower memory deployment, and tools for finetuning (e.g., via P-Tuning v2). It is optimized for dialogue and question answering with a balance between performance and deployability in consumer hardware settings. -
19
Magentic UI
A research prototype of a human-centered web agent
Magentic-UI is a research prototype developed by Microsoft that serves as a human-centered interface powered by a multi-agent system. It enables users to automate complex web tasks, such as browsing, form filling, and data analysis, while maintaining control over the process. The system emphasizes transparency and user involvement, making it suitable for tasks requiring both automation and human oversight. -
20
GPT All Star
AI-powered code generation tool for scratch development of web apps
AI-powered code generation tool for scratch development of web applications with a team collaboration of autonomous AI agents. This is a research project, and its primary value is to explore the possibility of autonomous AI agents. -
21
Screenshot to Code
A neural network that transforms a design mock-up into static websites
Screenshot-to-code is a tool or prototype that attempts to convert UI screenshots (e.g., of mobile or web UIs) into code representations, likely generating layouts, HTML, CSS, or markup from image inputs. It is part of a research/proof-of-concept domain in UI automation and image-to-UI code generation. -
22
ChatArena
ChatArena (or Chat Arena) is a Multi-Agent Language Game Environments
ChatArena is a library that provides multi-agent language game environments and facilitates research about autonomous LLM agents and their social interactions. -
23
fastdup
An unsupervised and free tool for image and video dataset analysis
fastdup is a powerful free tool designed to rapidly extract valuable insights from your image & video datasets. Assisting you to increase your dataset images & labels quality and reduce your data operations costs at an unparalleled scale. -
24
llama2-webui
Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere
Running Llama 2 with gradio web UI on GPU or CPU from anywhere (Linux/Windows/Mac). -
25
Yandex Smart Home
Adds support for Yandex Smart Home (Alice voice assistant)
Adds support for Yandex Smart Home (Alice voice assistant) into Home Assistant. The component allows you to add devices from Home Assistant to the Yandex smart home platform and manage them from any device with Alice. The component runs on Home Assistant version 2023.2 or later.
