Search Results for "python web crawler" - Page 2
Sort By:
Showing 158 open source projects for "python web crawler"
View related business solutions-
Gen AI apps are built with MongoDB AtlasMongoDB Atlas provides built-in vector search and a flexible document model so developers can build, scale, and run gen AI apps without stitching together multiple databases. From LLM integration to semantic search, Atlas simplifies your AI architecture—and it’s free to get started.
-
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
1
Yandex Smart Home
Adds support for Yandex Smart Home (Alice voice assistant)
Adds support for Yandex Smart Home (Alice voice assistant) into Home Assistant. The component allows you to add devices from Home Assistant to the Yandex smart home platform and manage them from any device with Alice. The component runs on Home Assistant version 2023.2 or later. -
2
CogVideo
text and image to video generation: CogVideoX (2024) and CogVideo
CogVideo is an open source text-/image-/video-to-video generation project that hosts the CogVideoX family of diffusion-transformer models and end-to-end tooling. The repo includes SAT and Diffusers implementations, turnkey demos, and fine-tuning pipelines (including LoRA) designed to run across a wide range of NVIDIA GPUs, from desktop cards (e.g., RTX 3060) to data-center hardware (A100/H100). Current releases cover CogVideoX-2B, CogVideoX-5B, and the upgraded CogVideoX1.5-5B variants, plus... -
3
ChatGLM3
ChatGLM3 series: Open Bilingual Chat LLMs | Open Source Bilingual Chat
...It keeps the series’ smooth dialog and low deployment cost while adding native tool use (function calling), a built-in code interpreter, and agent-style workflows. The family includes base and long-context variants (8K/32K/128K). The repo ships Python APIs, CLI and web demos (Gradio/Streamlit), an OpenAI-format API server, and a compact fine-tuning kit. Quantization (4/8-bit), CPU/MPS support, and accelerator backends (TensorRT-LLM, OpenVINO, chatglm.cpp) enable lightweight local or edge deployment. -
4
Composio
Composio equip's your AI agents & LLMs
Empower your AI agents with Composio - a platform for managing and integrating tools with LLMs & AI agents using Function Calling. Equip your agent with high-quality tools & integrations without worrying about authentication, accuracy, and reliability in a single line of code. -
Simple, Secure Domain RegistrationRegister or renew your domain and pay only what we pay. No markups, hidden fees, or surprise add-ons. Choose from over 400 TLDs (.com, .ai, .dev). Every domain is integrated with Cloudflare's industry-leading DNS, CDN, and free SSL to make your site faster and more secure. Simple, secure, at-cost domain registration.
-
5
LLaVA
Visual Instruction Tuning: Large Language-and-Vision Assistant
Visual instruction tuning towards large language and vision models with GPT-4 level capabilities. -
6
spaCy models
Models for the spaCy Natural Language Processing (NLP) library
spaCy is designed to help you do real work, to build real products, or gather real insights. The library respects your time, and tries to avoid wasting it. It's easy to install, and its API is simple and productive. spaCy excels at large-scale information extraction tasks. It's written from the ground up in carefully memory-managed Cython. If your application needs to process entire web dumps, spaCy is the library you want to be using. Since its release in 2015, spaCy has become an industry... -
7
GPT All Star
AI-powered code generation tool for scratch development of web apps
AI-powered code generation tool for scratch development of web applications with a team collaboration of autonomous AI agents. This is a research project, and its primary value is to explore the possibility of autonomous AI agents. -
8
Scrapling
An undetectable, powerful, flexible, high-performance Python library
Scrapling is a Python scraping framework built for the modern web, combining high-performance fetchers with a rapid parsing engine to handle dynamic sites and anti-bot countermeasures. It emphasizes being “undetectable,” flexible, and fast, offering an approachable API for both experienced scrapers and newcomers. The library targets the full scraping pipeline: session handling, fetching, rendering when needed, parsing, and export—while keeping ergonomics front and center. ... -
9
ChatArena
ChatArena (or Chat Arena) is a Multi-Agent Language Game Environments
ChatArena is a library that provides multi-agent language game environments and facilitates research about autonomous LLM agents and their social interactions. -
Build Securely on AWS with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
10
nanochat
The best ChatGPT that $100 can buy
nanochat is a from-scratch, end-to-end “mini ChatGPT” that shows the entire path from raw text to a chatty web app in one small, dependency-lean codebase. The repository stitches together every stage of the lifecycle: tokenizer training, pretraining a Transformer on a large web corpus, mid-training on dialogue and multiple-choice tasks, supervised fine-tuning, optional reinforcement learning for alignment, and finally efficient inference with caching. Its north star is approachability and... -
11
GPT Researcher
LLM based autonomous agent that does online comprehensive research
Say Hello to GPT Researcher, your AI agent for rapid insights and comprehensive research. GPT Researcher is the leading autonomous agent that takes care of everything from accurate source gathering to organization of research results. -
12
Klavis AI
MCP integration platforms for AI agents to use tools at any scale
Klavis AI is a Y Combinator X25-backed open-source infrastructure platform that enables AI agents to reliably connect with external tools and services at scale through Model Context Protocol (MCP). Founded by ex-Google DeepMind and ex-Lyft engineers, Klavis provides 50+ production-ready MCP servers with enterprise OAuth support for GitHub, Slack, Gmail, Salesforce, Linear, Notion, and more. The flagship product Strata solves tool overload through progressive discovery, achieving +13% higher... -
13
Aider
Aider is AI pair programming in your terminal
Aider is an open-source AI pair programming tool that runs directly in your terminal, allowing developers to collaborate with LLMs as if they were coding alongside a senior engineer. It maps your entire codebase to provide deep context, making it effective for both small scripts and large, complex projects. Aider supports leading cloud models like Claude 3.7 Sonnet, DeepSeek R1, OpenAI’s o1/o3/GPT-4o, as well as local LLMs for privacy-conscious workflows. Its Git integration ensures every... -
14
OpenHands
Open-source autonomous AI software engineer
Welcome to OpenHands (formerly OpenDevin), an open-source autonomous AI software engineer who is capable of executing complex engineering tasks and collaborating actively with users on software development projects. Use AI to tackle the toil in your backlog, so you can focus on what matters: hard problems, creative challenges, and over-engineering your dotfiles We believe agentic technology is too important to be controlled by a few corporations. So we're building all our agents in the... -
15
AskUI Vision Agent
Enable AI to control your desktop, mobile and HMI devices
AskUI’s Vision Agent is an automation framework that allows you—and AI agents—to control real desktops, mobile devices, and HMI systems by perceiving the UI and performing actions like clicking, typing, scrolling, and drag-and-drop. It is designed for multi-platform compatibility and supports multiple AI models so you can tailor perception and decision-making to your workload. The repository presents a feature overview, sample media, and frequent release notes, which show ongoing... -
16
Magentic UI
A research prototype of a human-centered web agent
Magentic-UI is a research prototype developed by Microsoft that serves as a human-centered interface powered by a multi-agent system. It enables users to automate complex web tasks, such as browsing, form filling, and data analysis, while maintaining control over the process. The system emphasizes transparency and user involvement, making it suitable for tasks requiring both automation and human oversight. -
17
langrocks
Tools like web browser, computer access and code runner for LLMs
Langrocks is a programming language experimentation toolkit that enables developers to create, test, and optimize custom programming languages. -
18
MagicTime
Time-lapse Video Generation Models as Metamorphic Simulators
This repository is the official implementation of MagicTime, a metamorphic video generation pipeline based on the given prompts. The main idea is to enhance the capacity of video generation models to accurately depict the real world through our proposed methods and dataset. Compared to general videos, metamorphic videos contain physical knowledge, long persistence, and strong variation, making them difficult to generate. -
19
HunyuanWorld 1.0
Generating Immersive, Explorable, and Interactive 3D Worlds
HunyuanWorld-1.0 is an open-source, simulation-capable 3D world generation model developed by Tencent Hunyuan that creates immersive, explorable, and interactive 3D environments from text or image inputs. It combines the strengths of video-based diversity and 3D-based geometric consistency through a novel framework using panoramic world proxies and semantically layered 3D mesh representations. This approach enables 360° immersive experiences, seamless mesh export for graphics pipelines, and... -
20
ClearML
Streamline your ML workflow
...It is designed as an end-to-end MLOps suite allowing you to focus on developing your ML code & automation, while ClearML ensures your work is reproducible and scalable. The ClearML Python Package for integrating ClearML into your existing scripts by adding just two lines of code, and optionally extending your experiments and other workflows with ClearML powerful and versatile set of classes and methods. The ClearML Server storing experiment, model, and workflow data, and supports the Web UI experiment manager, and ML-Ops automation for reproducibility and tuning. ... -
21
ChatGLM2-6B
ChatGLM2-6B: An Open Bilingual Chat LLM
...It upgrades the base model with GLM’s hybrid pretraining objective, 1.4 TB bilingual data, and preference alignment—delivering big gains on MMLU, CEval, GSM8K, and BBH. The context window extends up to 32K (FlashAttention), and Multi-Query Attention improves speed and memory use. The repo includes Python APIs, CLI & web demos, OpenAI-style/FASTAPI servers, and quantized checkpoints for lightweight local deployment on GPUs or CPU/MPS. -
22
Stable Diffusion Version 2
High-Resolution Image Synthesis with Latent Diffusion Models
Stable Diffusion (the stablediffusion repo by Stability-AI) is an open-source implementation and reference codebase for high-resolution latent diffusion image models that power many text-to-image systems. The repository provides code for training and running Stable Diffusion-style models, instructions for installing dependencies (with notes about performance libraries like xformers), and guidance on hardware/driver requirements for efficient GPU inference and training. It’s organized as a... -
23
Mosec
A high-performance ML model serving framework, offers dynamic batching
Mosec is a high-performance and flexible model-serving framework for building ML model-enabled backend and microservices. It bridges the gap between any machine learning models you just trained and the efficient online service API. -
24
fastdup
An unsupervised and free tool for image and video dataset analysis
fastdup is a powerful free tool designed to rapidly extract valuable insights from your image & video datasets. Assisting you to increase your dataset images & labels quality and reduce your data operations costs at an unparalleled scale. -
25
PokeeResearch-7B
Pokee Deep Research Model Open Source Repo
PokeeResearchOSS provides an open-source, agentic “deep research” model centered on a 7B backbone that can browse, read, and synthesize current information from the web. Instead of relying only on static training data, the agent performs searches, visits pages, and extracts evidence before forming answers to complex queries. It is built to operate end-to-end: planning a research strategy, gathering sources, reasoning over conflicting claims, and writing a grounded response. The repository...
