Search Results for "web crawler source code" - Page 2
Sort By:
Showing 80 open source projects for "web crawler source code"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Gemini 3 and 200+ AI Models on One PlatformBuild generative AI apps with Vertex AI. Switch between models without switching platforms.
-
1
Integuru v0
The first AI agent that builds permissionless integrations
Integuru is an open-source AI agent designed to automatically create integrations between software platforms by reverse-engineering their internal APIs. Instead of relying on official developer documentation or publicly available APIs, the system analyzes network traffic generated by user interactions within a web application. Developers capture browser requests and authentication data, which the agent then uses to infer the structure of the platform’s internal API endpoints. ... -
2
AWS Agent Skills
AWS Skills for Agents
AWS Agent Skills is a repository that curates AWS-focused agent skills — capability modules that give AI assistants like Claude Code and Codex deep, practical knowledge across key Amazon Web Services domains. Instead of streaming giant documentation sets or relying on episodic web search, this project compresses AWS best practices, usage patterns, edge cases, and real-world engineering guides into pre-structured skill definitions that are token-efficient and tailored for reasoning. The... -
3
Spark TTS
Spark-TTS Inference Code
Spark TTS is an open-source, PyTorch-based text-to-speech inference system that leverages large language models to produce highly natural, intelligible speech from text input. It uses an efficient single-stream architecture where speech tokens are directly reconstructed from the predictions of an LLM, removing the need for external acoustic models or complex vocoders and making the generation pipeline cleaner and faster. The project supports zero-shot voice cloning, meaning it can imitate a... -
4
gpt-oss
gpt-oss-120b and gpt-oss-20b are two open-weight language models
gpt-oss is OpenAI’s open-weight family of large language models designed for powerful reasoning, agentic workflows, and versatile developer use cases. The series includes two main models: gpt-oss-120b, a 117-billion parameter model optimized for general-purpose, high-reasoning tasks that can run on a single H100 GPU, and gpt-oss-20b, a lighter 21-billion parameter model ideal for low-latency or specialized applications on smaller hardware. Both models use a native MXFP4 quantization for... -
Earn up to 16% annual interest with Nexo.Generate interest, access liquidity without selling, and execute trades seamlessly. All in one platform. Geographic restrictions, eligibility, and terms apply.
-
5
DeepWiki Open
AI-Powered Wiki Generator for GitHub/Gitlab/Bitbucket Repositories
DeepWiki Open is an open-source, AI-powered wiki generator that automatically creates fully navigable, richly structured wiki documentation for GitHub, GitLab, or Bitbucket repositories by combining code analysis, vector embeddings, retrieval-augmented generation (RAG), and visualization tools. Users can enter a repository URL and the system will clone the project, build semantic embeddings of its codebase, extract architecture and relationships, generate human-readable documentation, and produce visual diagrams to help explain complex code structure. ... -
6
ChatGLM-6B
ChatGLM-6B: An Open Bilingual Dialogue Language Model
ChatGLM-6B is an open bilingual (Chinese + English) conversational language model based on the GLM architecture, with approximately 6.2 billion parameters. The project provides inference code, demos (command line, web, API), quantization support for lower memory deployment, and tools for finetuning (e.g., via P-Tuning v2). It is optimized for dialogue and question answering with a balance between performance and deployability in consumer hardware settings. Support for quantized inference... -
7
Build Your Own OpenClaw
A step-by-step guide to build your own AI agent
Build Your Own OpenClaw is a step-by-step educational framework that teaches developers how to construct a fully functional AI agent system from scratch, gradually evolving from a simple chat loop into a multi-agent, production-ready architecture. The project is structured into 18 progressive stages, each introducing a new concept such as tool usage, memory persistence, event-driven design, and multi-agent coordination, with each step including both explanatory documentation and runnable... -
8
ClearML
Streamline your ML workflow
ClearML is an open source platform that automates and simplifies developing and managing machine learning solutions for thousands of data science teams all over the world. It is designed as an end-to-end MLOps suite allowing you to focus on developing your ML code & automation, while ClearML ensures your work is reproducible and scalable. The ClearML Python Package for integrating ClearML into your existing scripts by adding just two lines of code, and optionally extending your experiments and other workflows with ClearML powerful and versatile set of classes and methods. ... -
9
self-llm
Tutorial tailored for Chinese babies on rapid fine-tuning
self-llm is an open source educational project created by the Datawhale community that serves as a practical guide for deploying, fine-tuning, and using open-source large language models on Linux systems. The repository focuses on helping beginners and developers understand how to run and customize modern LLMs locally rather than relying solely on hosted APIs. It provides step-by-step tutorials covering environment setup, model deployment, inference workflows, and efficient fine-tuning... -
Full-stack observability with actually useful AI | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
10
OpenHands
Open-source autonomous AI software engineer
...So we're building all our agents in the open on GitHub, under the MIT license. Our agents can do anything a human developer can: they write code, run commands, and use the web. We're partnering with AI safety experts like Invariant Labs to balance innovation with security. -
11
Agent Stack
Deploy and share agents with open infrastructure
Agent Stack is an open infrastructure platform designed to take AI agents from prototype to production, no matter how they were built. It includes a runtime environment, multi-tenant web UI, catalog of agents, and deployment flow that seeks to remove vendor lock-in and provide greater autonomy. Under the hood it’s built on the “Agent2Agent” (A2A) protocol, enabling interoperability between different agent ecosystems, runtime services, and frameworks. The platform supports agents built in... -
12
Marvin
A batteries-included library for building AI-powered software
...Marvin's job is to integrate AI directly into your codebase by making it look and feel like any other function. Marvin introduces a new concept called AI Functions. These functions differ from conventional ones in that they don’t rely on source code, but instead generate their outputs on-demand through AI. With AI functions, you don't have to write complex code for tasks like extracting entities from web pages, scoring sentiment, or categorizing items in your database. Just describe your needs, call the function, and you're done. AI functions work with native data types, so you can seamlessly integrate them into any codebase and chain them into sophisticated pipelines. ... -
13
Eigent
The Open Source Cowork Desktop to Unlock Your Exceptional Productivity
Eigent is an open-source cowork desktop application designed to help you build, manage, and deploy a custom AI workforce. It enables multiple specialized AI agents to collaborate in parallel, turning complex workflows into automated, end-to-end tasks. Built on the CAMEL-AI multi-agent framework, Eigent emphasizes productivity, flexibility, and transparent system design. You can run Eigent fully locally for maximum privacy and data control, or choose a cloud-connected experience for quick... -
14
FastKoko
Dockerized FastAPI wrapper for Kokoro-82M text-to-speech model
FastKoko is a self-hosted text-to-speech server built around the Kokoro-82M model and exposed through a FastAPI backend. It is designed to be easy to deploy via Docker, with separate CPU and GPU images so that users can choose between pure CPU inference and NVIDIA GPU acceleration. The project exposes an OpenAI-compatible speech endpoint, which means existing code that talks to the OpenAI audio API can often be pointed at a Kokoro-FastAPI instance with minimal changes. It supports multiple... -
15
A.I.G
Full-stack AI Red Teaming platform
AI-Infra-Guard is a powerful open-source security platform from Tencent’s Zhuque Lab designed to assess the safety and resilience of AI infrastructures, codebases, and components through automated scanning and evaluation tools. It brings together AI infrastructure vulnerability scanning, MCP server risk analysis, and jailbreak evaluation into a unified workflow so that enterprises and individuals can identify critical security issues without relying on external services. Users can deploy it... -
16
ChatGPT Clone
ChatGPT interface with better UI
ChatGPT Clone demonstrates a ChatGPT-style conversational interface wired to large-language-model backends, packaged so developers can self-host and extend. The goal is to replicate the core chat UX—message history, streaming tokens, code blocks, and system prompts—while letting you plug in different provider APIs or local models. It showcases a clean separation between the web client and the message orchestration layer so you can experiment with prompts, roles, and memory strategies. The... -
17
dots.ocr
Multilingual Document Layout Parsing in a Single Vision-Language Model
dots.ocr is a cutting-edge multilingual document parsing system built on a unified vision-language model that combines layout detection, text recognition, and structural understanding into a single architecture. Unlike traditional OCR pipelines that rely on multiple specialized components, dots.ocr integrates these processes end-to-end, reducing error propagation and improving consistency across tasks. The model is designed to recognize virtually any human script, making it highly effective... -
18
Search with Lepton
Lightweight demo to build a conversational AI search engine quickly
Search with Lepton is an open source demonstration project that shows how to build a conversational search engine using the Lepton AI framework. It combines traditional web search with large language models to provide natural language answers to user queries. It retrieves information from supported search engines and uses that context to generate responses through a retrieval-augmented generation approach. -
19
notebooklm-py
Unofficial Python API and agentic skill for Google NotebookLM
notebooklm-py is an unofficial Python API and agent-ready integration layer for Google NotebookLM that exposes NotebookLM functionality through code, the command line, and AI agent workflows. Its goal is to provide programmatic access not just to standard notebook operations, but also to many capabilities that are either limited or unavailable in the web interface, making it especially useful for automation and custom pipelines. The project covers notebook management, source ingestion, conversational querying, research workflows, and sharing controls, while also enabling the generation of a wide range of study and media artifacts. ... -
20
langrocks
Tools like web browser, computer access and code runner for LLMs
Langrocks is a programming language experimentation toolkit that enables developers to create, test, and optimize custom programming languages. -
21
nanochat
The best ChatGPT that $100 can buy
nanochat is a from-scratch, end-to-end “mini ChatGPT” that shows the entire path from raw text to a chatty web app in one small, dependency-lean codebase. The repository stitches together every stage of the lifecycle: tokenizer training, pretraining a Transformer on a large web corpus, mid-training on dialogue and multiple-choice tasks, supervised fine-tuning, optional reinforcement learning for alignment, and finally efficient inference with caching. Its north star is approachability and... -
22
Composio
Composio equip's your AI agents & LLMs
Empower your AI agents with Composio - a platform for managing and integrating tools with LLMs & AI agents using Function Calling. Equip your agent with high-quality tools & integrations without worrying about authentication, accuracy, and reliability in a single line of code. -
23
AI Agents Masterclass
Follow along with my AI Agents Masterclass videos
AI Agents Masterclass is an educational open-source repository designed to teach developers how to build, train, and deploy intelligent AI agents using modern tooling and workflow patterns. The project includes structured lessons, code examples, and practical exercises that cover foundational concepts like prompt engineering, chaining agents, tool usage, plan execution, evaluation, and safety considerations. -
24
Gradio
Create UIs for your machine learning model in Python in 3 minutes
Gradio is the fastest way to demo your machine learning model with a friendly web interface so that anyone can use it, anywhere! Gradio can be installed with pip. Creating a Gradio interface only requires adding a couple lines of code to your project. You can choose from a variety of interface types to interface your function. Gradio can be embedded in Python notebooks or presented as a webpage. A Gradio interface can automatically generate a public link you can share with colleagues that... -
25
Lagent
A lightweight framework for building LLM-based agents
Lagent is a lightweight open-source framework designed to help developers build autonomous agents powered by large language models. The framework provides tools and abstractions that allow language models to interact with external tools, execute tasks, and perform multi-step reasoning processes. Instead of using LLMs only for text generation, Lagent enables developers to transform models into agents capable of performing actions such as retrieving data, executing code, or interacting with APIs. ...
