Search Results for "web crawler source code"
Sort By:
Showing 19 open source projects for "web crawler source code"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Earn up to 16% annual interest with Nexo.Generate interest, access liquidity without selling, and execute trades seamlessly. All in one platform. Geographic restrictions, eligibility, and terms apply.
-
1
DeerFlow
Deep Research framework, combining language models with tools
DeerFlow is an open-source, community-driven “deep research” framework / multi-agent orchestration platform developed by ByteDance. It aims to combine the reasoning power of large language models (LLMs) with automated tool-use — such as web search, web crawling, Python execution, and data processing — to enable complex, end-to-end research workflows. Instead of a monolithic AI assistant, DeerFlow defines multiple specialized agents (e.g. -
2
GPT All Star
AI-powered code generation tool for scratch development of web apps
AI-powered code generation tool for scratch development of web applications with a team collaboration of autonomous AI agents. This is a research project, and its primary value is to explore the possibility of autonomous AI agents. -
3
/last30days
Claude Code skill that researches any topic across Reddit + X
/last30days is a specialized Claude Code skill designed to research current trends and practices across Reddit, X, and the wider web from the last 30 days, synthesize that data, and produce copy-paste-ready prompts or summaries that reflect what the community is actually talking about now. Rather than returning generic model responses, it intelligently analyzes social media and community discussions to identify what’s genuinely trending or working in practice across topics ranging from... -
4
Agent Development Kit (ADK)
Open-source, code-first Python toolkit for building, evaluating, etc.
ADK (Android Device Key) Python is a reference implementation by Google for working with Android attestation keys in Python. It facilitates the integration of Android attestation features into backends or systems that require verification of device identity and integrity. This is especially important in high-security applications where verifying that a device is genuine and uncompromised is critical. ADK Python helps developers verify hardware-backed keys, work with JSON Web Tokens (JWT),... -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
5
nanobot
🐈 nanobot: The Ultra-Lightweight Clawdbot / OpenClaw
nanobot is an ultra-lightweight personal AI assistant designed to deliver powerful agent capabilities without unnecessary complexity. Built in just ~4,000 lines of clean, readable code, it offers a minimalist alternative to heavyweight agent frameworks while retaining core intelligence and extensibility. nanobot is optimized for speed and efficiency, enabling fast startup times and low resource usage across environments. Its research-ready architecture makes it easy for developers to... -
6
AWS Agent Skills
AWS Skills for Agents
AWS Agent Skills is a repository that curates AWS-focused agent skills — capability modules that give AI assistants like Claude Code and Codex deep, practical knowledge across key Amazon Web Services domains. Instead of streaming giant documentation sets or relying on episodic web search, this project compresses AWS best practices, usage patterns, edge cases, and real-world engineering guides into pre-structured skill definitions that are token-efficient and tailored for reasoning. The... -
7
OpenHands
Open-source autonomous AI software engineer
...So we're building all our agents in the open on GitHub, under the MIT license. Our agents can do anything a human developer can: they write code, run commands, and use the web. We're partnering with AI safety experts like Invariant Labs to balance innovation with security. -
8
Eigent
The Open Source Cowork Desktop to Unlock Your Exceptional Productivity
Eigent is an open-source cowork desktop application designed to help you build, manage, and deploy a custom AI workforce. It enables multiple specialized AI agents to collaborate in parallel, turning complex workflows into automated, end-to-end tasks. Built on the CAMEL-AI multi-agent framework, Eigent emphasizes productivity, flexibility, and transparent system design. You can run Eigent fully locally for maximum privacy and data control, or choose a cloud-connected experience for quick... -
9
Agent Stack
Deploy and share agents with open infrastructure
Agent Stack is an open infrastructure platform designed to take AI agents from prototype to production, no matter how they were built. It includes a runtime environment, multi-tenant web UI, catalog of agents, and deployment flow that seeks to remove vendor lock-in and provide greater autonomy. Under the hood it’s built on the “Agent2Agent” (A2A) protocol, enabling interoperability between different agent ecosystems, runtime services, and frameworks. The platform supports agents built in... -
Add Two Lines of Code. Get Full APM.Works out of the box for Rails, Django, Express, Phoenix, and more. Monitoring exceptions and performance in no time.
-
10
notebooklm-py
Unofficial Python API and agentic skill for Google NotebookLM
notebooklm-py is an unofficial Python API and agent-ready integration layer for Google NotebookLM that exposes NotebookLM functionality through code, the command line, and AI agent workflows. Its goal is to provide programmatic access not just to standard notebook operations, but also to many capabilities that are either limited or unavailable in the web interface, making it especially useful for automation and custom pipelines. The project covers notebook management, source ingestion, conversational querying, research workflows, and sharing controls, while also enabling the generation of a wide range of study and media artifacts. ... -
11
Build Your Own OpenClaw
A step-by-step guide to build your own AI agent
Build Your Own OpenClaw is a step-by-step educational framework that teaches developers how to construct a fully functional AI agent system from scratch, gradually evolving from a simple chat loop into a multi-agent, production-ready architecture. The project is structured into 18 progressive stages, each introducing a new concept such as tool usage, memory persistence, event-driven design, and multi-agent coordination, with each step including both explanatory documentation and runnable... -
12
SWE-agent
SWE-agent takes a GitHub issue and tries to automatically fix it
SWE-agent turns LMs (e.g. GPT-4) into software engineering agents that can resolve issues in real GitHub repositories. On the SWE-bench, the SWE-agent resolves 12.47% of issues, achieving state-of-the-art performance on the full test set. We accomplish our results by designing simple LM-centric commands and feedback formats to make it easier for the LM to browse the repository, and view, edit, and execute code files. We call this an Agent-Computer Interface (ACI). -
13
A.I.G
Full-stack AI Red Teaming platform
AI-Infra-Guard is a powerful open-source security platform from Tencent’s Zhuque Lab designed to assess the safety and resilience of AI infrastructures, codebases, and components through automated scanning and evaluation tools. It brings together AI infrastructure vulnerability scanning, MCP server risk analysis, and jailbreak evaluation into a unified workflow so that enterprises and individuals can identify critical security issues without relying on external services. Users can deploy it... -
14
Composio
Composio equip's your AI agents & LLMs
Empower your AI agents with Composio - a platform for managing and integrating tools with LLMs & AI agents using Function Calling. Equip your agent with high-quality tools & integrations without worrying about authentication, accuracy, and reliability in a single line of code. -
15
AI Agents Masterclass
Follow along with my AI Agents Masterclass videos
AI Agents Masterclass is an educational open-source repository designed to teach developers how to build, train, and deploy intelligent AI agents using modern tooling and workflow patterns. The project includes structured lessons, code examples, and practical exercises that cover foundational concepts like prompt engineering, chaining agents, tool usage, plan execution, evaluation, and safety considerations. -
16
Klavis AI
MCP integration platforms for AI agents to use tools at any scale
Klavis AI is a Y Combinator X25-backed open-source infrastructure platform that enables AI agents to reliably connect with external tools and services at scale through Model Context Protocol (MCP). Founded by ex-Google DeepMind and ex-Lyft engineers, Klavis provides 50+ production-ready MCP servers with enterprise OAuth support for GitHub, Slack, Gmail, Salesforce, Linear, Notion, and more. The flagship product Strata solves tool overload through progressive discovery, achieving +13% higher... -
17
Shinkai: Local AI Agents
Shinkai allows you to create advanced AI (local) agents effortlessly
Shinkai is a free, open-source AI platform that lets anyone create powerful AI agents without coding. These agents can collaborate with each other, handle complex tasks, and operate in decentralized crypto environments. Key Features: - No-Code Agent Creation - Build specialized agents (trading bots, sentiment trackers, etc.) with simple descriptions - Multi-Agent Collaboration - Agents work together to solve complex problems - Crypto Integration - Built-in support for decentralized... -
18
AI-Agent-Host
The AI Agent Host is a module-based development environment.
The AI Agent Host integrates several advanced technologies and offers a unique combination of features for the development of language model-driven applications. The AI Agent Host is a module-based environment designed to facilitate rapid experimentation and testing. It includes a docker-compose configuration with QuestDB, Grafana, Code-Server and Nginx. The AI Agent Host provides a seamless interface for managing and querying data, visualizing results, and coding in real-time. The AI... -
19
Multi Agent based distributed application. The code can be processed over multiple common machines with fault-tolerance. It is designed to distributively run any Python's script, which can be applied to a given input data set.
- Previous
- You're on page 1
- Next
