Showing 11 open source projects for "crawling"

View related business solutions
  • Our Free Plans just got better! | Auth0 Icon
    Our Free Plans just got better! | Auth0

    With up to 25k MAUs and unlimited Okta connections, our Free Plan lets you focus on what you do best—building great apps.

    You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
    Try free now
  • Context for your AI agents Icon
    Context for your AI agents

    Crawl websites, sync to vector databases, and power RAG applications. Pre-built integrations for LLM pipelines and AI assistants.

    Build data pipelines that feed your AI models and agents without managing infrastructure. Crawl any website, transform content, and push directly to your preferred vector store. Use 10,000+ tools for RAG applications, AI assistants, and real-time knowledge bases. Monitor site changes, trigger workflows on new data, and keep your AIs fed with fresh, structured information. Cloud-native, API-first, and free to start until you need to scale.
    Try for free
  • 1
    Firecrawl MCP Server

    Firecrawl MCP Server

    Adds powerful web scraping and search to Cursor and Claude

    firecrawl-mcp-server is the official MCP integration for Firecrawl that brings high-recall web scraping, crawling, and search into IDEs and agent runtimes. It exposes tools for single-page scrape, multi-URL batch jobs, site discovery, and search enrichment, returning cleaned, structured content suitable for downstream LLM reasoning. The server is designed to run with Firecrawl’s hosted API or self-hosted deployments, making it flexible for enterprise data-governance requirements. ...
    Downloads: 1 This Week
    Last Update:
    See Project
  • 2
    Vishesh Auto Index

    Vishesh Auto Index

    Vishesh Auto Index 5 is Best Wap Auto Index for mp3 sites.

    Vishesh Auto Index is a rebirth of AutoIndex. A mobile focused autoindex script that allows users to create a mobile download site. Security level too high. Vishesh Auto Index 5 specially made for mp3/mp4 downloading websites. Vishesh Auto Index 5 My Official Blog:- https://visheshgrewal.blogspot.com Vishesh Auto Index:- https://vishesh-auto-index.blogspot.com Readme.txt for more information
    Downloads: 2 This Week
    Last Update:
    See Project
  • 3
    Headless Chrome Crawler

    Headless Chrome Crawler

    Distributed crawler powered by Headless Chrome

    ...Powered by Headless Chrome, the crawler provides simple APIs to crawl dynamic websites. Support both depth-first search and breadth-first search algorithm. Save screenshots for the crawling evidence, emulate devices and user agents, priority queue for crawling efficiency, obey robots.txt, and more. The static crawlers are based on simple requests to HTML files. They are generally fast, but fail scraping the contents when the HTML dynamically changes on browsers. Dynamic crawlers based on PhantomJS and Selenium work magically on such dynamic applications. ...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 4
    Node Crawler

    Node Crawler

    Web Crawler/Spider for NodeJS + server-side jQuery

    Most powerful, popular and production crawling/scraping package for Node, happy hacking.
    Downloads: 0 This Week
    Last Update:
    See Project
  • eProcurement Software Icon
    eProcurement Software

    Enterprises and companies seeking a solution to manage all their procurement operations and processes

    eBuyerAssist by Eyvo is a cloud-based procurement solution designed for businesses of all sizes and industries. Fully modular and scalable, it streamlines the entire procurement lifecycle—from requisition to fulfillment. The platform includes powerful tools for strategic sourcing, supplier management, warehouse operations, and contract oversight. Additional modules cover purchase orders, approval workflows, inventory and asset management, customer orders, budget control, cost accounting, invoice matching, vendor credit checks, and risk analysis. eBuyerAssist centralizes all procurement functions into a single, easy-to-use system—improving visibility, control, and efficiency across your organization. Whether you're aiming to reduce costs, enhance compliance, or align procurement with broader business goals, eBuyerAssist helps you get there faster, smarter, and with measurable results.
    Learn More
  • 5
    Nightmare

    Nightmare

    A high-level browser automation library

    ...The goal is to expose a few simple methods that mimic user actions (like goto, type and click), with an API that feels synchronous for each block of scripting, rather than deeply nested callbacks. It was originally designed for automating tasks across sites that don't have APIs, but is most often used for UI testing and crawling. Segment started with an open source project. Since then, we’ve open sourced hundreds of our repos. We want to continue supporting the community by publishing our code and other developers’ awesome open source projects. We use these open source projects as the foundations of our infrastructure to handle billions of API calls per day and to allow us to rapidly build and test code on the client.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 6
    LAIR! the game

    LAIR! the game

    LAIR is a Sandbox Dungeon Crawling Game

    LAIR! is a game designed to provide virtually limitless versatility in the environment that it provides. While you create a new character, it generates a dungeon for you to explore with thousands of creatures, treasures, and secrets. Every piece of content can be generated procedurally by the game engine, so new dungeons and treasures can appear at any time.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 7
    Zoozle Search & Download Suchmaschine

    Zoozle Search & Download Suchmaschine

    Zoozle 2008 - 2010 Webpage, Tools and SQL Files

    Download search engine and directory with Rapidshare and Torrent - zoozle Download Suchmaschine All The files that run the World Leading German Download Search Engine in 2010 with 500 000 unique visitors a day - all the tools you need to set up a clone. Code Contains: - PHP Files for zoozle - Perl Crawler for gathering new content to database and all other cool tools i have...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 8
    Subversion Indexer Widget for GDS
    svnindexer gives the ability of crawling into a Subversion repository to Google Desktop Search.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 9
    A hybrid science-fantasy RPG set in the near future. Sunderworld has the dungeon-crawling hack-and-slash goodness of Diablo and the rewarding puzzle-solving elements of Zelda. This is a work in early Alpha, and needs contributors.
    Downloads: 0 This Week
    Last Update:
    See Project
  • Reach Your Audience with Rise Vision, the #1 Cloud Digital Signage Software Solution Icon
    Reach Your Audience with Rise Vision, the #1 Cloud Digital Signage Software Solution

    K-12 Schools, Higher Education, Businesses, Restaurants

    Rise Vision is the #1 digital signage company, offering easy-to-use cloud digital signage software compatible with any player across multiple screens. Forget about static displays. Save time and boost sales with 500+ customizable content templates for your screens. If you ever need help, get free training and exceptionally fast support.
    Learn More
  • 10
    Methanol is a scriptable multi-purpose web crawling system with an extensible configuration system and speed-optimized architectural design. Methabot is the web crawler of Methanol.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 11
    An PHP + MySQL script that attempts to recreate features similar to the likes of Mininova, SeedPeer, and TPB frontend. Features a new design, crawling ability, cURL for single torrent scrapes + mass scrapes, and much much more! Based on T-Xore 0.4, Ibitz
    Downloads: 0 This Week
    Last Update:
    See Project
  • Previous
  • You're on page 1
  • Next