Showing 7 open source projects for "web crawler source code"

View related business solutions
  • Try Google Cloud Risk-Free With $300 in Credit Icon
    Try Google Cloud Risk-Free With $300 in Credit

    No hidden charges. No surprise bills. Cancel anytime.

    Use your credit across every product. Compute, storage, AI, analytics. When it runs out, 20+ products stay free. You only pay when you choose to.
    Start Free
  • Earn up to 16% annual interest with Nexo. Icon
    Earn up to 16% annual interest with Nexo.

    Access competitive interest rates on your digital assets.

    Generate interest, borrow against your crypto, and trade a range of cryptocurrencies — all in one platform. Geographic restrictions, eligibility, and terms apply.
    Get started with Nexo.
  • 1
    RobotsTxt

    RobotsTxt

    The repository contains Google's robots.txt parser

    This is a high-performance, production-tested library for parsing and evaluating robots.txt rules against crawler user agents. It implements the core semantics of the Robots Exclusion Protocol: user-agent sections, Allow/Disallow directives, wildcard handling, and precedence rules. The code is optimized for speed and low memory so large crawls can evaluate millions of URLs quickly. It also focuses on correctness—edge cases like overlapping patterns and longest-match resolution are handled...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 2
    openPOWERLINK

    openPOWERLINK

    An Open Source POWERLINK Stack

    openPOWERLINK is an Open Source Industrial Ethernet stack for communication in hard real-time. It can be operated as both, Managing Node (master) and Controlled Node (slave). With version 2.0, the source code has been cleanly split into an application-oriented user library and a time-critical stack driver. The latter may be moved to a dedicated communication processor or into a kernel module to deliver enhanced performance while still keeping the API in user space. ...
    Leader badge
    Downloads: 36 This Week
    Last Update:
    See Project
  • 3
    The Teachingbox uses advanced machine learning techniques to relieve developers from the programming of hand-crafted sophisticated behaviors of autonomous agents (such as robots, game players etc...) In the current status we have implemented a well founded reinforcement learning core in Java with many popular usecases, environments, policies and learners. Obtaining the teachingbox: FOR USERS: If you want to download the latest releases, please visit:...
    Downloads: 1 This Week
    Last Update:
    See Project
  • 4
    Resistor Value Identifier

    Resistor Value Identifier

    Identify electronic resistor values

    This HTML utility allows the user to select standard color codes or surface mount numbers, then it identifies the resistor value. There is no need to memorize color codes or multipliers. An online working example of this program can be used at ZoomAviation.com/programs.
    Downloads: 0 This Week
    Last Update:
    See Project
  • Full-stack observability with actually useful AI | Grafana Cloud Icon
    Full-stack observability with actually useful AI | Grafana Cloud

    Our generous forever free tier includes the full platform, including the AI Assistant, for 3 users with 10k metrics, 50GB logs, and 50GB traces.

    Built on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
    Create free account
  • 5

    NexusDataLink

    Connect, monitor and control your (embedded) systems remotely. m2m/IoT

    Connect, monitor and control your systems or embedded devices remotely (m2m/IoT) - for example your Raspberry Pi. The communication interface is defined in XML automatically providing a REST interface. NexusDataLink integrates smoothly in existing software or firmware and significantly reduces connection- or communication-related source code.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 6
    WebbotLib
    C library for robot creators using certain AVR microcontrollers with support for 3rd party hardware boards for various sensors and motor controllers. Comprehensive PDF manual included in the download. Version 2 code stream - requires that projects are created using Project Designer from my web site. If your board isn't listed in Project Designer then look at Board Designer on my web-site to create your own custom board - or if its a commercial board then let me know and, with permission,...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 7
    Intend to reproduce most of the conversational programming functionality of the Fagor 8055TC lathe controller. Web browser interface for easy portability between different OS. Visit project web-site for demo.
    Downloads: 0 This Week
    Last Update:
    See Project
  • Previous
  • You're on page 1
  • Next
MongoDB Logo MongoDB