Search Results for "web crawler source code"

This is a high-performance, production-tested library for parsing and evaluating robots.txt rules against crawler user agents. It implements the core semantics of the Robots Exclusion Protocol: user-agent sections, Allow/Disallow directives, wildcard handling, and precedence rules. The code is optimized for speed and low memory so large crawls can evaluate millions of URLs quickly. It also focuses on correctness—edge cases like overlapping patterns and longest-match resolution are handled...

openPOWERLINK is an Open Source Industrial Ethernet stack for communication in hard real-time. It can be operated as both, Managing Node (master) and Controlled Node (slave). With version 2.0, the source code has been cleanly split into an application-oriented user library and a time-critical stack driver. The latter may be moved to a dedicated communication processor or into a kernel module to deliver enhanced performance while still keeping the API in user space. ...

The Teachingbox uses advanced machine learning techniques to relieve developers from the programming of hand-crafted sophisticated behaviors of autonomous agents (such as robots, game players etc...) In the current status we have implemented a well founded reinforcement learning core in Java with many popular usecases, environments, policies and learners. Obtaining the teachingbox: FOR USERS: If you want to download the latest releases, please visit:...

This HTML utility allows the user to select standard color codes or surface mount numbers, then it identifies the resistor value. There is no need to memorize color codes or multipliers. An online working example of this program can be used at ZoomAviation.com/programs.

C library for robot creators using certain AVR microcontrollers with support for 3rd party hardware boards for various sensors and motor controllers. Comprehensive PDF manual included in the download. Version 2 code stream - requires that projects are created using Project Designer from my web site. If your board isn't listed in Project Designer then look at Board Designer on my web-site to create your own custom board - or if its a commercial board then let me know and, with permission,...