Search Results for "web crawler source code"

Supports strings, arrays, arrays of objects, and nested object structures. The schema is not tied to the structure of the page you're scraping, allowing you to pull the data in the structure of your choosing. The API is entirely composable, giving you great flexibility in how you scrape each page. Paginate through websites, scraping each page. X-ray also supports a request delay and a pagination limit. Scraped pages can be streamed to a file, so if there's an error on one page, you won't...

OpenWebSpider is an Open Source multi-threaded Web Spider (robot, crawler) and search engine with a lot of interesting features!

Small, fast and flexible Content Management System - CMS for PHP / MySQL A very small, fast, compact and flexible Content Management System (CMS) for PHP Webservers using a reasonable amount of functions. Easily maintain your web pages and online files in any webbrowser.

Download search engine and directory with Rapidshare and Torrent - zoozle Download Suchmaschine All The files that run the World Leading German Download Search Engine in 2010 with 500 000 unique visitors a day - all the tools you need to set up a clone. Code Contains: - PHP Files for zoozle - Perl Crawler for gathering new content to database and all other cool tools i have...

Framework (scripts, configuration, code) to build free and public services around travel and leisure data. That project makes an extensive use of already existing data sources such as Geonames and dbPedia, and adds some glue around those (eg, links).

===NOTICE=== After releasing a few updates, but far less than we wanted, we’ve made the decision to stop the OptimizeGoogle Project. The reasons for the decision were that there were not enough people on the team to keep it going. Google is changing things every day and it has become more and more frustrating to look at all the functions go broke piece by piece. The code will remain GPL, perhaps another person or team is interested in picking this up. For now, thank you for all...

Methanol is a scriptable multi-purpose web crawling system with an extensible configuration system and speed-optimized architectural design. Methabot is the web crawler of Methanol.

Torrent30 is an open source AJAX-powered bittorrent search interface. This project has moved to Google Code and is available at http://code.google.com/p/torrent30

Lan-Crawler is a crawler and indexer of public network files shared via SMB shares (Windows shares and UNIX systems running Samba). Meta data is downloaded for films and music. A dynamic Web UI is provided for searching files.

with Zip2Map, one can find the geo map of any zip code(now U.S. only). finding the zip code, returns the Map of the location with its name and state name. Google Maps api has been used with PHP-MySql and lots of Ajax to make it a real WEB 2.0 Application

Open-site PHP code.

Group-CCS development Components, templates, tools, accessories, tutorial, modules, translations, documentation, codes, scripts, everything that can improve the work of who uses the powerful tool of development, CCS - CodeCharge Studio.

Spider is web crawler written in the Java.Based on an Regular expression string the spider parses the internet for web pages matching this string and stores it in an MYSQL database.

studiMaps is a web based application for visualization and analysis of social networks. It consists of two software components: a web-crawler for getting data and the web based application for visualization.