Search Results for "depth-violet"

crawler4j is an open source web crawler for Java which provides a simple interface for crawling the Web. Using it, you can setup a multi-threaded web crawler in few minutes. You need to create a crawler class that extends WebCrawler. This class decides which URLs should be crawled and handles the downloaded page. shouldVisit function decides whether the given URL should be crawled or not. In the above example, this example is not allowing .css, .js and media files and only allows pages...

...See https://sourceforge.net/p/drupalcsvfields/code/HEAD/tree/trunk/examples/basictest/ for an usage example. It worked for me, but use it at your own risk. I have not tested it in depth. Please backup your data before enabling the modules produced by the utility.

Java Desktop application capable to "Crawl" a site and return a report of the status of all the link present at the page, then it moves to another internal page and so on. LinkCrawlers provides a nice HTML5 report with the information of all link per WebPage, Easy to Read. This tool is useful for Web QA testers

...This downloader packs loads of great features such as the search ability. Check the features section and be sure to let me know if you want a feature added. Coming Soon: - Wiki, explaining in depth how to use it more quickly (although its already pretty simple to use) - Ability to download the whole thread, not just images - Better multithreading - Ability to use proxies - Sort images download from searches into folders - Keep original image names - More responsive gui Be sure to let me know if you want any other features.

Bimage is combination of javascript and Servlet scripting meant to allow users to explore images with (near-)infinite depth. Bimage uses javascript to create the images and then save them to the database for later use. This allows users to come together to explore and generate complex, depthful images. Bimage is targeted for Google App Engine (GAE), and uses the datastore for a backend, though it should be adaptable to other backends. This project is set up for eclipse using the GAE plugin. ...

Other spiders has a limited link depth, follows links not randomized or are combined with heavy indexing machines. This spider will has not link depth limits, randomize next url, that will be checked for new urls.