Search Results for "crawl"
Sort By:
Showing 7 open source projects for "crawl"
View related business solutions-
Chat with your business data with Looker. More than just a modern business intelligence platform, you can turn to Looker for self-service or governed BI, build your own custom applications with trusted metrics, or even bring Looker modeling to your existing BI environment.
-
Venn is a secure workspace for remote work that isolates and protects work from any personal use on the same computer. Work lives in a secure local enclave that is company controlled, where all data is encrypted and access is managed. Within the enclave – visually indicated by the Blue Border around these applications – business activity is walled off from anything that happens on the personal side. As a result, work and personal uses can now safely coexist on the same computer.
-
1
WFDownloader App
Free batch downloader for image, wallpaper, video, audio, document,
Use as an image gallery, wallpaper, audio/music, video, document, and other media bulk downloader from supported websites. Also use to download sequential website urls that have a certain pattern (e.g. image01.png to image100.png). Also use app's built-in site crawler for advanced link search or extraction. There is also special support for forum media and open directory downloading. It's a programmable downloader and also works with password protected sites. Say goodbye to downloading one... -
2
fpart
Sort files and pack them into partitions
... can then be used by third party programs. Fpart also includes a live mode, which allows it to crawl very large filesystems and produce partitions in live. Hooks are available to act on those partitions (e.g. immediatly start a transfer using rsync(1)) without having to wait for the filesystem traversal job to be finished. Used this way, fpart can be seen as a powerful data migration tool. -
3
Headless Chrome Crawler
Distributed crawler powered by Headless Chrome
Crawlers based on simple requests to HTML files are generally fast. However, it sometimes ends up capturing empty bodies, especially when the websites are built on such modern frontend frameworks as AngularJS, React and Vue.js. Powered by Headless Chrome, the crawler provides simple APIs to crawl dynamic websites. Support both depth-first search and breadth-first search algorithm. Save screenshots for the crawling evidence, emulate devices and user agents, priority queue for crawling efficiency... -
4
pyspider
A powerful Spider(Web Crawler) system in Python
... and third service free instance. Or using MySQL or MongoDB and RabbitMQ to deploy a distributed crawl cluster. To deploy pyspider in product environment, running component in each process and store data in database service is more reliable and flexible. To deploy pyspider components in each single processes, you need at least one database service. pyspider now supports MySQL, MongoDB and PostgreSQL. You can choose one of them. -
Save time, money and your sanity. Deluxe Payment Exchange+ (DPX+) is our integrated payments solution that streamlines and automates your accounts payable (AP) disbursements. DPX+ ensures secure payments and offers suppliers alternate ways to receive funds, including mailed checks, ACH, virtual credit cards, debit cards, or eCheck payments. By simply integrating with your existing accounting software like QuickBooks®, you’ll implement efficient payment solutions for AP with ease—without costly development fees or untimely delays.
-
5
Proxyp
Multithreaded Proxy Enumeration Utility
Proxyp is a small multithreaded Perl script written to enumerate latency, port numbers, server names, & geolocations of proxy IP addresses. This script started as a way to speed up use of proxychains, which is why I've added an append option for resulting live IP addresses to be placed at the end of a file if need be. Requires IP::Country module and root/administrator privileges. "No man is free who is not master of himself" --Epictetus "For a man to conquer himself is the first... -
6
The “Media Crawler” is an extensible Eclipse RCP based desktop application which will crawl a given file system, extract metadata from files, map metadata to internal schemas and store the metadata in a databse. This project is ANDS-funded.
-
7
Toke is a webmining toolkit for web exploring, indexing and searching for Java. Toke allows to you crawl public or private web sites, in order to create web estatistics, web Pajek graphs, Lucene indexs and word frequency files for data clustering.
- Previous
- You're on page 1
- Next