vB-mTurk-Scraper-1.1 download

vB-mTurk-Scraper
Can scrape vBulletin forums for links to mTurk HITS

Written in: Python 2.* (Not 3 Compatible OR Tested)
Requires: BeautifulSoup and requests

To install BeautifulSoup run: pip install beautifulsoup4 OR: easy_install beautifulsoup4

To install requests run: pip install requests OR: easy_install requests

Information required to run program:

HITS Thread Number (Changes Daily: 5 digit number found in the thread URL)
Forum URL
Page To Start From
Number Of Pages To Scrape

This is a very very simple command-line script. Simply run python vB-mTurk-Scraper.py and it will guide you through setting the forum you want to scrape, entering todays thread number, which page you want to start from, and how many pages you want to go through.

Note: When entering the address to the forum, enter only the domain, ex: forum.com

It outputs all HITS to the html file mturklinks.html and always overwrites it self.

Read more @: http://git.io/vqpHS

Project Samples

vB-mTurk-Scraper running on Xubuntu 15.04

Project Activity

See All Activity >

Follow vB-mTurk-Scraper-1.1

vB-mTurk-Scraper-1.1 Web Site

Other Useful Business Software

Sales CRM and Pipeline Management Software | Pipedrive

The easy and effective CRM for closing deals

Pipedrive’s simple interface empowers salespeople to streamline workflows and unite sales tasks in one workspace. Unlock instant sales insights with Pipedrive’s visual sales pipeline and fine-tune your strategy with robust reporting features and a personalized AI Sales Assistant.

Try it for free

Rate This Project