Installation

Installation

Crawlzilla is only stable running in Ubuntu, Debian currently.
The other linux distributions may be some other bugs, we are trying hard to fix it.

Crawlzilla installation procedure:

---

1. Requirement

Crawlzilla is depend on:

• sun-java6-jdk (version >= 1.6)
• ssh
• dialog
• expect

---

2. Master(Server) Installation

2.1 Download Crawlzilla (stable version) form https://sourceforge.net/downloads/crawlzilla/stable/Crawlzilla-0.2/

2.2 Decompress the tar ball

$ tar zxvf crawlzilla-XXX.tar.gz

2.3 Executes slave installation

$ cd crawlzilla-XXX
$ sudo ./install

(By above steps, you can manage and operate Crawlzilla correctly with single node.
If you want to let Crawlzilla run with multiple nodes or cluster, you can follow step 3 Slave Installation)

---

3. Slave Installation

3.1 Copy slave installation from master

$ scp crawler@{Master}:/home/crawler/crawlzilla/source/client_deploy.sh .

(PS: {Master} is your master ip address or hotname.)

3.2 Executes slave deployment

$ ./client_deploy.sh

3.3 Executes slave insatllation

$ cd crawlzilla_client_install; ./client_install