Recent changes to Home

Home modified by Mahesh

Mahesh — Sun, 17 May 2015 06:34:32 -0000

--- v14
+++ v15
@@ -3,7 +3,7 @@

 Multi-bitrate Video processing requires lots of computing power and time to process full movie. There are different open source video transcoding and processing tools freely available in Linux, like libav-tools, ffmpeg, mencoder, and handbrake. However, none of these tools support **PARALLEL** computing easily.

-After some research, I found amazing [solution](http://blog.dustinkirkland.com/2014/07/scalable-parallel-video-transcoding-on.html) designed & developed by '[Dustin Kirkland](http://blog.dustinkirkland.com/2014/07/scalable-parallel-video-transcoding-on.html)' based on Ubuntu JUJU and [avconv](https://libav.org/avconv.html). But our requirement was little bit diffrent from Dustins's solution. Our requirement was to convert single video in Multi-bitrate and in formats like 3gp, flv. Also we want to build this solution on top of CentOS and ffmpeg. So I decided to design and develop "Simple Scalable, Parallel, Multi-bitrate Video Transcoding System" by myself. Here is my solution.
+After some research, I found amazing [solution](http://blog.dustinkirkland.com/2014/07/scalable-parallel-video-transcoding-on.html) designed & developed by '[Dustin Kirkland](http://blog.dustinkirkland.com/2014/07/scalable-parallel-video-transcoding-on.html)' based on Ubuntu JUJU and [avconv](https://libav.org/avconv.html). But our requirement was little bit diffrent from Dustins's solution. Our requirement was to convert single video in Multi-bitrate and in formats like 3gp, flv and upload them to single or multiple CDN(like Akamai or tata). Also we want to build this solution on top of CentOS and ffmpeg. So I decided to design and develop "Simple Scalable, Parallel, Multi-bitrate Video Transcoding System" by myself. Here is my solution.

 The Algorithm is same as Dustin's solution but with some changes:

Home modified by Mahesh

Mahesh — Sun, 17 May 2015 06:34:32 -0000

--- v13
+++ v14
@@ -47,7 +47,7 @@
 7. restart pure-ftp service
 8. Change Master/DB IP in all three scripts (MASTER_NODE variable)
 9. Make sure to change WORKERS variable in CallUpload.sh script(all servers hostname)
-10. Install mysql-server and create 'transcode' database. Create 'transcode' user with password same as username. Make sure user is able to connect from all of the worker nodes.
+10. Install mysql-server and import SQL file 'transcoding.sql'. Create 'transcode' user with password same as username. Make sure user is able to connect from all of the worker nodes.
 11. NFS Export /srv directory and mount it on all nodes with NFS client option "lookupcache=none"

 10. start screen named master and run below command

Home modified by Mahesh

Mahesh — Sun, 17 May 2015 06:34:32 -0000

--- v12
+++ v13
@@ -47,8 +47,9 @@
 7. restart pure-ftp service
 8. Change Master/DB IP in all three scripts (MASTER_NODE variable)
 9. Make sure to change WORKERS variable in CallUpload.sh script(all servers hostname)
-10. Install mysql-server and create transcode database. Create transcode user with password same as username. Make sure user is able to connect from all of the worker nodes.
-11. 
+10. Install mysql-server and create 'transcode' database. Create 'transcode' user with password same as username. Make sure user is able to connect from all of the worker nodes.
+11. NFS Export /srv directory and mount it on all nodes with NFS client option "lookupcache=none"
+
 10. start screen named master and run below command

    `# screen -S master`

Home modified by Mahesh

Mahesh — Sun, 17 May 2015 06:34:31 -0000

--- v11
+++ v12
@@ -34,7 +34,7 @@
 2. Download and copy all scripts to /srv directory
 3. Change file permission to 755
 4. Install Pure-FTPD and change CallUploadscript directive to yes in /etc/pure-ftpd.conf file
-5. Create test user and set password
+5. Create test user for FTP and set password

    `# useradd -m ftptest; passwd ftptest`

@@ -47,6 +47,8 @@
 7. restart pure-ftp service
 8. Change Master/DB IP in all three scripts (MASTER_NODE variable)
 9. Make sure to change WORKERS variable in CallUpload.sh script(all servers hostname)
+10. Install mysql-server and create transcode database. Create transcode user with password same as username. Make sure user is able to connect from all of the worker nodes.
+11. 
 10. start screen named master and run below command

    `# screen -S master`

Home modified by Mahesh

Mahesh — Sun, 17 May 2015 06:34:31 -0000

--- v10
+++ v11
@@ -36,25 +36,25 @@
 4. Install Pure-FTPD and change CallUploadscript directive to yes in /etc/pure-ftpd.conf file
 5. Create test user and set password

-`# useradd -m ftptest; passwd ftptest`
-
+   `# useradd -m ftptest; passwd ftptest`
+   
 6. Run below commands to change pure-ftpd init script

-`# sed -i 's#start() {#start() {\n\t/usr/sbin/pure-uploadscript -B -r /srv/CallUpload.sh#g' /etc/init.d/pure-ftpd`
+   `# sed -i 's#start() {#start() {\n\t/usr/sbin/pure-uploadscript -B -r /srv/CallUpload.sh#g' /etc/init.d/pure-ftpd`

-`# sed -i 's#stop() {#stop() {\n\tkillall -9 pure-uploadscript#g' /etc/init.d/pure-ftpd`
+   `# sed -i 's#stop() {#stop() {\n\tkillall -9 pure-uploadscript#g' /etc/init.d/pure-ftpd`

 7. restart pure-ftp service
 8. Change Master/DB IP in all three scripts (MASTER_NODE variable)
 9. Make sure to change WORKERS variable in CallUpload.sh script(all servers hostname)
 10. start screen named master and run below command

-`# screen -S master`
+   `# screen -S master`

-`# bash -x /srv/transcode-master.sh > /home/master.log  2>&1`
+   `# bash -x /srv/transcode-master.sh > /home/master.log  2>&1`

 11. start screen named nodes and run below command

-`# screen -S nodes`
+   `# screen -S nodes`

-`# bash -x /srv/transcode-nodes.sh > /home/nodes.log  2>&1`
+   `# bash -x /srv/transcode-nodes.sh > /home/nodes.log  2>&1`

Home modified by Mahesh

Mahesh — Sun, 17 May 2015 06:34:31 -0000

--- v9
+++ v10
@@ -36,22 +36,25 @@
 4. Install Pure-FTPD and change CallUploadscript directive to yes in /etc/pure-ftpd.conf file
 5. Create test user and set password

-   `# useradd -m ftptest; passwd ftptest`
+`# useradd -m ftptest; passwd ftptest`
+
 6. Run below commands to change pure-ftpd init script

-   `# sed -i 's#start() {#start() {\n\t/usr/sbin/pure-uploadscript -B -r /srv/CallUpload.sh#g' /etc/init.d/pure-ftpd`
+`# sed -i 's#start() {#start() {\n\t/usr/sbin/pure-uploadscript -B -r /srv/CallUpload.sh#g' /etc/init.d/pure-ftpd`

-   `# sed -i 's#stop() {#stop() {\n\tkillall -9 pure-uploadscript#g' /etc/init.d/pure-ftpd`
+`# sed -i 's#stop() {#stop() {\n\tkillall -9 pure-uploadscript#g' /etc/init.d/pure-ftpd`

 7. restart pure-ftp service
 8. Change Master/DB IP in all three scripts (MASTER_NODE variable)
 9. Make sure to change WORKERS variable in CallUpload.sh script(all servers hostname)
 10. start screen named master and run below command

-   `# screen -S master`
-   `# bash -x /srv/transcode-master.sh > /home/master.log  2>&1`
+`# screen -S master`
+
+`# bash -x /srv/transcode-master.sh > /home/master.log  2>&1`

 11. start screen named nodes and run below command

-   `# screen -S nodes`
-   `# bash -x /srv/transcode-nodes.sh > /home/nodes.log  2>&1`
+`# screen -S nodes`
+
+`# bash -x /srv/transcode-nodes.sh > /home/nodes.log  2>&1`

Home modified by Mahesh

Mahesh — Sun, 17 May 2015 06:34:31 -0000

--- v8
+++ v9
@@ -35,11 +35,12 @@
 3. Change file permission to 755
 4. Install Pure-FTPD and change CallUploadscript directive to yes in /etc/pure-ftpd.conf file
 5. Create test user and set password
+
    `# useradd -m ftptest; passwd ftptest`
 6. Run below commands to change pure-ftpd init script

    `# sed -i 's#start() {#start() {\n\t/usr/sbin/pure-uploadscript -B -r /srv/CallUpload.sh#g' /etc/init.d/pure-ftpd`
-***
+
    `# sed -i 's#stop() {#stop() {\n\tkillall -9 pure-uploadscript#g' /etc/init.d/pure-ftpd`

 7. restart pure-ftp service
@@ -51,5 +52,6 @@
    `# bash -x /srv/transcode-master.sh > /home/master.log  2>&1`

 11. start screen named nodes and run below command
+
    `# screen -S nodes`
    `# bash -x /srv/transcode-nodes.sh > /home/nodes.log  2>&1`

Home modified by Mahesh

Mahesh — Sun, 17 May 2015 06:34:31 -0000

--- v7
+++ v8
@@ -39,6 +39,7 @@
 6. Run below commands to change pure-ftpd init script

    `# sed -i 's#start() {#start() {\n\t/usr/sbin/pure-uploadscript -B -r /srv/CallUpload.sh#g' /etc/init.d/pure-ftpd`
+***
    `# sed -i 's#stop() {#stop() {\n\tkillall -9 pure-uploadscript#g' /etc/init.d/pure-ftpd`

 7. restart pure-ftp service

Home modified by Mahesh

Mahesh — Sun, 17 May 2015 06:34:31 -0000

--- v6
+++ v7
@@ -35,23 +35,20 @@
 3. Change file permission to 755
 4. Install Pure-FTPD and change CallUploadscript directive to yes in /etc/pure-ftpd.conf file
 5. Create test user and set password
-`# useradd -m ftptest; passwd ftptest`
+   `# useradd -m ftptest; passwd ftptest`
 6. Run below commands to change pure-ftpd init script

-`# sed -i 's#start() {#start() {\n\t/usr/sbin/pure-uploadscript -B -r /srv/CallUpload.sh#g' /etc/init.d/pure-ftpd`
+   `# sed -i 's#start() {#start() {\n\t/usr/sbin/pure-uploadscript -B -r /srv/CallUpload.sh#g' /etc/init.d/pure-ftpd`
+   `# sed -i 's#stop() {#stop() {\n\tkillall -9 pure-uploadscript#g' /etc/init.d/pure-ftpd`

-`# sed -i 's#stop() {#stop() {\n\tkillall -9 pure-uploadscript#g' /etc/init.d/pure-ftpd`
-
-7. start pure-ftp service
+7. restart pure-ftp service
 8. Change Master/DB IP in all three scripts (MASTER_NODE variable)
 9. Make sure to change WORKERS variable in CallUpload.sh script(all servers hostname)
 10. start screen named master and run below command

-`# screen -S master`
-
-`# bash -x /srv/transcode-master.sh > /home/master.log  2>&1`
+   `# screen -S master`
+   `# bash -x /srv/transcode-master.sh > /home/master.log  2>&1`

 11. start screen named nodes and run below command
-
-`# screen -S nodes`
-`# bash -x /srv/transcode-nodes.sh > /home/nodes.log  2>&1`
+   `# screen -S nodes`
+   `# bash -x /srv/transcode-nodes.sh > /home/nodes.log  2>&1`

Home modified by Mahesh

Mahesh — Sun, 17 May 2015 06:34:30 -0000

--- v5
+++ v6
@@ -39,13 +39,19 @@
 6. Run below commands to change pure-ftpd init script

 `# sed -i 's#start() {#start() {\n\t/usr/sbin/pure-uploadscript -B -r /srv/CallUpload.sh#g' /etc/init.d/pure-ftpd`
+
 `# sed -i 's#stop() {#stop() {\n\tkillall -9 pure-uploadscript#g' /etc/init.d/pure-ftpd`
+
 7. start pure-ftp service
 8. Change Master/DB IP in all three scripts (MASTER_NODE variable)
 9. Make sure to change WORKERS variable in CallUpload.sh script(all servers hostname)
 10. start screen named master and run below command
+
 `# screen -S master`
+
 `# bash -x /srv/transcode-master.sh > /home/master.log  2>&1`
+
 11. start screen named nodes and run below command
+
 `# screen -S nodes`
 `# bash -x /srv/transcode-nodes.sh > /home/nodes.log  2>&1`