#50 Multicast progress percent fix (0.31)

Unstable (example)
closed
None
5
2014-05-26
2011-07-07
Cam
No

The multicast status on tasks>active multicast tasks page does not show the percent for multi-partition images

The issue there's an IF statement in /opt/fog/service/commons/lib/MulticastTask.php that has no handling for type 2&3 image types

The fix, the percent calculation requires the file size, but its a directory, so add this function (to the end of the file, but before the ?> tag)
to /opt/fog/service/common/functions.php which will allow finding the size of a dir

<code>
function dirSize($directory) {
$size = 0;
foreach(new RecursiveIteratorIterator(new RecursiveDirectoryIterator($directory)) as $file){
$size+=$file->getSize();
}
return $size;
}
</code>

so now we can get the combined file size of the image, now implemnt it into /opt/fog/service/commons/lib/MulticastTask.php

at about line 320 change this:
<code>
$file = $this->getImagePath();
if ( $this->getImageType() == 0 || $this->getImageType() == 1 )
$intImageSize = trim(`stat -c%s $file 2>/dev/null`);
else
{
$intImageSize = -1;
}
</code>
to this:
<code>
$file = $this->getImagePath();
if ( $this->getImageType() == 0 || $this->getImageType() == 1 )
$intImageSize = trim(`stat -c%s $file 2>/dev/null`);

elseif($this->getImageType() == 2 || $this->getImageType() == 3) $intImageSize = dirSize($file);

else
{
$intImageSize = -1;
}
</code>

and then verify the MYSQL settings in /opt/fog/service/etc/config.php are correct (these don't seem to get setup when installing FOG)

then run /etc/init.d/FOGMulticastManager restart
run this command twice and make sure it outputs no errors
if it does run /opt/fog/service/FOGMulticastManager/FOGMulticastManager and it should say what lines the error is on

Discussion

  • Cam

    Cam - 2011-07-08

    IMPORTANT - I only realized just now
    the dirSize function gets the compressed file size but the multicast sends decompressed data
    resulting in silly readings (134% for example)

    Use this instead of the dirSize function above
    [code]

    function getunzipsize($FileRead){
    $FileOpen = fopen($FileRead, "rb");
    fseek($FileOpen, -4, SEEK_END);
    $buf = fread($FileOpen, 4);
    $GZFileSize = end(unpack("V", $buf));
    fclose($FileOpen);
    return $GZFileSize;
    }

    function dirSize($directory) { //cam added from pp site
    $handle = opendir($directory);
    while (false !== ($file = readdir($handle))) {
    if(substr($file, -3, 3)=="img"){ //if file is an img file
    $size += getunzipsize($directory."/".$file); //find its decompressed size and add it
    }
    }
    return $size/2;
    }

    [/code]

    I'm not sure why but the percent gets halved later in the code, the /2 at the end doubles the result to protect against this.
    Dont forget to restart the service again.

     

    Related

    Code: code

  • Blackout

    Blackout - 2011-08-04

    What if the images are not stored on the Web host?

    i.e. i run a virtual for a "fog controller" which does not store the images, they are on other storage nodes

     
  • Schlitz Gesicht

    Schlitz Gesicht - 2011-09-23

    Thanks for your patch. I tried it with the new dirSize function, but the percentage reading is still off (e.g. 1100% when almost complete). And if I'm not mistaken it went even higher when I multicasted to more machines (around 5000%).
    Weren't multicasts changed in 0.30 to send compressed image files anyway?

     
  • Cam

    Cam - 2011-09-23

    Sorry about this,

    I now see why they didn't have the multicast percent.
    When I did the testing on my VM I was using an image of about 2GB (XP)

    Turns out the way gzip stores its decompressed size is in the last 4 bytes of the file.
    the 32bit number overflows at 4GB. so my 2GB image worked great, but when I put it on my production server I also got ridiculous numbers (Windows 7, 24GB)

    In the end I think I just made it output the data sent. when it gets near 24 I know its almost done.

    "What if the images are not stored on the Web host?"
    well the storage node should also be running the multicast service, in which case it would need to be patched.

     
  • mastacontrola

    mastacontrola - 2014-05-26
    • status: open --> closed
    • assigned_to: mastacontrola
    • Group: --> Unstable (example)
     
  • mastacontrola

    mastacontrola - 2014-05-26

    With the move to partclone and the patching included to make the progress system work, we can now see the percentage of all hosts within the multicast task. This updates the tasks on the Active Tasks element and should be an accurate representation of the "percent" within the multicast tasks as well.

    Closing ticket as it is now "fixed" I think. If there's any issues with this still, just let us know on the forums.

     

Log in to post a comment.