Clonezilla / Discussion / Clonezilla server edition: Clonezilla Bittorrent Server-Client 3x disks clone hanging

Marko Šinko - 2019-08-09

Hello.

I am using Clonezilla Server Lite (stable version) to clone multiple (2x 1TB + 1 120GB disks) to all Clients (45) that have the same hardware as the server (server has 16GB of ram, is that enough?).

The clone process starts and it works fine (6GB/min on all clients) but it random stops. For example "P: 35%" and it wont continue to download (on all clients stuck at same %), example image provided.

If I restart the server PC and redo the bittorrent cloning setup the download on the clients continues from the last point it stopped.

Is there a fix to that? I am losing a lot of time to restart the server and create the Bittorrent images of 1TB drives (around 40mins).

Could I have those images stored so I don't have to redo them 4-5x for all the disks to clone till completion.

Also sometimes some client (1 in 10) stops to download completely and I need to redo the whole process for that client.

Thank you for help.

ps. I don't know how to access log files, so if someone knows how I can access server from another pc and read the log, would be great :D

Last edit: Marko Šinko 2019-08-09

problem.jpg

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Date Huang - 2019-08-09

hi
which version did you use?
could you try to use testing version?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Marko Šinko - 2019-08-09

I have used "stable - 2.6.2-15", which testing version should I use?

The server has 16GB of ram, could that be a problem for 1TB drives and 40 clients?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Date Huang - 2019-08-09

16g ram is enough for server
try to use latest tesing version

log file is in /var/log/
and if possible, pack all the file and send to us
thanks

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Marko Šinko - 2019-08-09

How can I access from remote the server?
If I use putty it gets "connection rejected".

Will try to use the latest version and tell how it works with that, thank you for the info.

How can I access log file if I have clonezilla loaded into ram, can't I remote connect to server and access the files?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Date Huang - 2019-08-09

remove duplicate msg

Last edit: Date Huang 2019-08-09

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Date Huang - 2019-08-09

I'm not sure how to access remote server.
maybe you can install ssh service on the server
or just use the scp to send info out
(I only developed the BT module)

and plz provide the output of ps aux when the proccessing is stuck.(from server)
(I want to make sure that other program works normally)

thanks

Last edit: Date Huang 2019-08-09

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Date Huang - 2019-08-09

By the way
Please test this
modify the config file at /etc/drbl/drbl-ocs.conf
ezio_seeder_opt="-t 1 -k 60"
remove -t 1

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Marko Šinko - 2019-08-10

Thanks for the information. I have tried that and it's not working.

I'm sure the config file was edited and saved before starting the Server Lite.
I did not have that problem cloning my 120GB hard drives, hope you can see what's wrong from the logs.

Clonezilla version used: 2.6.2-20
Exact the same behaviour as the previos version.

Here is a link to 3 different log files.

"Log Server" - it was exported at the moment server stopped sending data. At this point I need to restart server and redo the bittorrent images of the hard drives so that the clients can continue.
Server was configued to seed 3 SSD drives over bittorrent directly to all Client PC's (all clients have the same 3 SSD drives, connected at same inputs on motherboard)
From my experience around each 500GB of transfered data the server stops sending more data till restart. I have around 2000GB of data, so need to restart the server 3x to finish the cloning process

"Log Stuck Client" - some clinets like this one random hangs at random percentages (mostly between 3% and 10%) and if that happens I need to restart the whole process again, don't know what happense and how to make the client continue the disk cloning, because of some clients like that I need to redo the whole process

"Log Client waiting for stuck Server" - this are the most of the clients, they all stop at certain percentage (all at same percentage) and will continue if I restart and make on the server again bittorrent disk images and they start seeding again

Hope you can provide me with some info about the situation and what's wrong. Thank you!

After 3x server restart most of the clients finish cloning of all 3 SSD drives but that takes huge amount of extra time (bittorrent image creation takes almost 2hours for 2000GB and I need to do that 3x average)

Last edit: Marko Šinko 2019-08-10

Logs.zip

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Date Huang - 2019-08-10

did you restart the process after modifying?
and we are trying to reproduce this issue in our side
thanks for reporting it

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Marko Šinko - 2019-08-10

I have updated the log files, please check them when you have time :)

I have loaded Clonezilla into RAM and started to clone 3 SSD drives at once using bittorrent.

There are 2x 1TB SSD drives and 1x 128GB SSD drive.

When copy the 128GB drive there are no problems.
If I copy 1TB SSD drive (even if I only clone that one drive instead of 3) it always hangs on the server side. At around 500GB transfered I need to redo the Server Lite to be able to finish the cloning.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Date Huang - 2019-08-11

Thanks for log
we will dig for info.

BTW you refresh the record about biggest size image using BT.
We always test BT module with 150G image

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Marko Šinko - 2019-08-12

Thank you :)

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Date Huang - 2019-08-12

we are still digging

if possible, please try 2.6.1-25

https://sourceforge.net/projects/clonezilla/files/clonezilla_live_stable/2.6.1-25/

Last edit: Date Huang 2019-08-12

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Marko Šinko - 2019-08-12

Sure, will try and keep you informed. Smaller disks copy without problem but the problem only occurs between 300-600GB copied. If I restart the server and redo the disk bittorrent images, the clients continue and finish the job (or hang again after another 300-600GB copied)

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Date Huang - 2019-08-12

maybe you can try to deploy one disk at a time
not 3 disks at the same time

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Marko Šinko - 2019-08-12

Sure, will try and keep you informed. Smaller disks copy without problem but the problem only occurs between 300-600GB copied. If I restart the server and redo the disk bittorrent images, the clients continue and finish the job (or hang again after another 300-600GB copied)

I have tried just 1 disk, but the disks that are 1000GB have problems to copy whole, it stucks at around 50-60%

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Date Huang - 2019-08-12

hmm
ok
try to use older version or wait for us to update

we still dig info

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steven Shiau - 2019-08-21

Please give testing Clonezilla live 2.6.3-3 or 20190821-* a try:
https://clonezilla.org/downloads.php
We have fixed some issues, and improve the BT-related functions.
Please let us know the results.
Thank you very much.

Steven

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Marko Šinko - 2019-08-23

Hello,

thanks for the update. I have tried 2.6.3-3 version of Clonezilla and it's working great now!

I have tried to copy single 1TB data drive (900GB data on the disk)couple of times and it worked every time, without any hanging from the server site.

I have also tried to copy multiple disks at once (2x 1TB + 1x120GB) and it worked for all disks without any problems. No corruption or bad sectors

I have copied to 45 PC's data at once for 3x disks at the same time, worked like a charm.

Really powerfull tool for direct disk cloning, thank you very much for your help.
I will keep you updated if I find anything not working properly.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steven Shiau - 2019-08-24

This is great! Thanks for your feedback.
BTW, if you do not mind, please share the stats you have, like to disk size, file system size, the total time to deploy the whole environment... It would be easier for us to understand the performance.
Appreciate.

Steven

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Date Huang - 2019-08-24

Thanks a lot!!

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Saham Ali - 2019-12-12

Has this been resolved?
Trying to use the latest stable and alternative versions seem to do the same thing, stop at 99% and do not continue.

What build is working for everyone? 2.6.3-3 ??

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Date Huang - 2019-12-13
  
  try to use latest testing version
  
  and could you describe more about your snvrionment and condition you met
  rks
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
  - Saham Ali - 2019-12-13
    
    Yup,
    Tried the latest Stable, then tried
    2.6.5-5
    clonezilla-live-20191024-eoan-amd64
    
    Both seem to do the same thing.
    Current setup is
    PC with 1TB SATA HDD , with a single image loaded on the drive. connected via ethernet to a GS108 switch.
    
    2 x PC connected via ethernet to same netgear GS108 switch.
    
    BT Imaging starts, Gets to about 99% but never finishes.
    
    If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Clonezilla Bittorrent Server-Client 3x disks clone hanging

A partition and disk imaging/cloning program

Forums

Help

Clonezilla Bittorrent Server-Client 3x disks clone hanging

Clonezilla Bittorrent Server-Client 3x disks clone hanging

A partition and disk imaging/cloning program

Forums

Help

Clonezilla Bittorrent Server-Client 3x disks clone hanging document.SUBSCRIPTION_OPTIONS = { "thing": "topic", "subscribed": false, "url": "subscribe", "icon": { "css": "fa fa-envelope-o" } };

Clonezilla Bittorrent Server-Client 3x disks clone hanging