I'm cloning a VMFS 5 partition which contains 2 VMDK files attached to a Linux VM. The fisrt VMDK file is a system disk of 16GB, the second is a 500GB data disk. I used the following command to create an image of the partiton to another internal disk :
Everything looks OK after cloning, no error message. But when I restored the image, the large virtual data disk (500GB) is not available any more : neither partition nor data can be found on the disk. I checked the image file genereted by PartClone, it is only 18GB ! Can you help me to sort it out. Here is the output :
Thanks
user@debian:/home/partimag$ sudo partclone.vmfs5 -L /home/partimag/log.txt -c -s /dev/sda3 -C --output myimage.img
Partclone v0.2.66 http://partclone.org
Starting to clone device (/dev/sda3) to image (myimage.img)
Reading Super Block
Elapsed: 00:00:08, Remaining: 00:00:00, Completed: 100.00%
Total Time: 00:00:08, 100.00% completed!
done!
File system: VMFS
Device size: 579.3 GB = 552448 Blocks
Space in use: 556.0 GB = 530214 Blocks
Free Space: 23.3 GB = 22234 Blocks
Block size: 1048576 Byte
Elapsed: 00:04:29, Remaining: 00:00:00, Completed: 100.00%, Rate: 4.16GB/min,
current block: 167160, total block: 552448, Complete: 100.00%
Total Time: 00:04:29, Ave. Rate: 4.2GB/min, 100.00% completed!
Syncing... OK!
Partclone successfully cloned the device (/dev/sda3) to the image (myimage.img)
Cloned successfully.
user@debian:/home/partimag$ cat log.txt
Partclone v0.2.66 http://partclone.org
Starting to clone device (/dev/sda3) to image (myimage.img)
Reading Super Block
we need memory: 1121796 bytes
image head 4160, bitmap 69056, crc 1048580 bytes
Calculating bitmap... Please wait... vmfs5clone.c: Block 0x69442023 is used but not allocated.
vmfs5clone.c: Block 0x6d783f3c is used but not allocated.
vmfs5clone.c: Block 0x69442023 is used but not allocated.
vmfs5clone.c: Used:0, Free:0, Status err:0
done!
File system: VMFS
Device size: 579.3 GB = 552448 Blocks
Space in use: 556.0 GB = 530214 Blocks
Free Space: 23.3 GB = 22234 Blocks
Block size: 1048576 Byte
Total block 552448
Syncing... OK!
Partclone successfully cloned the device (/dev/sda3) to the image (myimage.img)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
In my ESXi 5.1 environment, I created a virtual disk of 500G attached to a Linux virtual machine. It's configured as a second disk of the VM, formatted as ext4 partition with some data copied in it. While cloned and restored, I couldn't read the disk, there was no partition found in this disk.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
hi,
just want to make sure the version is esxi5.1 or esxi5.1update1?
and the machine is created from VMware vSphere Client?
and the space is correct for you?
Device size: 579.3 GB = 552448 Blocks
Space in use: 556.0 GB = 530214 Blocks
I am downloading esxi 5.1 and 5.1update1 to give a test.
Thanks.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Not really.
We were able to reproduce the issue. However, it seems the issue exists only when the vmdk file is very large. Then it becomes very difficult to debug and reproduce the problem.
Is your vmdk file is very large? Could you please let us know the size?
Thanks.
Steven.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'm also encountering this exact same problem. It only does it on VMFS-5 datastores for me. If I create the same exact ESXi 5.0 or 5.1 configuration but with a VMFS-3 primary datastore, the whole cloning process works fine and my virtual machines load up perfectly regardless of how large the virtual disk is.
I'm primarily seeing this problem on a 600 GB thin-provisioned virtual disk running on ESXi 5.1 with a Linux-based operating system in the Virtual Machine. I also created an additional 600 GB virtual machine running Windows 2003 on the same host and cloned the entire host using Clonezilla/Partclone to test if the problem had anything to do with the guest operating system type. Upon deploying this ESX image to a different host machine (identical hardware as the original), neither the Linux nor the Windows Virtual machines appeared to have any data on their virtual disks when I attempted to boot them up. Smaller virtual disks seem to clone OK though, at least for Windows-based virtual machines(I didn't really try it with any Linux-based ones in a smaller virtual disk).
Please let me know if there is any other info that I could contribute or other test scenarios that I could try and reply back with some feedback in order to help get to the root of the problem and troubleshoot it further.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I did a bunch of testing today using Clonezilla Live and Partclone to create images of an ESXi 5.1 host with virtual machine data stored in the default VMFS-5 data store (locally on the host machine's disk) and a single Virtual Machine in it. I iteratively made different sized virtual disks for my virtual machine, put a Windows 2003 image onto the virtual machine, and then shut down the host and cloned the entire host to another identical version of the host hardware using PartClone.
What I found out was this:
-Everything works fine with cloning and restoring our image if the guest VM is configured with a virtual disk that is 256 GB or smaller in size. If you make it 257 GB, or anything greater than that, it will appear to be blank upon restoring the image of the entire host to a different (but identical hardware-wise) machine.
-The Cloning problem happens regardless of whether we are thin-provisioning our guest VM's hard drive or thick-provisioning it (so provisioning has nothing to do with the problem).
Basically, virtual disks larger than 256 GB in VMFS-5 appear to be the culprit when used in conjunction with PartClone. I'm not sure what VMWare is doing in VMFS-5 over 256 GB but it does not appear that any currently available versions of PartClone are compatible with it.
Hope this helps, and please let me know if there is anything else I can contribute!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks for your troubleshooting.
This issue has been reported to our upstream, vmfs-toos: https://github.com/glandium/vmfs-tools/issues/12
We are still wait for the upstream to fix this. Maybe you can also try what Thomas has found, and give some more info on the upstream's forum.
Appreciate!
Steven.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The issue appears to be related to Block Size. VMFS-2 and VMFS-3 used to have specific VMDK file-size limitations based on the block size of your datastore:
Block Size 1 MB = 256 GB Max Virtual Disk size
Block Size 2 MB = 512 GB Max Virtual Disk size
Block Size 4 MB = 1 TB Max Virtual Disk size
Block Size 8 MB = 2 TB Max Virtual Disk size (minus 512 B)
VMFS-5 automatically uses a 1 MB block size but can support virtual disk sizes up to 2 TB.
I did a test where I installed ESXi 4.1 on a host, and created a 2 MB Block sized primary Datastore on the local ESXi host hard drive. I then installed ESXi 5.1 over that and upgraded the datastore to VMFS-5, which keeps the original block size. So now I have an ESXi 5.1 system with a VMFS-5 primary datastore that has a 2 MB block size instead of 1MB. I created a Virtual Machine in that datastore with a 500 GB virtual disk, and cloned the host over to another identical host machine using PartClone. The result was that this now worked. This was failing on me when tried to clone a system that had a 500 GB virtual disk before on VMFS-5 when I had a 1MB VMFS-5 block size.
I then did the same test, but this time I created a 520 GB virtual disk in my virtual machine(just over the old VMFS-3 512 GB virtual disk limit for a 2MB block size). When I clone the ESXi host over to another identical host machine using partclone, this time the virtual disk shows up as empty.
So it appears that Partclone is adhering to the old block-size to max-file-size limitations from VMFS-2 and VMFS-3, even though they aren't applicable any longer to VMFS-5.
I was looking through vmfs5clone.c and noticed at the top there is a constant called "VMFS_BLK_MAP_MAX_INODES" with a value of 32. Is this assuming a 32-bit max Inode size(2^32)? Shouldn't this be a 64-bit size (2^64) now on VMFS-5, assuming that is what this constant is for?
I wonder if that's the problem and it's causing the clone creation to skip some of the INodes during the creation process for large virtual disks?
The problem is definitely in the clone creation process because my VMFS img.gz.aa file is very small (less than 65 MB) in my failed cloned images where I had large virtual disks that were over the original VMFS-3 size threshold(greater than 256 GB). When the clone was successful (on images with 256 GB or less sized virtual disks), the img.gz.aa file for that VMFS partition is very large (several GB, as it should be).
Last edit: ArtieMan 2013-12-20
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
OK, I built a version that "VMFS_BLK_MAP_MAX_INODES" with a value of 64 here: http://stevenshiau.org/misc/partclone/0.2.66.drbl2/
Could you please boot clonezilla live, download the deb, then install it by "dpkg -i *.deb".
Do a test again there?
Please let us know the results.
Thanks.
Steven.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'm probably going to be away from my lab until Monday so I won't be able to test it until then, unless I find some free time to swing by the lab this weekend and test it.
Hopefully I can figure out how to install the package into my running Clonezilla Live environment. I assume it involves basically just copying the right .deb file to a USB memory stick, booting the host off of the live disk, going to the command line and mounting the USB stick, then running dpkg -i on the .deb file in the mounted USB stick? Can I get back into the Clonezilla GUI after loading this so that I can use the GUI to perform my clone by running some script from the command line? If so... how do I do that?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
With internet connection:
(1) Boot Clonezilla live 2.2.0-31-i686-pae
(2) Enter command line prompt
(3) sudo -i
(4) ocs-live-netcfg
(5) wget http://stevenshiau.org/misc/partclone/0.2.66.drbl2/partclone_0.2.66.drbl2_i386.deb
(6) dpkg -i partclone_0.2.66.drbl2_i386.deb
(7) exit (or run command "clonezilla")
Then you should back to the main menu of Clonezilla live.
Without internet connection:
(1) Download http://stevenshiau.org/misc/partclone/0.2.66.drbl2/partclone_0.2.66.drbl2_i386.deb
(2) copy it to your Clonezilla live USB stick, say in /
(3) Boot Clonezilla live USB stick
(4) Enter command line prompt
(5) sudo -i
(6) cd /lib/live/mount/medium
(7) dpkg -i partclone_0.2.66.drbl2_i386.deb
(8) exit (or run command "clonezilla")
Then you should back to the main menu of Clonezilla live.
In the above I am assuming you use i686-pae version, if you use amd64 version, remember to download and use amd64 version of partclone (http://stevenshiau.org/misc/partclone/0.2.66.drbl2/partclone_0.2.66.drbl2_amd64.deb)
Thanks for testing there after weekend. Please keep us posted.
Steven.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks for the detailed explanation. I just ran through a successful dry-run on my home computer using Clonezilla Live 1.2.12-60 to make sure the process of adding in the updated PartClone works and there weren't any Partclone compatibility issues with that version of the OS/tools(using an older version makes it easy to verify that the Partclone package installation worked since there was a much older version in that Clonezilla). This wasn't cloning ESX though so it didn't actually use the modified Partclone file... I just wanted to make sure that I understood the injection process and Partclone still works in general when merged with an older Clonezilla Live version.
I'm going to try to test it tomorrow instead of Monday on a real ESXi 5.x host. Will let you know how it goes. If it works, I'll do some more testing to make sure the previously working VMFS-5 functionality was not compromised by the update.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I might setup my own Debian development environment next week so I can try some code changes on my own. This issue has to be relative to Block Sizes of the Datastore.
Last edit: ArtieMan 2013-12-21
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks. Please keep us posted, and as I mentioned, if you find something new, it would be post on the upstream vmfs-tools bug tracer which Thomas has reported one: https://github.com/glandium/vmfs-tools/issues/12
Thanks.
Steven.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
It seems that VMFS-5 Cloning does not work at all with Live-Disk versions newer than 1.2.0-67(the latest available 1.2.x version I could find on Sourceforge file repository). The release of 2.x seems to have broken even the previously functional VMFS-5 cloning capabilities that worked in 1.2.0-67 and prior. I didn't try it on a VMFS-3 datastore so I can't comment on that, but it actually crashes the Live-environment when capturing the partition containing the datastore if you try to create a clone of a VMFS-5 ESXi host using the new 2.x Clonezilla Live disks(I tried with both the x486 and AMD64 versions of the main Debian release versions).
I've been looking through the VMFS-Tools version 2.5 code and there's quite a bit going on in there. I did a bunch of comparative side-by-side debugging using a 1.2.0-67 Live Disk between two systems (both with VMFS-5 datastores, but one with a 256GB virtual disk and the other with a 257GB virtual disk) and using the various debugvmfs commands. I found the problem and it's in \libvmfs\vmfs_file.c in the vmfs_file_pread function of VMFS-Tools version 2.5. There is no defined case option to handle POINTER_BLOCK (3) types of blocks(the Enum for this is defined in \vmfs_block.h). We have to figure out how to write a function that will reside in vmfs_block.c that would be named "vmfs_block_read_pb" and call it from the case statement in vmfs_file.c for Pointer_Block types when they are encountered(0x3). The same needs to be done for the vmfs_file_pwrite function to call a new function named "vmfs_block_write_pb". That's going to take some serious evaluation and debugging to figure out what's going on at a low level with VMFS to get it to work with these Pointer Blocks. This apparently happens because VMFS-5 uses a double Pointer-Block format when the file size grows above 256 times the file block size... and now it's like a double-linked list where Pointer-Blocks point to other Pointer-Blocks when VMDK size > 256GB rather than just pointing directly to the data (which allows them to overcome the 256GB file size limit with 1MB Block sizes).
I don't have a functioning C-code development/debugging setup to trace through or make code changes at the moment for vmfs-tools. I'd like to try to figure this out next week if time allows but I'm not sure if it'll happen immediately soon. I figure this information should be very helpful if anyone else is more familiar with the inner workings of the vmfs-tools code or how VMFS-5 handles these > 256GB VMDK files.
Steven,
Do you have the ability to compile VMFS-Tools in your environment? I might have a code change that would be immediately helpful for debugging purposes if it would be easy to build.
Last edit: ArtieMan 2014-01-03
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks for keeping us posted.
"Do you have the ability to compile VMFS-Tools in your environment? I might have a code change that you could try which might fix things if it would be easy to build." -> Yes. I have. So you will send me the patches? Or?
Steven.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
You wouldn't happen to have any guidance on how to build a .deb package for VMFS-Tools would you?
I installed Debian today on a development machine, installed all of the necessary dependency packages to build VMFS-Tools, made some minor changes to the code (basic printf statement changes so I could verify my modified code is actually being generated, plus added in the missing CASE statement for PB_Block types), and compiled/built it.
My code appears to have compiled fine and I can run it on the development machine and see my changes, but I'm not quite sure of how to build it into a .deb package in order to efficiently migrate it over to a Clonezilla-Live test environment and try it out.
I did try copying the newly compiled VMFS-Tools files over to a USB-Memory stick and manually replacing those files in my Clonezilla-Live environment with their modified counterparts, but this is going to get painful real quick if I have to keep doing it for every little code change I make. I suppose I could write a manual copy/replace script to just copy everything over, but I'd rather try to do it the right way with a .deb package if possible.
Last edit: ArtieMan 2014-01-07
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi,
I'm cloning a VMFS 5 partition which contains 2 VMDK files attached to a Linux VM. The fisrt VMDK file is a system disk of 16GB, the second is a 500GB data disk. I used the following command to create an image of the partiton to another internal disk :
partclone.vmfs5 -L /home/partimage/log.txt -c -s /dev/sda3 -C --output myimage.img
Everything looks OK after cloning, no error message. But when I restored the image, the large virtual data disk (500GB) is not available any more : neither partition nor data can be found on the disk. I checked the image file genereted by PartClone, it is only 18GB ! Can you help me to sort it out. Here is the output :
Thanks
user@debian:/home/partimag$ sudo partclone.vmfs5 -L /home/partimag/log.txt -c -s /dev/sda3 -C --output myimage.img
Partclone v0.2.66 http://partclone.org
Starting to clone device (/dev/sda3) to image (myimage.img)
Reading Super Block
Elapsed: 00:00:08, Remaining: 00:00:00, Completed: 100.00%
Total Time: 00:00:08, 100.00% completed!
done!
File system: VMFS
Device size: 579.3 GB = 552448 Blocks
Space in use: 556.0 GB = 530214 Blocks
Free Space: 23.3 GB = 22234 Blocks
Block size: 1048576 Byte
Elapsed: 00:04:29, Remaining: 00:00:00, Completed: 100.00%, Rate: 4.16GB/min,
current block: 167160, total block: 552448, Complete: 100.00%
Total Time: 00:04:29, Ave. Rate: 4.2GB/min, 100.00% completed!
Syncing... OK!
Partclone successfully cloned the device (/dev/sda3) to the image (myimage.img)
Cloned successfully.
user@debian:/home/partimag$ cat log.txt
Partclone v0.2.66 http://partclone.org
Starting to clone device (/dev/sda3) to image (myimage.img)
Reading Super Block
we need memory: 1121796 bytes
image head 4160, bitmap 69056, crc 1048580 bytes
Calculating bitmap... Please wait... vmfs5clone.c: Block 0x69442023 is used but not allocated.
vmfs5clone.c: Block 0x6d783f3c is used but not allocated.
vmfs5clone.c: Block 0x69442023 is used but not allocated.
vmfs5clone.c: Used:0, Free:0, Status err:0
done!
File system: VMFS
Device size: 579.3 GB = 552448 Blocks
Space in use: 556.0 GB = 530214 Blocks
Free Space: 23.3 GB = 22234 Blocks
Block size: 1048576 Byte
Total block 552448
Syncing... OK!
Partclone successfully cloned the device (/dev/sda3) to the image (myimage.img)
Thanks for the bug report. We will try to reproduce the issue and if it's reproducible, then it could be fixed.
Steven.
Thank you for your attention, I noticed the issue often occurs with the huge VMDK files.
Thanks for the info. How big is the size when you mentioned "huge"?
Steven.
In my ESXi 5.1 environment, I created a virtual disk of 500G attached to a Linux virtual machine. It's configured as a second disk of the VM, formatted as ext4 partition with some data copied in it. While cloned and restored, I couldn't read the disk, there was no partition found in this disk.
hi,
just want to make sure the version is esxi5.1 or esxi5.1update1?
and the machine is created from VMware vSphere Client?
and the space is correct for you?
Device size: 579.3 GB = 552448 Blocks
Space in use: 556.0 GB = 530214 Blocks
I am downloading esxi 5.1 and 5.1update1 to give a test.
Thanks.
Hi,
I use the esxi5.1 (no upadete1). The space is correct as mentioned, it's in fact a RAID 0 LUN with four 146GB physical drives.
Thanks
Last edit: ztwy 2013-10-03
Hi,
Any news about the issue ?
Thanks
Last edit: ztwy 2013-10-16
Not really.
We were able to reproduce the issue. However, it seems the issue exists only when the vmdk file is very large. Then it becomes very difficult to debug and reproduce the problem.
Is your vmdk file is very large? Could you please let us know the size?
Thanks.
Steven.
That's exactly what I said. My VMDK file is about 500Go, however in thin provisioning. It means only about 270 Go of real data in the file.
Thanks
OK, thanks.
Thomas is working on this issue at least for 2 weeks...
Anyway, I believe he will fix that.
Steven.
Thank you very much, the problem is very annoying for some of our clonning projects. Fingers crossed to be fixed.
I'm also encountering this exact same problem. It only does it on VMFS-5 datastores for me. If I create the same exact ESXi 5.0 or 5.1 configuration but with a VMFS-3 primary datastore, the whole cloning process works fine and my virtual machines load up perfectly regardless of how large the virtual disk is.
I'm primarily seeing this problem on a 600 GB thin-provisioned virtual disk running on ESXi 5.1 with a Linux-based operating system in the Virtual Machine. I also created an additional 600 GB virtual machine running Windows 2003 on the same host and cloned the entire host using Clonezilla/Partclone to test if the problem had anything to do with the guest operating system type. Upon deploying this ESX image to a different host machine (identical hardware as the original), neither the Linux nor the Windows Virtual machines appeared to have any data on their virtual disks when I attempted to boot them up. Smaller virtual disks seem to clone OK though, at least for Windows-based virtual machines(I didn't really try it with any Linux-based ones in a smaller virtual disk).
Please let me know if there is any other info that I could contribute or other test scenarios that I could try and reply back with some feedback in order to help get to the root of the problem and troubleshoot it further.
I did a bunch of testing today using Clonezilla Live and Partclone to create images of an ESXi 5.1 host with virtual machine data stored in the default VMFS-5 data store (locally on the host machine's disk) and a single Virtual Machine in it. I iteratively made different sized virtual disks for my virtual machine, put a Windows 2003 image onto the virtual machine, and then shut down the host and cloned the entire host to another identical version of the host hardware using PartClone.
What I found out was this:
-Everything works fine with cloning and restoring our image if the guest VM is configured with a virtual disk that is 256 GB or smaller in size. If you make it 257 GB, or anything greater than that, it will appear to be blank upon restoring the image of the entire host to a different (but identical hardware-wise) machine.
-The Cloning problem happens regardless of whether we are thin-provisioning our guest VM's hard drive or thick-provisioning it (so provisioning has nothing to do with the problem).
Basically, virtual disks larger than 256 GB in VMFS-5 appear to be the culprit when used in conjunction with PartClone. I'm not sure what VMWare is doing in VMFS-5 over 256 GB but it does not appear that any currently available versions of PartClone are compatible with it.
Hope this helps, and please let me know if there is anything else I can contribute!
Thanks for your troubleshooting.
This issue has been reported to our upstream, vmfs-toos: https://github.com/glandium/vmfs-tools/issues/12
We are still wait for the upstream to fix this. Maybe you can also try what Thomas has found, and give some more info on the upstream's forum.
Appreciate!
Steven.
The issue appears to be related to Block Size. VMFS-2 and VMFS-3 used to have specific VMDK file-size limitations based on the block size of your datastore:
Block Size 1 MB = 256 GB Max Virtual Disk size
Block Size 2 MB = 512 GB Max Virtual Disk size
Block Size 4 MB = 1 TB Max Virtual Disk size
Block Size 8 MB = 2 TB Max Virtual Disk size (minus 512 B)
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003565
VMFS-5 automatically uses a 1 MB block size but can support virtual disk sizes up to 2 TB.
I did a test where I installed ESXi 4.1 on a host, and created a 2 MB Block sized primary Datastore on the local ESXi host hard drive. I then installed ESXi 5.1 over that and upgraded the datastore to VMFS-5, which keeps the original block size. So now I have an ESXi 5.1 system with a VMFS-5 primary datastore that has a 2 MB block size instead of 1MB. I created a Virtual Machine in that datastore with a 500 GB virtual disk, and cloned the host over to another identical host machine using PartClone. The result was that this now worked. This was failing on me when tried to clone a system that had a 500 GB virtual disk before on VMFS-5 when I had a 1MB VMFS-5 block size.
I then did the same test, but this time I created a 520 GB virtual disk in my virtual machine(just over the old VMFS-3 512 GB virtual disk limit for a 2MB block size). When I clone the ESXi host over to another identical host machine using partclone, this time the virtual disk shows up as empty.
So it appears that Partclone is adhering to the old block-size to max-file-size limitations from VMFS-2 and VMFS-3, even though they aren't applicable any longer to VMFS-5.
I was looking through vmfs5clone.c and noticed at the top there is a constant called "VMFS_BLK_MAP_MAX_INODES" with a value of 32. Is this assuming a 32-bit max Inode size(2^32)? Shouldn't this be a 64-bit size (2^64) now on VMFS-5, assuming that is what this constant is for?
I wonder if that's the problem and it's causing the clone creation to skip some of the INodes during the creation process for large virtual disks?
The problem is definitely in the clone creation process because my VMFS img.gz.aa file is very small (less than 65 MB) in my failed cloned images where I had large virtual disks that were over the original VMFS-3 size threshold(greater than 256 GB). When the clone was successful (on images with 256 GB or less sized virtual disks), the img.gz.aa file for that VMFS partition is very large (several GB, as it should be).
Last edit: ArtieMan 2013-12-20
OK, I built a version that "VMFS_BLK_MAP_MAX_INODES" with a value of 64 here:
http://stevenshiau.org/misc/partclone/0.2.66.drbl2/
Could you please boot clonezilla live, download the deb, then install it by "dpkg -i *.deb".
Do a test again there?
Please let us know the results.
Thanks.
Steven.
Sweet! You guys are quick!
I'm probably going to be away from my lab until Monday so I won't be able to test it until then, unless I find some free time to swing by the lab this weekend and test it.
Hopefully I can figure out how to install the package into my running Clonezilla Live environment. I assume it involves basically just copying the right .deb file to a USB memory stick, booting the host off of the live disk, going to the command line and mounting the USB stick, then running dpkg -i on the .deb file in the mounted USB stick? Can I get back into the Clonezilla GUI after loading this so that I can use the GUI to perform my clone by running some script from the command line? If so... how do I do that?
Thanks. Two options,
With internet connection:
(1) Boot Clonezilla live 2.2.0-31-i686-pae
(2) Enter command line prompt
(3) sudo -i
(4) ocs-live-netcfg
(5) wget http://stevenshiau.org/misc/partclone/0.2.66.drbl2/partclone_0.2.66.drbl2_i386.deb
(6) dpkg -i partclone_0.2.66.drbl2_i386.deb
(7) exit (or run command "clonezilla")
Then you should back to the main menu of Clonezilla live.
Without internet connection:
(1) Download http://stevenshiau.org/misc/partclone/0.2.66.drbl2/partclone_0.2.66.drbl2_i386.deb
(2) copy it to your Clonezilla live USB stick, say in /
(3) Boot Clonezilla live USB stick
(4) Enter command line prompt
(5) sudo -i
(6) cd /lib/live/mount/medium
(7) dpkg -i partclone_0.2.66.drbl2_i386.deb
(8) exit (or run command "clonezilla")
Then you should back to the main menu of Clonezilla live.
In the above I am assuming you use i686-pae version, if you use amd64 version, remember to download and use amd64 version of partclone (http://stevenshiau.org/misc/partclone/0.2.66.drbl2/partclone_0.2.66.drbl2_amd64.deb)
Thanks for testing there after weekend. Please keep us posted.
Steven.
Thanks for the detailed explanation. I just ran through a successful dry-run on my home computer using Clonezilla Live 1.2.12-60 to make sure the process of adding in the updated PartClone works and there weren't any Partclone compatibility issues with that version of the OS/tools(using an older version makes it easy to verify that the Partclone package installation worked since there was a much older version in that Clonezilla). This wasn't cloning ESX though so it didn't actually use the modified Partclone file... I just wanted to make sure that I understood the injection process and Partclone still works in general when merged with an older Clonezilla Live version.
I'm going to try to test it tomorrow instead of Monday on a real ESXi 5.x host. Will let you know how it goes. If it works, I'll do some more testing to make sure the previously working VMFS-5 functionality was not compromised by the update.
Tested it out today.... that didn't fix it. :(
I might setup my own Debian development environment next week so I can try some code changes on my own. This issue has to be relative to Block Sizes of the Datastore.
Last edit: ArtieMan 2013-12-21
Thanks. Please keep us posted, and as I mentioned, if you find something new, it would be post on the upstream vmfs-tools bug tracer which Thomas has reported one:
https://github.com/glandium/vmfs-tools/issues/12
Thanks.
Steven.
It seems that VMFS-5 Cloning does not work at all with Live-Disk versions newer than 1.2.0-67(the latest available 1.2.x version I could find on Sourceforge file repository). The release of 2.x seems to have broken even the previously functional VMFS-5 cloning capabilities that worked in 1.2.0-67 and prior. I didn't try it on a VMFS-3 datastore so I can't comment on that, but it actually crashes the Live-environment when capturing the partition containing the datastore if you try to create a clone of a VMFS-5 ESXi host using the new 2.x Clonezilla Live disks(I tried with both the x486 and AMD64 versions of the main Debian release versions).
I've been looking through the VMFS-Tools version 2.5 code and there's quite a bit going on in there. I did a bunch of comparative side-by-side debugging using a 1.2.0-67 Live Disk between two systems (both with VMFS-5 datastores, but one with a 256GB virtual disk and the other with a 257GB virtual disk) and using the various debugvmfs commands. I found the problem and it's in \libvmfs\vmfs_file.c in the vmfs_file_pread function of VMFS-Tools version 2.5. There is no defined case option to handle POINTER_BLOCK (3) types of blocks(the Enum for this is defined in \vmfs_block.h). We have to figure out how to write a function that will reside in vmfs_block.c that would be named "vmfs_block_read_pb" and call it from the case statement in vmfs_file.c for Pointer_Block types when they are encountered(0x3). The same needs to be done for the vmfs_file_pwrite function to call a new function named "vmfs_block_write_pb". That's going to take some serious evaluation and debugging to figure out what's going on at a low level with VMFS to get it to work with these Pointer Blocks. This apparently happens because VMFS-5 uses a double Pointer-Block format when the file size grows above 256 times the file block size... and now it's like a double-linked list where Pointer-Blocks point to other Pointer-Blocks when VMDK size > 256GB rather than just pointing directly to the data (which allows them to overcome the 256GB file size limit with 1MB Block sizes).
I don't have a functioning C-code development/debugging setup to trace through or make code changes at the moment for vmfs-tools. I'd like to try to figure this out next week if time allows but I'm not sure if it'll happen immediately soon. I figure this information should be very helpful if anyone else is more familiar with the inner workings of the vmfs-tools code or how VMFS-5 handles these > 256GB VMDK files.
Steven,
Do you have the ability to compile VMFS-Tools in your environment? I might have a code change that would be immediately helpful for debugging purposes if it would be easy to build.
Last edit: ArtieMan 2014-01-03
Thanks for keeping us posted.
"Do you have the ability to compile VMFS-Tools in your environment? I might have a code change that you could try which might fix things if it would be easy to build." -> Yes. I have. So you will send me the patches? Or?
Steven.
You wouldn't happen to have any guidance on how to build a .deb package for VMFS-Tools would you?
I installed Debian today on a development machine, installed all of the necessary dependency packages to build VMFS-Tools, made some minor changes to the code (basic printf statement changes so I could verify my modified code is actually being generated, plus added in the missing CASE statement for PB_Block types), and compiled/built it.
My code appears to have compiled fine and I can run it on the development machine and see my changes, but I'm not quite sure of how to build it into a .deb package in order to efficiently migrate it over to a Clonezilla-Live test environment and try it out.
I did try copying the newly compiled VMFS-Tools files over to a USB-Memory stick and manually replacing those files in my Clonezilla-Live environment with their modified counterparts, but this is going to get painful real quick if I have to keep doing it for every little code change I make. I suppose I could write a manual copy/replace script to just copy everything over, but I'd rather try to do it the right way with a .deb package if possible.
Last edit: ArtieMan 2014-01-07