After reading a post you had written about virtualization on lessfs and cloning, I wondered whether file copy within the same FS would be near instantaneous since lessfs may be able to shortcut the I/O and just add a file entry in fileblock. Am I missing something in thinking this would be possible? Because then VM cloning would be very fast, much like COW filesystems like ZFS & BTRFS. What do you think?
Regards,
Chris Bennett (cgb)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I should point out that in testing my theory, it took the same amount time for a file copy within the filesystem, as it did copying the same file from outside, into lessfs for a second time.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The problem here is that there we need to update several references.
1. The blockusage database records how often a block is used so we can tell when it's save to delete a block of data.
When we create a new file this reference needs to be updated per block.
2. The fileblock database holds the structure of the file. It has a key that consists of inode-blocknr and a value, the tigerhash. When we create a new file/inode the fileblock entries will have to be written.
In the future the feature that you are requesting would be one my list. For now good kernel NFS support, snapshots and replication come first.
An easy to implement feature would be to create support for a file 'fastcopy' feature with the telnet interface. In this case a dedicated thread would construct the new inode from the original inode. It would save us from the overhead that comes with tiger hash calculation etc. It would speed up copying virtuals with 30~40%. But again, don't hold your breath. ;-)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Sounds good Mark, it's not a 'need it now' feature, it was just something I realised should be possible with a block-deduplicating without the overhead of a full checksum comparison. Keep up the good work! :)
Chris Bennett (cgb)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi Mark,
After reading a post you had written about virtualization on lessfs and cloning, I wondered whether file copy within the same FS would be near instantaneous since lessfs may be able to shortcut the I/O and just add a file entry in fileblock. Am I missing something in thinking this would be possible? Because then VM cloning would be very fast, much like COW filesystems like ZFS & BTRFS. What do you think?
Regards,
Chris Bennett (cgb)
I should point out that in testing my theory, it took the same amount time for a file copy within the filesystem, as it did copying the same file from outside, into lessfs for a second time.
The problem here is that there we need to update several references.
1. The blockusage database records how often a block is used so we can tell when it's save to delete a block of data.
When we create a new file this reference needs to be updated per block.
2. The fileblock database holds the structure of the file. It has a key that consists of inode-blocknr and a value, the tigerhash. When we create a new file/inode the fileblock entries will have to be written.
In the future the feature that you are requesting would be one my list. For now good kernel NFS support, snapshots and replication come first.
An easy to implement feature would be to create support for a file 'fastcopy' feature with the telnet interface. In this case a dedicated thread would construct the new inode from the original inode. It would save us from the overhead that comes with tiger hash calculation etc. It would speed up copying virtuals with 30~40%. But again, don't hold your breath. ;-)
>> In the future the feature that you are requesting would be one my list. For now good kernel NFS support, snapshots and replication come first.
Go Mark Go! Yea for NFS support!
I did find a FUSE version of NFS, but didn't know too much to install. I'm not in any hurry, i'll just wait for u
Sounds good Mark, it's not a 'need it now' feature, it was just something I realised should be possible with a block-deduplicating without the overhead of a full checksum comparison. Keep up the good work! :)
Chris Bennett (cgb)