Wiki for documentation

  • Hi there,

    First of all, Mark, thank you for releasing the code for this project.  I was very excited to finally see a deduplicating filesystem available for Linux.  With a bit of fun pulling in the latest versions of tokyocabinet & fuse, I've managed to get lessfs working and performance is reasonably good for my dev server (10MB/s write, ~17 MB/s re-write after blocks are already in the pool).

    I've put together a shell script to build & install tokyocabinet, fuse & lessfs into /opt, as at least on my server (Centos 5.2), RPMs are not available for the latest code.  Ideally, RPMs will be used, but in the meantime, it would be good to share the script so other's can rapidly build & begin testing lessfs.

    A wiki would be good to begin collating instructional information, as well as a place for me to upload my script :)

    Is something like that in the works?  Can a wiki codebase be deployed on sourceforge's hosted area, or does it need to be hosted externally?


    Chris Bennett (cgb)

    • John M
      John M


      I'm with you, but honestly I think this might be something you or I could start on, since Mark is busy coding the new features. I'm not sure where the wiki would be stored though

      I agree the docs are a bit sketchy but can be pulled together after looking at the tar for a few hours :p

      I've built it on Debian 5.0 with the 2.6.26 kernel with ease.


    • Cool no worries, I might see if I can string together some time & hosting somewhere to get the ball rolling.  I just didn't want to be duplicating effort if a wiki or some other form of documentation was just around the corner.

      Thanks for the quick reply :)


    • Andreas Lund
      Andreas Lund

      Are you getting these figures on a 32bit system or are you running 64bit? If 32bit, did you roll your own kernel or did you find an rpm somewhere?

    • Ahhh I should clarify what my setup was - It's a bit misleading to say it's vanilla Centos 5.2 :)

      I used OpenVZ - and have recently upgraded to their devel kernel 2.6.27 with OpenVZ patches, so I could run ZFS-backed VZ containers for COW cloning.

      So for the purposes of benchmarking, my setup is:
      - kernel-2.6.27-briullov.1.i686.rpm from OpenVZ yum repository
      - tokyocabinet-1.4.25 built from source
      - fuse-2.8.0-pre2 built from source
      - lessfs-0.2.1 built from source

      Sorry for the confusion :)

      Chris Bennett (cgb)

      • Andreas Lund
        Andreas Lund

        For some bizarre reason I can't boot 2.6.27-briullov.1, also tried rolling my own 2.6.27 kernel based on the RHEL5 .config and it fails exactly the same way; mounting the rootfs fails and the kernel panics.

        This test machine uses a plain PATA boot drive, I can't see any reason why it should fail. I did mkinitrd manually but still no go, did you have to jump through any extra hoops to make it work?

        • Hi Andreas,

          Yes, there is.  I'm not sure which kernel it came in for, but IDE disks are now detected as sdX, I think as part of kernel changing to libata for SATA & IDE disks.

          If your /etc/fstab uses ext labels for the source device, that part should be fine.  Otherwise, update hda entries for your root filesystem and relevant partitions to sda.

          The fault is with your initrd.  When mkinitrd runs under the old kernel, it detects your device as hda and installs this line in the 'init' file of your initrd:

          mkrootdev -t ext3 -o defaults,ro hdaX

          But when you boot 2.6.27, it doesn't know about a hda and you're pivot root fails :(


          cp /boot/initrd-2.6.27-briullov.1.img /boot/initrd-2.6.27-briullov.1.img.bak
          mkdir -p /tmp/init
          cd /tmp/init
          cat /boot/initrd-2.6.27-briullov.1.img | gunzip -dc | cpio -i
          13494 blocks

          vi init
          # update hda to sda

          find ./ | cpio -H newc -o > /boot/initrd-2.6.27-briullov.1.img.2

          and you should be set!  At least, it worked for me :)

          Chris Bennett (cgb)

          • Oops:
            find ./ | cpio -H newc -o > /boot/initrd-2.6.27-briullov.1.img.2
            should be
            find ./ | cpio -H newc -o > /boot/initrd-2.6.27-briullov.1.img

            If you have any further troubles, don't hesitate to post them and I'll see if I can help.

            Chris Bennett (cgb)

            • Andreas Lund
              Andreas Lund

              My fstab uses LABEL=/ for root but I'll have another stab at this later today and see if I can work it out. Thanks :-)

            • Andreas Lund
              Andreas Lund

              I'm quite sure you have the reason nailed, because the error message is "mount: could not find filesystem '/dev/root', but I have now tried
              mkrootdev -t ext3 -o defaults,ro sda1
              mkrootdev -t ext3 -o defaults,ro sdb1
              mkrootdev -t ext3 -o defaults,ro sdc1
              mkrootdev -t ext3 -o defaults,ro sdd1
              and finally
              mkrootdev -t ext3 -o defaults,ro LABEL=/
              instead of the original
              mkrootdev -t ext3 -o defaults,ro hda1
              but it still doesn't work.

              The reason I tried several devices is that I have two PATA hard drives plus a SCSI CD-ROM and an USB drive connected and I don't know in which order they register.

              Unfortunately, it still doesn't work so I'll have to look into mkrootdev some more.

          • Oh, there's more.. I should have just pasted you the diff! :)

            -mkrootdev -t ext3 -o defaults,noatime,mand,ro hda2
            +mkrootdev -t ext3 -o defaults,ro sda2

            I just copied the options from my old 2.6.18 init.  noatime & mand may work fine if you leave them in.  See how you go.

            Chris Bennett (cgb)

    • Andreas Lund
      Andreas Lund

      Had to replace my old root drive after I started getting SMART emails. Today I finally came around to fixing this, booted 2.6.27-briullov.1 successfully and now I'm seeing write speeds between 8 and 10 Mbytes/sec when using big_writes and 128K.

      The limiting factor right now seems to be the other PATA drive I'm copying off, atleast that's what atop is showing. Looking forward to see what happens when the file system grows to 50 gig, that's when it used to drop from 1 MB/sec to 100K/sec with 4K writes :-)

    • Mark Ruijter
      Mark Ruijter

      Usually when performance drops after reaching a certain database size this indicates that the bucketsize selected in /etc/lessfs.cfg is to low. The default 'lessfs.cfg' example included with the code only allows the database to grow up to: 2GB after which performance degrades.
      To store 50GB the bucketsize should be at least : 27262976

      2MB bucketsize equals 1000000 database entries.
      To store 50GB :
      50GB = 50*1024*1024*1024 = 53687091200 bytes
      53687091200/4096 (4k blocksize) = 13107200 entries in the databases.
      13107200/1000000 (divide by 10^6 entries)
      13 *2 (10^6 entries requires 2MB) = 26
      26 * 1024*1024 = 27262976 -> This is the bucketsize you need for 50GB with a 4k blocksize.

      I hope this helps,


      • Andreas Lund
        Andreas Lund

        Can I (safely) tune this after the file system has been created and populated?