Project of the Month, April 2006

• Project name: Filesystem in Userspace
• Date founded/started: 2001-10-23
• URL: http://fuse.sourceforge.net/
• Project page: http://sourceforge.net/projects/fuse

Description of project

Filesystems are part of our everyday lives, yet we rarely think about how they work or the variety of tasks they help us perform. Extracting files from a ZIP archive or burning a CD are examples where we may use a “filesystem-like” interface, yet in these cases most of the actual work is done by an ordinary userspace program instead of a kernel driver.

FUSE adds the infrastructure to the OS (no fancy GUIs) to make developing such filesystems easier. The results are indistinguishable from “real” filesystems: all programs, whether graphical or command line, can use them without modification.

One of our most important goals was to allow the filesystem daemon to be run by an unprivileged user without compromising the security of the system. In other words, you don’t need to be root to create or mount a FUSE filesystem.

While the design might have a similarity to the microkernel approach, FUSE does not intend to replace the high-performance kernel-internal interface used by most network and disk filesystems. Rather, it makes way for a new class of filesystems that are hard or impossible to implement inside the kernel.

Trove info

• Development Status: 5 – Production/Stable
• Operating System: Linux, FreeBSD
• License: LGPL (libfuse), GPL (Linux kernel module, utils), BSD (FreeBSD kernel module)
• Programming language: C

Why and how did you get started?

Miklos: I started in the summer of 1995, when my brother and I decided to write a Norton Commander clone for Linux and DOS. It always bothered us that file managers were able to handle only a single level in expanding archived files. A ZIP file within a ZIP file was too much for them. So our file manager would cope with an arbitrary combination of archives, remote files, and everything else.

The project was fun, but it never became useful enough to be released. The virtual file handling part did survive however, under the name of AVFS. At first it was a completely userspace solution, using the LD_PRELOAD hack, and worked on Solaris and Linux with Libc-5 and early GLibc. Because of changes in GLibc this had to be abandoned in favor of a kernel-based solution using the CODA module.

The interface provided by CODA had its own limitations, so after some effort to fix these I decided to write a brand new kernel module suited perfectly to AVFS’s needs. Thus FUSE was born, and became successful as a separate project. Now AVFS is maintaned by Ralf Hoffmann, but I still have some plans for it.

Csaba: I ported FUSE to FreeBSD as a Google Summer of Code 2005 project. I joined after merging my FreeBSD support code into the library.

Valient: A few years ago I had the spare time and desire to start a new open source project. I’ve long been interested in userspace filesystems, but I wanted something that I’d actually use, so that the project wouldn’t remain in eternal beta. Since I was traveling internationally with my laptop, I chose to write an encrypted filesystem (Encfs). While it was originally based on another userspace filesystem, I ported it to FUSE before the initial public release. Encfs became one of the first public filesystems based on FUSE. As I learned more about the demands on a filesystem, I tried to help out with FUSE, mostly answering questions on the mailing list, so Miklos could spend more time on the code.

What is the software’s intended audience?

Basically filesystem developers. Of course, users of these filesystems also need to have the libraries and kernel module installed, but otherwise there shouldn’t be much direct interaction between the end user and FUSE.

How many people do you believe are using your software?

Hard to tell. I think there are at least 10,000 active users. The number of downloads per release varies between 5,000 and 10,000, and there are packages in Debian, Gentoo, FC4 extras, and Mandriva.

What are a couple of notable examples of how people are using your software?

Miklos: Just a random sample of the more interesting FUSE-based filesystems:

EncFS; SMB for FUSE; GMailFS; SSHFS; NTFSMount; HTTP-FUSE KNOPPIX; WikipediaFS; FlickrFS; Lkarmafs.

More information about these and others can be found in the project wiki.

Csaba: It’s amazing how flexible the tool is. For example, I just used it for file access monitoring to successfully debug my crashing X sessions. A 15-minute hack gave me the functionality of Linux “losetup” -o [offset] option on FreeBSD, where the analogue util/device doesn’t support offset shifts by itself.

What gave you an indication that your project was becoming successful?

The first security advisory. Not everybody might count this a success, but it’s an indication that the project is being taken seriously.

What has been your biggest surprise?

Miklos: When I received a bug report from Franco Broi about a hang that happened when the request counter (a 32-bit integer) changed sign after more than two billion filesystem operations performed on a single mount. Incidentally, Franco found almost all of the more subtle bugs in FUSE. The moral is that no amount of artificial stress testing can replace heavy real-world use.

Csaba: When I implemented readdir for the FreeBSD module it worked like a charm right after the first compilation.

What has been your biggest challenge?

Miklos: Getting the project included into the mainline Linux kernel. It is both a nice and frustrating experience. Nice because lots of people review the code, engage in enlightening discussions, and generally help with making things better. Frustrating, because sometimes it’s hard to get some point across to a respected developer, which holds up the process and makes everybody nervous.

Csaba: Living up to FUSE standards when I tried to get my contribution (FreeBSD compatibility) merged in.

Why do you think your project has been so well received?

Miklos: The simplicity of the library API is attractive to filesystem developers. It is possible to implement a working filesystem in just a few lines of code, and then add more functionality gradually.

Csaba: Miklos’ good design, ability, clairvoyance, supportiveness, and dedication. And on the technical side, our killer feature is the easy-to-use API.

Where do you see your project going?

Miklos: The kernel part is mostly complete, but the remaining rarely needed features are quite hard to implement. And there are still rough edges and a lot of space for improvement in the library. So no big changes are expected, but rather a slow and painful march toward perfection.

Csaba: Being the standard cross-platform userspace filesystem framework. One day it will be as common as DHCP or SSH.

What’s on your project wish list?

Miklos: Maybe the most interesting feature would be allowing hybrid objects — for example, a file viewed as a directory. Not strictly FUSE-dependent, this has been discussed many times on the Linux Kernel Mailing List.

The top feature requests are file locking and i18n support. Oh, and better documentation.

Csaba: Offer a clean way of upgrading to the latest API.

What are you most proud of?

Miklos: I’m proud to be part of this open source thing. And helping Linux achieve world domination. 😉

Valient: Miklos is too modest here. I have to give props to him for doing a great job managing the project and being persistent about pushing it for kernel inclusion.

If you could change something about the project, what would it be?

Miklos: The logo (hint, hint)

Csaba: Had it used statvfs() instead of statfs() from the start.

How do you coordinate the project?

There’s not much to coordinate. The Linux kernel module and the libraries are my territory, and the FreeBSD kernel module is done by Csaba.

Do you work on the project full-time, or do you have another job?

I have a job.

If you work on the project part-time, how much time would you say you spend, per week, on it?

Miklos: On average, 10-15 hours.

Csaba: Depends. There were weeks when I did hardly anything other than FUSE-related stuff. Now I’m a bit laid back. I don’t yet see the convergence.

What is your development environment like?

Miklos:

Machines: VIA Epia CL10000

Operating system: Emacs (running on top of Debian Linux)

For testing the kernel module I use UML, userspace code is checked with valgrind. Stress testing is done with the help of the LTP suite.

Csaba:

A standard aging PC and laptop, and some Qemu boxen. For FUSE-related work I mostly use FreeBSD 6.x and CURRENT. When I boot into Linux to compare, experiment, or debug, I use Frugalware Linux which does a good job at breaking the “Linux: it just works / good for hacking / no bloat: pick any two” law.

Editors: I hate writing code in Emacs and I hate reading code in Vim. Therefore I use both.

Milestones:

• 2003-02-19 FUSE-1.0 – First stable release
• 2004-02-04 FUSE-1.1 – Linux 2.6 support
• 2004-10-14 FUSE-1.9 – Build shared library
• 2005-10-28 Linux-2.6.14 – Kernel part of FUSE merged
• 2006-01-14 FUSE-2.5 – FreeBSD support
• ????-??-?? FUSE-3.0 – Clean out compatibility cruft from library

How can others contribute?

There’s a dire lack of good documentation, especially a programmers guide, and I’d be happy to see FUSE ported to further OSes. Any help with these would be appreciated. Contact me on the project mailing list about this or any other issue.