[fuse-devel] inefficient directory listings (network file system)

[Alex C. <aco...@gm...>]

However, I find that for implementing networked file systems, the
current lowlevel API is lacking some features.

So when an readdir issue is called, you need to go through the network
and ask for the information, you have your application specific offset
and the total buffer size. However, on the receiving end the
application has now way of knowing a priori
how many entries can fit on the given buffer size, since that depends
on how fuse encodes entries.

If you do a conservative estimate then you will send less entries than
required, which will require the sender to send another RPC.
If you are optimistic you will send to much information and you will
have wasted bandwidth.

In both cases are inefficient (in the sense that they are not using
the minimum required number of bandwidth/latency). The only way to
improve this situation is to  guarantee that your estimate is always
optimistic and then cache the data on the client instead of sending
another request. This however adds unnecesssary complexity to an
otherwise simple task, furthermore anyone who wants to be efficient
will be implementing their own caching schemes on the fuse client.

In the past you could use fuse_dirent_size (now deprecated), but even
that function is not ideal (but it works) since it would require
linking the server side with fuse. Ideally we would like to use a
macro for this purpose (and AFAIK fuse_dirent_size is actually just
executing a macro), I know this also has its drawbacks since if the
macro changes applications would need to be recompiled to work
properly, but I can't think of a better solution. I hoped we could
start a discussion on how to solve this issue, and this seems the
right place to do it.

Regards,
-- 
Alex