[fuse-devel] trouble with SAMBA + FUSE for files of unknown size

[Casey B. <cbo...@ba...>]

I'll describe the setup, then the issue.

We are working on building a dynamic file system that will generate reports on an as-needed basis.  To do this, we have:
a) A java service that accesses a database to see what reports are available, generates reports when asked, and keeps a cache of reports.  The first time a report is requested, it is created and then cached.
b) A FUSE file system that talks to the java service.  Data for files is streamed back to it and on to the user.

This part works fine.  For reports that have not been cached, we are using a filesize of 0 to indicate unknown size.  In FUSE, we can set a flag for direct I/O that signals us to just get all the bytes, don't worry about the size, and this works OK.  For example, I can have a file, test.txt that has a size 0 (it hasn't been created yet, and when I `cat test.txt`, it gets generated and I get the contents.

c) When we add SAMBA to the mix, the filesize is problematic.  For files we know the size of (those that have been cached), it works fine.  However, for files of unknown size, SAMBA doesn't seem to have a direct option, and it just returns a file with 0 bytes.

Idea 1:  In following the FUSE logs, for a 0-sized file, SAMBA is attempting to open the file, but never reads it.  It just flushes and releases it.  Is there a setting for direct I/O in SAMBA to try simply get all the bytes (it would be like reading a pipe)?

Idea 2:  I tried lying about the size and setting it to 1GB when the file size is unknown.  This works, sort of, but SAMBA keeps making requests to get 1GB of bytes - even though it is also asking for file attributes before each read.  It takes way to long, and the service is returning the file for every request.  This is not ideal.  Any ideas on how to shortcut SAMBA after the first read, so it knows the filesize has been updated?

Idea 3:  Does anyone have any?  I'm open to suggestions.  I can send a log - what would be a useful log level?

I've asked the samba group, but I haven't found answers.  I didn't find a lot of FUSE users, either, so I'm hoping that maybe someone here can help.

Thanks.
-Casey



This message is intended for the addressee only and may contain privileged or confidential information. Unless you are the intended recipient, you may not use, copy or disclose to anyone any information contained in this message. If you have received this message in error, please notify the author by replying to this message and then kindly delete the message. Thank you.