directory read speed

Help
Antanas
2006-11-28
2013-04-16
  • Antanas
    Antanas
    2006-11-28

    Hello,

    While using davfs2 0.2.8 i noticed that is is much faster in reading (even empty) directories. As i understand in davfs2 1.1.x there is some kind of caching implemented. Could it be the cause for slow down? Speed when reading very large amount of small files is also very bad. It took me abount 20min to open 6000 files in directory for 0.2.8, could it be faster on 1.1.x ?

    Antanas

     
    • Werner Baumann
      Werner Baumann
      2006-11-28

      Hello Antanas,

      caching is mainly ment to avoid frequent up- and download of *files* (like Webbrowsers do). Operations that change the structure of directories will usually result in a direct call to the server, even with davfs2 1.x.x.

      The problem with directories that contain a large amount of small files usually arises when you use graphical user interfaces. Many of this will *open* just any file (to decide what user friendly icon to show). But even with caching, when a file is opened the client will have to look up the server to see whether there is a newer version (HTTP: GET If-Modified). With small files this is almost the same load as just downloading the file.

      There are some configuration options in davfs2 1.1.3 to improve the situation:
      - gui_optimize: this will try to avoid lookups for every file, but use PROPFIND to get the modification information for all files in one directory with one request.
      - file_refresh: when you increase this time, davfs2 will do less lookups, but trust on the cached information. This may be dangerous, when different people access the server concurrently, but if only one user is connected at one time, you may increase it significantly.
      - use_locks: setting this to 0 will avoid some roundtrips when opening files for writing. But you should not use it in case of conurrent acces to the server.
      - table_size: you might increase this value to about twice the number of files and directories. This will speed up search for files and directories in the cache (but this operations are done in working memory and are much faster than HTTP-requests anyway).

      Question:
      > While using davfs2 0.2.8 i noticed that is is much faster in reading (even empty) directories.
      Did you really notice operations where version 0.2.8 is faster than version 1.x.x?
      If yes, please report in more detail.

      Greetings
      Werner