[ jEdit-devel ] Solution to Slow File System Browser under Windows

[Joshua G. <pr...@co...>]

Hi,

DISCLAIMER: I'm sorry if this gets a little long winded but theres a lot
of detail to describe.

For the past few weeks I have been spending some weekends digging 
intothe slow performance of  jEdit's File System Browser over SMB 
networkshares.  I've been using jEdit for 3yrs and have just lived with 
thisissue, but now that my company is considering standardizing on jEdit 
asit's code editor, it has become a larger issue because few are 
willingto accept it due to this issue.  For example browsing a network 
folderwith 1000 files using jEdit takes close to 17seconds over a 
802.11bconnection, but using Windows Explorer takes less than a second. 
  Thegood news is that after one to many cups of coffee, I believe I 
have asolution to the problem.  Here's the details of what I found and 
how I discovered the solution.

After checking out the source from CVS I hunted down the FileVFS 
classand it's inner class LocalDirectoryEntry.  As a test, I then 
hard-coded in values for all but the 'name', 'path' and 'type', thus 
eliminating the remaining calls on the File object.  To my surprise, 
this dropped the speed from 17 seconds to around 2 seconds.  I then 
restored the calls for each attribute one-by-one and I found that no 
single attribute was slow, each one contributed 3-4 seconds to the load 
time.  So clearly there seemed to be an issue with the java.io.File 
object and how it accessed attributes.  A little google searching turned 
up JSR-203 (originally targeted for JDK1.5, now targeted for JDK1.6) 
that will add support for bulk-retrieval of file attributes.  Further 
review of the java.io.File class indicated that as suspected, it only 
retrieves one attribute at a time from the underlying file system.  So 
at that point it was clearthat Java was to fault and for a while I was 
concerned that nothingcould be done.

In disbelief that such a flaw existed, I decided to give theJFileChooser 
demo that comes with the JDK a try and see if it had thesame slowness 
when using the detail view and accessing the same networkdirectory. 
Once again I was surprised because it seemed thatJFileChooser listed the 
detail view of the directory in only 1 second.Further, I tried using the 
jEdit File Browser from my Linux desktop tobrowse the same network drive 
and I found that it also only took 1second.  So apparently this flaw is 
only an issue under Windows andapparently the Sun engineers did 
something to get JFileChooser's accessto attributes under windows 
running faster then what java.io.File provided.

So into the JFileChooser source I went and after many hours of 
pouringover all the details and running a number of tests, it looks like 
Sunspecifically addressed this slowness issue in there code by doing 
thefollowing:

1. JFileChooser gets it's file listing from the the FileSystemView 
classin the 'javax.swing.filechooser' package, not from java.io.File. 
Youcall FileSystemView.getFileSystemView() to get an instance of the 
classand then you call fsv.getFiles() to get your file listing. 
The'getFiles()' method returns an array of 'File' objects, which in 
thecase of Windows, are actually instances of 
the'sun.awt.shell.Win32ShellFolder2' class that subclasses 
'java.io.File'.Looking through the source code for 'Win32ShellFolder2' 
reveals that theclass performs native bulk retrieval of certain 
attributes and that itoverrides the 'isHidden()' & 'isDirectory()' 
method of the 'File'class.  Oddly though, the class does not override 
the 'length()','lastModified()', 'canRead()' or 'canWrite()' methods 
which means thespeed of those methods does not change.  Why this is the 
case isexplained in point 2.

2. JFileChooser only accesses the attributes for the files that 
arelisted in the visible area of the JTable.  Currently jEdit's 
browser,lists all the files in a directory, creates a DirectoryEntry 
instancesfor each 'File' and copies each attribute from the File to 
theDirectoryEntry.  In contrast, JFileChooser lists all the files in 
adirectory but does not access the attributes for a given file until 
theTableModel.getValueAt() method is called while rendering the 
visiblearea of the JTable.  This allows the FileBrowser to display very 
fastsince it only access attributes for the 30 or so files visible in 
thebrowser at any time.  However, there are two attributes that 
JFileChooser mustaccess for each file before displaying any files.  The 
isHidden() andisDirectory() properties are necessary for the model to 
render anycontent, which is why those attributes are accessed in a 
native bulk waythrough the 'Win32ShellFolder2' class.

So with this gained insight I proceeded to make the necessary changes 
tothe jEdit 4.3 pre 2 code base that I have checked out.Making the first 
change to jEdit was straight forward, I modified 
theFileVFS._listDirectory() method to use FileSystemView.getFiles() 
insteadof File.listFiles() and the result was the load time dropped from 
17seconds to 14 seconds.

Implementing the second change was a bit more involved.  Since I 
neededto delay access the to a file's attributes as long as possible, I 
had tomodify VFS.DirectoryEntry so that it provided access to it's 
fields(name, path, symlinkPath, deletePath, type, length, hidden, 
canRead &canWrite) through methods.  That way I could lazy load the 
attributesfrom the File class when a given method was called.  This was 
long andtedious since every reference to a given field had to be changed 
to theappropriate method call.  I then had to 
modifyVFSDirectoryEntryTable.resizeColumnsAppropriately() so that it did 
notcalculate the width of the attribute columns based on the width of 
thevalues.  It still calculates the width of the name column since 
it'sfast to read the file name, but the attribute columns are hard set 
to aspecific widths based on the maximum likely value of the attribute. 
  Forinstance the last modified column would not have a value larger 
than"12/12/12 12:12 PM" so I calculate the width based on that 
staticvalue.  This solution may not be perfect since the Browser is also 
usedfor other types of file views which may have different attributes 
that arenot as static in their width, but I'm hoping a solution to this 
issuecan be found.  That was pretty much it for the changes and with 
thesecond change in place the load time dropped from 14 seconds to 1 second.

I'm planning on building a special 4.2 final build with these changes 
inplace since my company needs this fix soon and 4.3 is still in 
earlydevelopment.

Any thoughts? Is there a chance this could be changed for 4.3?

Thanks,

-Josh Gertzen