Re: [ jEdit-devel ] Solution to Slow File System Browser under Windows

[Slava P. <sl...@je...>]

Thank you very much for tracking this down! I will incorporate your 
suggestions in 4.3pre2.

Joshua Gertzen wrote:
> Hi,
> 
> DISCLAIMER: I'm sorry if this gets a little long winded but theres a lot
> of detail to describe.
> 
> For the past few weeks I have been spending some weekends digging 
> intothe slow performance of  jEdit's File System Browser over SMB 
> networkshares.  I've been using jEdit for 3yrs and have just lived with 
> thisissue, but now that my company is considering standardizing on jEdit 
> asit's code editor, it has become a larger issue because few are 
> willingto accept it due to this issue.  For example browsing a network 
> folderwith 1000 files using jEdit takes close to 17seconds over a 
> 802.11bconnection, but using Windows Explorer takes less than a second. 
>  Thegood news is that after one to many cups of coffee, I believe I have 
> asolution to the problem.  Here's the details of what I found and how I 
> discovered the solution.
> 
> After checking out the source from CVS I hunted down the FileVFS 
> classand it's inner class LocalDirectoryEntry.  As a test, I then 
> hard-coded in values for all but the 'name', 'path' and 'type', thus 
> eliminating the remaining calls on the File object.  To my surprise, 
> this dropped the speed from 17 seconds to around 2 seconds.  I then 
> restored the calls for each attribute one-by-one and I found that no 
> single attribute was slow, each one contributed 3-4 seconds to the load 
> time.  So clearly there seemed to be an issue with the java.io.File 
> object and how it accessed attributes.  A little google searching turned 
> up JSR-203 (originally targeted for JDK1.5, now targeted for JDK1.6) 
> that will add support for bulk-retrieval of file attributes.  Further 
> review of the java.io.File class indicated that as suspected, it only 
> retrieves one attribute at a time from the underlying file system.  So 
> at that point it was clearthat Java was to fault and for a while I was 
> concerned that nothingcould be done.
> 
> In disbelief that such a flaw existed, I decided to give theJFileChooser 
> demo that comes with the JDK a try and see if it had thesame slowness 
> when using the detail view and accessing the same networkdirectory. Once 
> again I was surprised because it seemed thatJFileChooser listed the 
> detail view of the directory in only 1 second.Further, I tried using the 
> jEdit File Browser from my Linux desktop tobrowse the same network drive 
> and I found that it also only took 1second.  So apparently this flaw is 
> only an issue under Windows andapparently the Sun engineers did 
> something to get JFileChooser's accessto attributes under windows 
> running faster then what java.io.File provided.
> 
> So into the JFileChooser source I went and after many hours of 
> pouringover all the details and running a number of tests, it looks like 
> Sunspecifically addressed this slowness issue in there code by doing 
> thefollowing:
> 
> 1. JFileChooser gets it's file listing from the the FileSystemView 
> classin the 'javax.swing.filechooser' package, not from java.io.File. 
> Youcall FileSystemView.getFileSystemView() to get an instance of the 
> classand then you call fsv.getFiles() to get your file listing. 
> The'getFiles()' method returns an array of 'File' objects, which in 
> thecase of Windows, are actually instances of 
> the'sun.awt.shell.Win32ShellFolder2' class that subclasses 
> 'java.io.File'.Looking through the source code for 'Win32ShellFolder2' 
> reveals that theclass performs native bulk retrieval of certain 
> attributes and that itoverrides the 'isHidden()' & 'isDirectory()' 
> method of the 'File'class.  Oddly though, the class does not override 
> the 'length()','lastModified()', 'canRead()' or 'canWrite()' methods 
> which means thespeed of those methods does not change.  Why this is the 
> case isexplained in point 2.
> 
> 2. JFileChooser only accesses the attributes for the files that 
> arelisted in the visible area of the JTable.  Currently jEdit's 
> browser,lists all the files in a directory, creates a DirectoryEntry 
> instancesfor each 'File' and copies each attribute from the File to 
> theDirectoryEntry.  In contrast, JFileChooser lists all the files in 
> adirectory but does not access the attributes for a given file until 
> theTableModel.getValueAt() method is called while rendering the 
> visiblearea of the JTable.  This allows the FileBrowser to display very 
> fastsince it only access attributes for the 30 or so files visible in 
> thebrowser at any time.  However, there are two attributes that 
> JFileChooser mustaccess for each file before displaying any files.  The 
> isHidden() andisDirectory() properties are necessary for the model to 
> render anycontent, which is why those attributes are accessed in a 
> native bulk waythrough the 'Win32ShellFolder2' class.
> 
> So with this gained insight I proceeded to make the necessary changes 
> tothe jEdit 4.3 pre 2 code base that I have checked out.Making the first 
> change to jEdit was straight forward, I modified 
> theFileVFS._listDirectory() method to use FileSystemView.getFiles() 
> insteadof File.listFiles() and the result was the load time dropped from 
> 17seconds to 14 seconds.
> 
> Implementing the second change was a bit more involved.  Since I 
> neededto delay access the to a file's attributes as long as possible, I 
> had tomodify VFS.DirectoryEntry so that it provided access to it's 
> fields(name, path, symlinkPath, deletePath, type, length, hidden, 
> canRead &canWrite) through methods.  That way I could lazy load the 
> attributesfrom the File class when a given method was called.  This was 
> long andtedious since every reference to a given field had to be changed 
> to theappropriate method call.  I then had to 
> modifyVFSDirectoryEntryTable.resizeColumnsAppropriately() so that it did 
> notcalculate the width of the attribute columns based on the width of 
> thevalues.  It still calculates the width of the name column since 
> it'sfast to read the file name, but the attribute columns are hard set 
> to aspecific widths based on the maximum likely value of the attribute. 
>  Forinstance the last modified column would not have a value larger 
> than"12/12/12 12:12 PM" so I calculate the width based on that 
> staticvalue.  This solution may not be perfect since the Browser is also 
> usedfor other types of file views which may have different attributes 
> that arenot as static in their width, but I'm hoping a solution to this 
> issuecan be found.  That was pretty much it for the changes and with 
> thesecond change in place the load time dropped from 14 seconds to 1 
> second.
> 
> I'm planning on building a special 4.2 final build with these changes 
> inplace since my company needs this fix soon and 4.3 is still in 
> earlydevelopment.
> 
> Any thoughts? Is there a chance this could be changed for 4.3?
> 
> Thanks,
> 
> -Josh Gertzen
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
> Tool for open source databases. Create drag-&-drop reports. Save time
> by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
> Download a FREE copy at http://www.intelliview.com/go/osdn_nl