Re: [Exist-open] file:directory-list() super slow

[Joe W. <jo...@gm...>]

Hey Len,

There have been changes to the code involving the directory-list function
after 5.2.0 that were included in 5.3.0:


https://github.com/eXist-db/exist/commits/develop/extensions/modules/file/src/main/java/org/exist/xquery/modules/file/DirectoryList.java

I don't suppose you know how to use git bisect? You could use this tool to
find out which of the commits are associated with the performance
regression.

Joe

On Fri, Apr 29, 2022 at 8:05 PM Len Schultz <le...@wi...> wrote:

> Hello,
>
> file:directory-list(’/slowdir’,’*.*') has been super slow.  ‘/slowdir’ is
> a small directory on my workstation’s flash drive.  This directory does
> contain some symbolic links to large directories on the large external disk
> drives, so it does appear to be traversing down the directory tree.  Every
> time I run it, my large external drives are thrashing.  That explains why
> it’s slow.
>
>
> https://exist-db.org/exist/apps/fundocs/view.html?uri=http://exist-db.org/xquery/file
>  states:
>
> List all files, including their file size and modification time, found in *or
> below* a directory, $directory. Files are located in the server's file
> system, using filename patterns, $pattern. File pattern matching is based
> on code from Apache's Ant, thus following the same conventions. For
> example: '*.xml' matches any file ending with .xml *in the current
> directory*, - '**/*.xml' matches files in any directory *below the
> specified directory*. This method is only available to the DBA role.
>
>
> So my $pattern is only matching the current directory, but exist-db is
> still scanning all subdirectories.  Any ideas of how I can avoid it
> scanning subdirectories?
>
> I’m running exist-db 5.3.  I swear this was fast with 5.2, so maybe
> something changed in the code where it now always scans subdirectories?
> But my subdirectories have gotten much larger over time, so maybe that’s
> the difference.
>
> --len
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>