Work at SourceForge, help us to make it a better place! We have an immediate need for a Support Technician in our San Francisco or Denver office.

Close

#1087 ooRexx SysFileTree Performance Problem

4.1.3
closed
Mark Miesfeld
none
5
2014-06-11
2012-07-15
Jerry Senowitz
No

In an effort to come up with an alternate solution that would avoid the problems that I have been having with SysFileTree (Bug ID 340570), I installed Regina 3.6 on my Windows XP 32 bit system (SP3). I ran test4.rex from 340570 with a minor mod since there is no var() function in Regina.

The computer I am using for this test is a Dell Dimension 8300 w/ a Pentium 4 @ 2.6 GHz w/ Hyperthreading. 768 MBytes of DDR memory. Hard drives are 32bit FAT. Old iron by today's standard.

This is the test:
/ Test for item #3405740 /
parse source . . dp "." .
attrs = "*"
ename = "C:*"
sw = 0
say "Collecting SysFileTree information"
rc = SysFileTree(ename,"f","BLS",attrs)
if rc \= 0
then do
say
say "SysFileTree error:" rc
exit 1
end
say "Collection complete," f.0 "stem variables."
do i=1 to f.0
if f.i = "F".i
then do
sw = 1
say "Stem variable f."i "has not been assigned a value."
end
end
if sw
then do
say "Dumping variable pool."
call SysFileDelete dp".txt"
call SysDumpVariables dp".txt"
say "Dump complete."
end
else say "Problem #3405740 didn't occur."
exit 0

Here are the resulting consoles using time("R") and time("E"). (ooRexx verion 3.1.1 was used in this test because it is the only reliable version I have. Versions 3.1.2 and newer all experience the bug that is reported in 3405740):

[ooRexx]test4
Collecting SysFileTree information
Collection complete, 78684 stem variables.
Problem #3405740 didn't occur.
Elapsed time: 21.297 secs
[ooRexx]test4
Collecting SysFileTree information
Collection complete, 78684 stem variables.
Problem #3405740 didn't occur.
Elapsed time: 11.234 secs
[ooRexx]test4
Collecting SysFileTree information
Collection complete, 78684 stem variables.
Problem #3405740 didn't occur.
Elapsed time: 11.172 secs
[ooRexx]test4
Collecting SysFileTree information
Collection complete, 78684 stem variables.
Problem #3405740 didn't occur.
Elapsed time: 11.156 secs

[Regina]test4
Collecting SysFileTree information
Collection complete, 78684 stem variables.
Problem #3405740 didn't occur.
Elapsed time: 12.828 secs
[Regina]test4
Collecting SysFileTree information
Collection complete, 78684 stem variables.
Problem #3405740 didn't occur.
Elapsed time: 1.313 secs
[Regina]test4
Collecting SysFileTree information
Collection complete, 78684 stem variables.
Problem #3405740 didn't occur.
Elapsed time: 1.313 secs
[Regina]test4
Collecting SysFileTree information
Collection complete, 78684 stem variables.
Problem #3405740 didn't occur.
Elapsed time: 1.282 secs

The 1st run for each, Windows XP begins it's file/folder caching. the second + runs, this data, I assume is cached or the disk heads are right over the file/folder tracks.

The first run for each, Regina does the information collection in ~.6 the time that it takes ooRexx to the same job. In the 3rd run, Regina does the same job in ~.12 the time that ooRexx takes. Runs 2 and 4 offer similar peformance savings.

This is a serious performance issue dealing with SysFileTree when it looks at an entire or major subset tree structure for a hard drive. Why is ooRexx taking so long to do the same task as Regina? And, is there any correlation between this and what is happening in problem 3405740?

Discussion

  • Mark Miesfeld
    Mark Miesfeld
    2012-07-30

    Committed revision 8126 in trunk

    Testing looks good, but will do more testing before committing to the 4.1 fixes branch

    There was an obvious optimization in the Windows version which is included in the commit. I get times that are about half of what they were. How close that comes to Regina's time will have to be tested.

     
  • Mark Miesfeld
    Mark Miesfeld
    2012-08-09

    Committed revision 8170 in trunk

    Programs attached to do the calculations:

    percentFaster.rex, test4cmpOORexx.rex test4cmpRegina.rex

    percentFaster.rex calculates the percentage faster between 2 numbers. test4cmpOORexx.rex times SysFileTree on a system finding all files on the C: drive and calculates an average, using ooRexx. test4cmpRegina.rex does the exact same thing using Regina.

    Using info supplied by Jerry initially:

    Averaging the 2nd, 3rd, and 4th times for each:

    ooRexx: 11.187
    Regina: 1.303

    1.303 is 88.353% faster than 11.187

    Test run on a 64-bit Windows system using ooRexx 4.1.2 and Regina 3.6

    Average elapsed time using ooRexx: 18.6740667
    Average elapsed time using Regina Rexx: 7.2488

    7.2488 is 61.1825313% faster than 18.6740667

    Test run on the same system using the current SysFileTree implementation in trunk:

    Average elapsed time using ooRexx: 10.0756
    Average elapsed time using Regina Rexx: 10.1503333

    10.1503333 is 0.741725555% slower than 10.0756

    The output from the test runs that produced the above are also attached as a file named:
    test.results

    Based on the above, it seems the current implementation of SysFileTree in ooRexx and Regina Rexx have pretty comparable perfomance.

     
  • Mark Miesfeld
    Mark Miesfeld
    2012-08-09

    My last note should have indicated the commit was for the 4.1 fixes branch:

    Committed revision 8175. 4.1 fixes branch

     


Anonymous


Cancel   Add attachments