From: SourceForge.net <no...@so...> - 2011-01-21 17:03:03
|
Bugs item #2846301, was opened at 2009-08-28 17:21 Message generated for change (Comment added) made by zibumsty You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=684730&aid=2846301&group_id=119701 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: External Functions Group: v4.0 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Zacharias Bumsty (zibumsty) Assigned to: Nobody/Anonymous (nobody) Summary: SysStemSort very,very slow .... Initial Comment: After a call to SysFileTree(filespec, 'file.', 'SFL') which results in e.g. 30.000 files a subsequent call to SysStemSort ('file.',,,,,41) has a dramatically long runtime (compared with V3.2) V4.0: 10.000 files = 1. sec, 20.000 files = 9 sec, 30.000 files = 26 sec V 3.2: 10.000 files = 0,02 sec, 20.000 files = 0,05 sec, 30.000 files = 0,08 sec <<<<<<<<<<<<<<<<<! Hardware: Intel Quadcore Q6600, 4 GB RAM, ... + Win XP-Prof + SP3 Best regards, Zach ---------------------------------------------------------------------- >Comment By: Zacharias Bumsty (zibumsty) Date: 2011-01-21 18:03 Message: Due to the upload restriction of 256 kB the already attached file 'SysFileTreeList' contains only 20.000 lines. The testcase shows the dramatic increase of the sorttime (only factor 300!), but no crashes of REXX. Therefore I attached the second part of the list with 10.000+ lines. After unzipping copy it to the 1st part, then the REXX crashes should occur. Thanks and best regards, Zach ---------------------------------------------------------------------- Comment By: Zacharias Bumsty (zibumsty) Date: 2011-01-21 17:46 Message: Did some further investigations with V4.1 - same problem. The function 'SysStemSort' shows a unstable behaviour, sometime it works as fast a V3.2, but sometimes it is incredible slow by a factor of 1000 ! or even REXX crashes. It seems this problem has to do with of the CONTENT of the stem to sort. Therefore I created the following stable testcase: Instead of calling 'SysFileTree' with unspecified output, the file >SysFileTreeList.txt< is attached which represents a redirected output from this function. This file is loaded into a stem and then sorted by 'SysStemSort' with the 'firstcol' parameter used from 1 to 50 (the filename starts at column 41). It works without problem from firstcol 1 to 31, but from 32 on either REXX crashes or the runtime jumps fro 40 msecs to 40 secs (factor 1000 !) This looks for me as the indication of a memory leakage or buffer overflow. I tested this case on 3 different systems (XP 2GB, Win7 Prof 32bit - 4GB and Win7 Prof 32bit - 2GB). All systems shows the same symptoms. Just run 'SysStemSortBug.rex' and use the file 'SysFileTreeList.txt' Best regards, Zach ---------------------------------------------------------------------- Comment By: Mark Miesfeld (miesfeld) Date: 2010-10-14 21:05 Message: Rick has made some recent changes in this area and I ran the following tests. Using a relatively slow laptop. The difference between 3.2.0 and 4.2.0 are not nearly as dramatic. Although with 50,000 files, there is still a relatively large gap. For 30,000 files it looks like about 3:1 and for 50,000 files about 6:1 I was hoping the gap would be close enough to close this, but I guess not. Although, it is not really a bug, maybe a Request for Enhancement would be better: C:\work>sysStemSortSlow.rex REXX-ooRexx_3.2.0(MT) 6.02 30 Oct 2007 Searching for files ... File count: 36301 Sorting files start ... Elapsed time for sort: 0.188000 C:\work>sysStemSortSlow.rex REXX-ooRexx_3.2.0(MT) 6.02 30 Oct 2007 Searching for files ... File count: 36301 Sorting files start ... Elapsed time for sort: 0.172000 C:\work>sysStemSortSlow.rex REXX-ooRexx_4.2.0(MT) 6.04 14 Oct 2010 Searching for files ... File count: 36301 Sorting files start ... Elapsed time for sort: 0.500000 C:\work>sysStemSortSlow.rex REXX-ooRexx_4.2.0(MT) 6.04 14 Oct 2010 Searching for files ... File count: 36301 Sorting files start ... Elapsed time for sort: 0.454000 C:\work> C:\Tools>sysStemSortSlow.rex REXX-ooRexx_4.2.0(MT) 6.04 14 Oct 2010 Searching for files ... File count: 50071 Sorting files start ... Elapsed time for sort: 1.547000 C:\Tools>sysStemSortSlow.rex REXX-ooRexx_4.2.0(MT) 6.04 14 Oct 2010 Searching for files ... File count: 50071 Sorting files start ... Elapsed time for sort: 1.579000 C:\Tools>sysStemSortSlow.rex REXX-ooRexx_3.2.0(MT) 6.02 30 Oct 2007 Searching for files ... File count: 50071 Sorting files start ... Elapsed time for sort: 0.265000 C:\Tools>sysStemSortSlow.rex REXX-ooRexx_3.2.0(MT) 6.02 30 Oct 2007 Searching for files ... File count: 50071 Sorting files start ... Elapsed time for sort: 0.250000 C:\Tools> ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=684730&aid=2846301&group_id=119701 |