64-bit math support for large disks

BSzili
2014-02-26
2014-05-27
<< < 1 .. 5 6 7 8 9 .. 14 > >> (Page 7 of 14)
  • BSzili
    BSzili
    2014-04-14

    @kas1e
    That's not exactly new information, we already know my changes broke directory listing. All of the new functions I added to the library were slightly broken in some way or another, which I fixed in the previous two commits.
    Now they work perfectly for me, so unless someone knows how could I reproduce it on my machine I can't fix the new bug you found.

     
  • kas1e
    kas1e
    2014-04-14

    @BSZili
    If i see it right, you forget to update aos4_68k_to_ppc_vectors.c file by new functions. That file are necessary as well and in use. Will do so later, probably that will fix all the bugs.

     
  • BSzili
    BSzili
    2014-04-14

    @kas1e
    Isn't that the 68k->PPC library glue? It shouldn't cause any problems, unless you throw in 68k executables to the mix - or so I assumed. I decided to leave updating it to others, and only updated the library headers.

     
  • BSzili
    BSzili
    2014-04-14

    @kas1e
    OK, I think I've found the source of this bug as well. Since the FIB is now put on the stack, the return value of Examine must be checked as well. I'll fix it when I get home.

     
  • kas1e
    kas1e
    2014-04-14

    @BSzili
    Oki, good to know. I anyway update 68k_vectors file as well to be in sync with rest

     
  • xenic
    xenic
    2014-04-14

    @all
    I tested the latest commits (rev 982) in OS3 and OS4. The lister display, GetSizes and links all worked except when performing GetSizes on a directory containing links. I changed the "GETFIBSIZE(...) = GETFIBSIZE(...)" line in function_files.c to a direct 64 bit copy inside ifdefs and it works for OS3/OS4 now. If that change causes problems for AROS, BSzili will need to find a way that works on all platforms. Committed revision 983.

     
  • BSzili
    BSzili
    2014-04-14

    @xenic
    The problem is not caused by the GETFIBSIZE macro, but this mistake:

    GETFIBSIZE(&handle->anchor->ap_Info) = GETFIBSIZE(sinfo);
    

    should have been:

    GETFIBSIZE(&handle->anchor->ap_Info) = GETFIBSIZE(&sinfo->sli_Fib);
    

    For some reason I tought the sinfo variable points to a FIB. D'oh.

     
  • xenic
    xenic
    2014-04-14

    @BSzili
    I noticed that the new ExamineLock64() functions in the program are #ifdefed into the code like this:

    #ifdef USE_64BIT
    ExamineLock64(lock,(FileInfoBlock64 *)fib);
    #else
    Examine(lock,fib);
    #endif
    UnLock(lock);

    I think it would be cleaner to do those "ifdefs" in the Examine64 and Match64 functions in the library. Those functions all call the normal (old) Examine() or Match() functions before getting the 64 bit values anyway. We just need to add " -DUSE_64BIT " to the library makefile and add something like this to the 64 bit functions:

    success = Examine(lock, (struct FileInfoBlock *)fib);
    #ifndef USE_64BIT
    return success;
    #endif

    It's just a suggestion. Things will work either way.

     
  • BSzili
    BSzili
    2014-04-14

    @xenic
    The exact reason I added those ifdefs is the library. If you run the main program with an older version of the library, it will crash on the new fuctions.

     
  • xenic
    xenic
    2014-04-14

    @BSzili

    The problem is not caused by the GETFIBSIZE macro, but this mistake:

    Sorry, I should have looked more closely before changing it. You can change it back to your corrected version if you want.

     
  • BSzili
    BSzili
    2014-04-14

    @xenic
    Okay, I'll change it back with the rest of the fixes.

     
  • xenic
    xenic
    2014-04-14

    @BSzili

    The exact reason I added those ifdefs is the library. If you run the main program with an older version of the library, it will crash on the new fuctions.

    If you are worried about running the new program with old library then we will need to #ifdef the Match64 functions and the 64bit math and string functions you added earlier. It depends on how far back you want to remain compatable. I'm sure that using the current program with the library from last summer will have complications.

    However, the main.c file has this line:

    if (!(DOpusBase=OpenLibrary("dopus5:libs/dopus5.library",LIB_VER))

    which means if we bump the library version in Include/dopus/version.h then the program won't start unless it finds the correct library version.

    I think it would be better to bump the library version to avoid problems with using old libraries rather than fill the program with more #ifdefs.

     
  • BSzili
    BSzili
    2014-04-14

    @xenic
    I'm already using the library version to detect the availability of the 64-bit support. I kept the backwards compatibility with the original library if the program is compiled without USE_64BIT. If this is undesired, then we can drop the backward compatibility altogether, and require the new version of the library.

     
  • xenic
    xenic
    2014-04-14

    @BSzili

    If this is undesired, then we can drop the backward compatibility altogether, and require the new version of the library.

    Yes. That's what we should do. If we make further changes in the program in the future that won't work with older libraries we don't want another round of #ifdefs for backwark compatability with older libraries. Most programs do a library check before starting to see if a minimum required version is available. If you update to a new version of YAM without updating external MCC libs (like NList), it won't run. We should be far-sighted about future improvements to DOpus5 and do the same.

    kasle and I decided on this near the beginning of the port from SASC. We didn't want people trying to use old library/modules with our new OS3 version of the program. The modules use the same version as the library too. We tried to pick a starting version for the library that was larger than the versions in the old library or modules. In addition, we realized that the Program version can not be increased above 5 because of possible confusion with Windows versions of DOpus that might not please the original authors. Therefore, kas1e decided that the beta (test versions) of the Dopus5 program would only be differentiated by the compile date and that only the revision would be increased for stable releases.

     
  • xenic
    xenic
    2014-04-14

    @kas1e
    Since BSzili brought up the fact that our new library has new functions and that the current program will crash if used with older libraries, I'm planning on increasing the library version to avoid that kind of problem even with the overnight compiles. Is it O.K. with you to increase the library version now that we have new functions in the library??

     
  • kas1e
    kas1e
    2014-04-14

    @xenic
    of course, i am 100% ok with it

     
  • xenic
    xenic
    2014-04-14

    @BSzili
    Glad you pointed that out. It totally defeated the versioning system we established earlier in the porting process. I guess there's no way you could have known what we've done if you weren't monitoring the code at the time we added the versioning system. Any version changes should be done in Include/dopus/version.h so that we have consistant version control. Have you added or changed version variables anywhere else??

    I'm going to bump the library version to avoid any confusion for people who are D/L the nightly build and keeping old ones around.

     
  • xenic
    xenic
    2014-04-14

    @all
    I just checked and all our C commands and modules are using a different minimum version for the library. I'll need to add the library version variable everywhere and commit it all at once so it will take a little while. Once I've committed it, everyone will probably need to do a fresh compile of everything instead of just compiling changed files.

     
  • xenic
    xenic
    2014-04-14

    @all
    I changed all Dopus5 components to check for the version defined in LIB_VERSION and bumped the library version to 70. As I mentioned above a complete fresh compile will probably be needed for everything to work correctly. In the future we only need to increase the version number when changes in the library mean that new program versions may not work correctly with old libraries. Otherwise we would only increase the library revision number for new releases. Committed revision 988.

     
  • kas1e
    kas1e
    2014-04-15

    @all
    Bad news: even in our latest 989 commit, aos4 version still didn't works and bug with function_readdir.c still here. I build it fresh, full recompile , etc.

    @xenic

    I tested the latest commits (rev 982) in OS3 and OS4. The lister display, GetSizes and links all worked

    Are you sure you test os4 version too ? Maybe just os3 one ? For me os4 still and as before show up empty lister where i have "reading dir" in title, and it all happens forever until not eat all my ram. I do fully fresh rebuild of everything of course.

     
    Last edit: kas1e 2014-04-15
  • kas1e
    kas1e
    2014-04-15

    @All
    Probably some good news: that problem happens currently only with RAM disk. I.e. while all that "reading of dir and eating of memory" happens on ram, and if i close that lister, and open any others parition, it all works fine, and in debug log i have a lot of entries for files like:

    [64bit.c:282 L_ExamineNext64] Got 64-bit size for file1: 1211180777472
    [64bit.c:282 L_ExamineNext64] Got 64-bit size for file2: 1211180777472
    [64bit.c:282 L_ExamineNext64] Got 64-bit size for file3: 1211180777472
    [64bit.c:282 L_ExamineNext64] Got 64-bit size for fileX: 1211180777472
    

    But once i perform that on ram (i.e. dbl-click on ram icon), then its all start to eat memory like madman in empty lister with never ending loop called "reading directory". And, for RAM, it is always like this:

    [64bit.c:236 L_ExamineLock64] Got 64-bit size for def_ram.info: 1013612281856
    [64bit.c:236 L_ExamineLock64] Got 64-bit size for def_ram.info: 1013612281856
    [64bit.c:236 L_ExamineLock64] Got 64-bit size for def_ram.info: 1013612281856
    [64bit.c:236 L_ExamineLock64] Got 64-bit size for def_ram.info: 1013612281856
    ....
    

    I.e. for ram its bring us line 236 (and not 282 as for all others), and numbers are different (but i assume it just because of files/links).

    I have firstly a guess, that its can be because of links. Because all what ram partition have, is :

    8/0.RAM Disk:> cd ram:
    
    8/0.RAM Disk:> list
    Disk.info                      Link ----rwed Today     12:04:41
    > ENVARC:Sys/def_RAM.info
    Clipboards                      Dir ----rwed Today     12:15:00
    T                               Dir ----rwed Today     12:22:05
    2 directories - 1 unresolved softlink - 6 blocks used
    
    8/0.RAM Disk:>
    

    I.e. the file on which all that problems happens are def_ram.info which are link.

    But then, all links works ok in all other partitions.

    I then just delete disk.info link from ram: at all for tests, and still, reading dir on RAM: fail. And even more: without any debug strings from 64bit.c

    So, currently i come to some conclusion that its some other changes in function_readdir. I can try step by step revert back all changes we do in fucntion_readdir.c till working version.

    At moment trying to revert changes about newcreate_file_enrtry_fib() - still same problems with original create_file_entry().

     
    Last edit: kas1e 2014-04-15
  • kas1e
    kas1e
    2014-04-15

    @all
    So,

    revert changes in function_readdir.c about newcreate_file_enrtry_fib() - still same problems with original create_file_entry() as well.

    Building function_readdir.c without 64bit support: bug still here.

    Sign.. all i can say currently its the same non-helpfull info : changes in commit 961 in function_readdir.c produce bug in RAM:, while in 959 and 960 it still ok.

     
    Last edit: kas1e 2014-04-15
  • kas1e
    kas1e
    2014-04-15

    @all
    Some more info:

    If in function_readdir from commit 961 (which introduce problem), i just remove usage of 64bit functions, and keep original ones, bug still here (!) That explain why it was here also when i build latest commit without 64bit dos support.

    BUT ! if i also remove GETFIBSIZE macro usage in two places and put back original lines (i.e. fileinfo->fib_Size=0; and fileinfo->fib_Size), then it works.

    What mean, that even original non-64 bit code didn't works with GETFIBSIZE macro and give us that bug in RAM:

    Then for sake of tests, i put back usage of 64bit functions, and keep code without GETFIBSIZE usage, then bug appears to be here again, even without GETFIBSIZE ! But probably, what done in GETFIBSIZE somehow may be done somewhere else in 64bit functions or related to it somehow.

     
    Last edit: kas1e 2014-04-15
  • BSzili
    BSzili
    2014-04-15

    @kas1e
    If you replace the 64-bit Examine functions with the originals, but keep using the GETFIBSIZE macro, then you read garbage from the uninitialised fib_Reserved part of the FIB, and use it as the file size.

    edit: Step by step reverting won't really reveal anything other than r961 and onwards being broken. All the new functions were broken in various ways, which I fixed in much later revisions.

     
    Last edit: BSzili 2014-04-15
<< < 1 .. 5 6 7 8 9 .. 14 > >> (Page 7 of 14)