xadopus.module: gcc port done

kas1e
2013-09-10
2014-06-01
1 2 > >> (Page 1 of 2)
  • kas1e

    kas1e - 2013-09-10

    So i port xadopus.module to 68k-gcc now, and it seems to works (at least on os4) with necessary ppc-glue stab (which need to put to dopus5:libs/ , attached). At least i can do the same as with original sasc one : dbl-click on archive (when in filetype on .lha i have command: xadopen) it open archive in new lister and i can d&d from it to another lister.

    That what i do:

    -- asm and saveds on ASM SAVEDS
    -- necessary includes and stuff where is need it
    -- a bit redone libinit.c (to open xadmaster.library, etc)
    -- that module have not only 2 usual function like Module_Entry and Module_Identify, but also Progress_Hook, so i also add that to libinit as 3st function (because in .fd it present too), as well as do necessary changes everywhere in that terms.
    -- all those LIBFUNC/LIBROTO/REG(xx) changes
    -- makefile for os3
    -- + some trivial bugs there and there

    Through there is bunch of warnings as usual (most of harmless i assume, but dunno), and 1 small place which i wasn't able to fix/understand myself.

    Lines 608-610 have by default that:

    sel=SelectionList(list,data->listw,NULL,DOpusGetString(locale,MSG_SELECT_DEST),-1,NULL,NULL,DOpusGetString(locale,MSG_OK),DOpusGetString(locale,MSG_ABORT));
    

    Which when we compile it scream about "too few arguments". It is one of those "a5" 68k functions for which we have assembler replacements to make them works on gcc normally. I.e. in the initial xadopus.c it only have 9 arguments, while it should have 11. Dunno how it right, but i just add at end 2 more NULLs, like this:

    sel=SelectionList(list,data->listw,NULL,DOpusGetString(locale,MSG_SELECT_DEST),-1,NULL,NULL,DOpusGetString(locale,MSG_OK),DOpusGetString(locale,MSG_ABORT), NULL,NULL);
    

    But even when i replace SelectionList on my own with 2 more nulls, all seems works in terms of unpacking archives.

    Also will need to do changes as BSZili do for themes.module and ftp.module : those hook.dc functions, for which BSzili do those DC_CALL macroses. For 68k version they works as it now, but for os4/mos for sure should be changed.

     
    Last edit: kas1e 2013-09-11
  • kas1e

    kas1e - 2013-09-11

    @all

    Fixed another error about "dereferencing pointer to incomplete type" by adding necessary structure to the xadopus.h from include/dopus/module.h (which we can't include as it will conflicts with all the stuff we have in the include/libraries/module.h, as well as it will make no sense to have that separate include/dopus/module.h file, all related to module should be in proto/library files).

    So while i add for now necessary struct to xadopus.h, it will be good idea to put missing stuff from include/dopus/modules.h to include/libraries/module.h

    Also fix all 'suggest parentheses around assignment used as truth value' warnings.

    commits 557 and 558.

    So for 68k-gcc build currently need to think about that SelectionList differences, as well as remove all warnings (almost all of them 'initialization makes integer from pointer without a cast' ).

     
    Last edit: kas1e 2013-09-11
  • kas1e

    kas1e - 2013-09-16

    @BSzili
    Tryed our latest version of xadmaster.module , and 68k version stop working. Once i dbl-click on archive, it bring a lister and a warning window "opening file failed".

    I do some recheck of DC_CALL macroses, and found that GetSource was wrong (line 904). I fix it and do commit (569), but it still didn't fix the problem.

    The i found, that you seems just forget to wrap that line and comment it out:

    filename = (STRPTR)data.hook.dc_ExamineEntry(Entry, EE_NAME);
    

    So, fixed, and commit 570 (through dunno if it ok to use STRPTR on infoptr for now..)

    And then, dbl-click on archive start to works, but some new interesting moment arise : it didn't unpack files normally (i.e. show not all files in archives, etc). And i found that it because of my change about EntryType and DirEntryType on line 198-199 currently ! I.e. we do the same kind of changes in the ftp.module before, and in end it seems that its not the same.

    I just checked now: build 68k version with original string:

    tree->fib.fib_NumBlocks=tree->fib.fib_DiskKey=tree->fib.fib_EntryType=0;
    

    All works as expected all files shows fine.

    And with replacement on DirEntryType , like:

    tree->fib.fib_NumBlocks=tree->fib.fib_DiskKey=tree->fib.fib_DirEntryType=0;
    

    And then it start to works badly, no shown files in the unpacked archives, etc.

    So, question now is : wtf ? didn't in the same should be ? By tests seems that not. As well, as then it mean we can have problems in ftp.module because of that as well.

    I check source of xadopus.c , and there DirEntryType used a lot in different places, but exactly on that string it uses only as EntryType (i.e. originally it was intended to be different).

    While we can kept it on os3 as it originally, what we can do then for os4 ? I just currently tried to comment it out fully and all works fine. So i comment it out just for os4 for now, but maybe worth to delete that string at all (as it seems pure "clear" of some thing which in no use). Commit #571

     
    Last edit: kas1e 2013-09-16
  • BSzili

    BSzili - 2013-09-16

    They are indeed different, I'll investigate the the EntryType vs DirEntryType issue. This guide suggest that they should be set to the same value: http://amigadev.elowar.com/read/ADCD_2.1/AmigaMail_Vol2_guide/node0060.html
    Well, I guess the practice is different. I wonder why did they remove this field on os4, see the fib_Obsolete field:
    struct FileInfoBlock
    {
    int32 fib_DiskKey; / -- FILESYSTEM PRIVATE !! /
    int32 fib_DirEntryType; / Use FIB_IS_ macros to identify object. /
    TEXT fib_FileName[108]; / Null terminated. /
    uint32 fib_Protection; / Bit mask of protection, rwxd are 3-0. /
    int32 fib_Obsolete; / -- OBSOLETE !! - no more access from V50 /
    uint32 fib_Size; / Number of bytes in file, only good to 4 gig /
    uint32 fib_NumBlocks; / Number of blocks in file /
    struct DateStamp
    fib_Date; / Date file last changed /
    TEXT fib_Comment[80]; / Null terminated comment associated with file /

    /* Note: the following fields are not supported by all filesystems.
       They should be initialized to 0 sending an ACTION_EXAMINE packet.
       When Examine() is called, these are set to 0 for you.
       AllocDosObject() also initializes them to 0. */
    uint16       fib_OwnerUID;        /* owner's UID */
    uint16       fib_OwnerGID;        /* owner's GID */
    uint8        fib_Reserved[32];
    

    };

     
    Last edit: BSzili 2013-09-16
  • kas1e

    kas1e - 2013-09-16

    @BSZili
    I commit os4 port as well: #572, seems works ok (at least dbl-click show new lister, unpack the archive, and i can d&d from it to another listers).

    For now will try to rebuild everything we have till that commit and test.

    In ftp.module i see that they also uses differently EntryType and DirEntryType, but as it was only 2 place where we change it if i remember right, then it imho can't be roots of owerflow of second buffer ..

     
  • BSzili

    BSzili - 2013-09-16

    That's most likely a different issue. I just checked the source of a few filesystems in the AROS source tree, and they set fib_EntryType and fib_DirEntryType to the same value on a ACTION_EXAMINE_OBJECT packet. I'm not even sure what fib_EntryType is supposed to do. You could try to define it to the fib_Obsolete field, or ask around in os4 forums. I wonder if the os4 SFS implementation is open source. For starters I'll check out the source of the AROS'.

     
  • BSzili

    BSzili - 2013-09-16

    I just looked into xadopus module, and I get now what was the problem... that last line just zeroes out the unused fields in FileInfoBlock:
    tree->fib.fib_NumBlocks=tree->fib.fib_DiskKey=tree->fib.fib_EntryType=0;
    If you zero out fib_direntrytype which is actually used, then you won't be able to tell which entry is a file and which entry is a directory. This settles it, that field shouldn't be bothered, since it's not used by the filesystem handlers or dopus5 for that matter. I'll delete it in a later commit, after doing some other cleanups.

     
  • kas1e

    kas1e - 2013-09-16

    @all
    Ok, xadopus.module done for os3/os4/aros. All should be ok now (through, that part with SelectionList replace to be re-checked).

     
  • kas1e

    kas1e - 2013-09-17

    And now ported to morphos too. But due to crashes in morphos version of filetype.module, we can't normally check if xadopus.module working fine too (because need to change filetypes). To be seen when filetype will be fixed on morphos

     
  • xenic

    xenic - 2013-09-17

    @BSzilli
    Here is what "The Amiga Guru Book" by Ralph Babel says about fib.EntryType:

    fib.EntryType - This field is not documented. For compatibility reasons, though, most handlers will set it to the same value as fib.DirEntryType. Programs should not rely on this behavior!

    If the best OS3 DOS documentation can't document fib.EntryType then Dopus5 probably should not be using it. The OS4 DOS documentation replaces it with "fib.Obsolete" like this:

    uint 32 fib_Obsolete; '/ -- OBSOLETE !! - no more access from V50 /'

     
  • xenic

    xenic - 2013-09-17

    @kas1e
    I just returned from vacation and haven't had much time to look at Dopus5. However, I have noticed several issues.

    In the xadopus module you have defined the xad library base as "struct xadMasterBase *xadMasterBase". In the new OS4 XadMaster includes at OS4Depot (xadmaster_includes.tar.bz2 ) the library base is declared as "struct Library *XadMasterBase" (the struct type is different and XadMasterBase is spelled differently). The new xad includes also have pragma pack() instructions for OS4 structure alignment. Which includes are you using?? Should we switch to the new includes or do I need to replace my new Xad includes with the old OS3 includes??

    Somebody removed the file "version.c" from the Program code and it no longer compiles. You have suggested that we update the Dopus5 version number and use the same version for the Program, Library and modules. We need to leave the version.c file in the Program code until we replace it with a master version file in the includes (version.h ??) that we can include globally and edit all Dopus5 modules, library and program to use the global versioning. It doesn't make sense to remove version.c and add the version strings to some other file when we will eventually need to change the versioning system as I mentioned above.

     
  • kas1e

    kas1e - 2013-09-17

    @xenic
    for xadmaster on os3 i use xadmaster includes from aminet, and for xadmaster for os4 i use ones from os4depot. i just placed them in sdks, and code as it now compiles for os3 and os4 and works (tested both versions on os4).

    as for version.c : you are right it should be there, we just ancidetaly delete it, but later put it back. i.e in latest current commit which are 605: its should be there and all should compiles. maybe you just on some previos commit ? we today do a lot if them, maybe you not in sync with test ?

    ps. palette ghosting seems disappear on os4 version, dunno when and how we fix it, seems some good sideeffect after some cleanups.

     
  • xenic

    xenic - 2013-09-17

    @kas1e
    O.K. I need to edit my Xad includes because they were changed for some reason.

    I'll checkout the latest files; they have been changing pretty fast.

    I investigated the OS4 palette issue before I left for vacation and didn't find the problem so it's good that it works now.

     
  • kas1e

    kas1e - 2013-09-18

    @Xenic

    I investigated the OS4 palette issue before I left for vacation and didn't find the problem so it's good that it works now.

    Sadly we do not know what exactly fix the problem on os4, because for morphos it still crashes like before and knowing how os4 one fixes for sure can help with mos fix..

    Here is what "The Amiga Guru Book" by Ralph Babel says about fib.EntryType:

    We already deal with as well, just didn't update that forum post before: i exchange few mails with ColinW (os4 DOS coder), and in general all what he say, is that NEVER EVER use fib.EntryType, and all code which still do it : broken and should be replaced on fib.DirEntryType.

    But with xadopus.module it was a bit different. Author just by some reasons used everywhere fib.DirEntryType, but in one place just "zeroed" fib.EntryType (maybe he think it will "clear" some things, dunno). So we just comment out that line at all and it fix all the problem.

    Currently it through just commented for os4, but imho we need to delete that line in whole from source, as it indeed just wrong to have it.

     
    Last edit: kas1e 2013-09-18
  • kas1e

    kas1e - 2013-09-19

    @all
    Tested latest builds of everything, and os4 native version of xadopus.module, crashes when we start to copy files from archive, with such stack-trace:

    Stack trace:
    (0x6469BCB0) [XADopus_progress.c:32] L_ProgressHook()+0x40 (section 1 @ 0x56B0)
    (0x6469BDF0) native kernel module kernel+0x00075bc0
    (0x6469BE50) module LIBS:xadmaster.library at 0x7FD18F88 (section 5 @ 0xF64)
    (0x6469BE80) module LIBS:xadmaster.library at 0x7FD19978 (section 5 @ 0x1954)
    (0x6469BEF0) module LIBS:xadmaster.library at 0x7FD18238 (section 5 @ 0x214)
    (0x6469BF70) module LIBS:xadmaster.library at 0x7FD221B4 (section 5 @ 0xA190)
    (0x6469BFA0) module LIBS:xadmaster.library at 0x7FD222C8 (section 5 @ 0xA2A4)
    (0x6469BFB0) module LIBS:xadmaster.library at 0x7FD42944 (section 5 @ 0x2A920)
    (0x6469BFF0) module LIBS:xadmaster.library at 0x7FD42C64 (section 5 @ 0x2AC40)
    (0x6469C030) module LIBS:xadmaster.library at 0x7FD1DA68 (section 5 @ 0x5A44)
    (0x6469C090) module LIBS:xadmaster.library at 0x7FD1816C (section 5 @ 0x148)
    (0x6469C110) [XADopus.c:585] _copy()+0x9f8 (section 1 @ 0x4184)
    (0x6469C220) [XADopus.c:1219] L_Module_Entry()+0x12e4 (section 1 @ 0x5564)
    (0x6469CE00) [init/aos4_ppc_libstub.c:16] libstub_L_Module_Entry()+0x28 (section 1 @ 0x5B2C)
    (0x6469CE10) [function_internal.c:194] function_internal_command()+0x52c (section 1 @ 0x66EF4)
    (0x6469CEE0) [function_run.c:320] function_run()+0x3e4 (section 1 @ 0x63E2C)
    (0x6469CF40) [function_run.c:40] function_run_function()+0x94 (section 1 @ 0x64148)
    (0x6469CF50) [function_filetype.c:163] function_filetype()+0x330 (section 1 @ 0x6AAB0)
    (0x6469CFB0) [function_launch.c:401] function_launch_codePPC()+0x218 (section 1 @ 0x61914)
    (0x6469CFE0) native kernel module kernel+0x0008d98c
    (0x6469CFF8) 0x0000FFF8 [cannot decode symbol]
    

    Strange that i didn't notice it before. OS3 version works fine of course, seems again call be those Callback or something.

    EDIT: ok, found , its just usual os4 ABI moment, added A2 reg where need it, so it fixed that crash, commit #610

     
    Last edit: kas1e 2013-09-19
  • kas1e

    kas1e - 2013-09-19

    And another bug in xadopus.module. Just set in the filetype on .lha for "drag & drop" : command XADExtract (so when you will d&d .lha archive to some lister, it will automatically unpacks).

    So, it start unpacking, but then die on some moment with such stack trace:

    Stack trace:
    (0x644CD220) [XADopus.c:1035] L_Module_Entry()+0x6f4 (section 1 @ 0x470C)
    (0x644CDE00) [XADopus.c:1034] L_Module_Entry()+0x6e4 (section 1 @ 0x46FC)
    (0x644CDE10) [function_internal.c:194] function_internal_command()+0x52c (section 1 @ 0x66EF4)
    (0x644CDEE0) [function_run.c:320] function_run()+0x3e4 (section 1 @ 0x63E2C)
    (0x644CDF40) [function_run.c:40] function_run_function()+0x94 (section 1 @ 0x64148)
    (0x644CDF50) [function_filetype.c:163] function_filetype()+0x330 (section 1 @ 0x6AAB0)
    (0x644CDFB0) [function_launch.c:401] function_launch_codePPC()+0x218 (section 1 @ 0x61914)
    (0x644CDFE0) native kernel module kernel+0x0008d98c
    (0x644CDFF8) 0x0000FFF8 [cannot decode symbol]
    

    After press on ignore DSI it bring "opening file failed" window.

    Bug happens and on os4 native and on os3 native versions, so we can assume it will be the same on aros and mos.

     
    Last edit: kas1e 2013-09-19
  • kas1e

    kas1e - 2013-09-20

    @all

    Some more info about crash in XADExtract : It happens just on .info files only. I.e. if we just create an lha archive with single .info file inside, then XADExtract fail, while XADOpen works fine.

    I even test original sas-s module (i.e. what we have from Amis), and bug is here. I even test 68k version from aminet on our fully os3-gcc compile of everything , and bug still the same and here.

    More of it, i just test original 68k-sasc compile (just with PFASYNC fix), with xadopus.module from aminet on OS4 , and it just give us the same crash.

     
    Last edit: kas1e 2013-09-20
  • kas1e

    kas1e - 2013-09-20

    Fixed in commit #613 by BSzili ! There was some original-bad loop without necessary checking.

     
  • kas1e

    kas1e - 2014-05-14

    @All

    Something weird happens with xadopus.module. Its not only crashes on exit from dopus5 when lister with content of archive open, but it just crashes when i a bit operate with it and close its lister, with the same crashlog as i point there:
    https://sourceforge.net/p/dopus5allamigas/bugs/39/

    I.e. its enough for me just now to do that:

    -- hard reboot
    -- run dopus5
    -- go to work
    -- click on some test.lha archive which is just:

    testdirectory1
    testdirectory1/file1.txt
    testdirectory1/file2.txt
    testdirectory2
    testdirectory1/file3.txt
    testdirectory1/file4.txt

    -- select testdirectory1
    -- copy it to any place
    -- close lister with archive content => crash.

    Stacktrace:

    Stack trace:
    (0x65F69EB0) native kernel module kernel+0x00020f0c
    (0x65F69EE0) native kernel module kernel+0x00020f0c
    (0x65F69EF0) [aos4_ppc_libstubs.c:1446] libstub_L_GetSemaphore()+0x1c (section 1 @ 0x49910)
    (0x65F69F00) [buffers.c:1171] buffer_lock()+0x38 (section 1 @ 0x33E38)
    (0x65F69F10) [function_support.c:433] function_cleanup()+0x60 (section 1 @ 0x69790)
    (0x65F69F40) [function_support.c:531] function_do_lister_changes()+0x38 (section 1 @ 0x6997C)
    (0x65F69F50) [function_run.c:58] function_run_function()+0xd8 (section 1 @ 0x658A4)
    (0x65F69F60) [function_filetype.c:163] function_filetype()+0x330 (section 1 @ 0x6C158)
    (0x65F69FC0) [function_launch.c:401] function_launch_codePPC()+0x21c (section 1 @ 0x630D4)
    (0x65F69FF0) native kernel module kernel+0x0006033c
    (0x65F6A000) 0x00010000 [cannot decode symbol]
    

    That with debug kernel + munge, so may, or may not be reproducable on user kernel. But on aros-hosted that imho should bring segfault as well.

    Additionally, there seems another big problem, which may or may not be related to that trashing.

    Just get attached archive. Which is pure:

    Kickstart/
    Kicksart/testfile2
    KickstartClassic/
    KickstartClassic/testfile2

    Dbl-click on it in lister, and unpack copy that content anywhere you want. And result is:

    Kickstart directory will contain some strange "lassic" directory with testfile2 inside.

    By dopus4 and by unarc all unpacks fine, which mean problems is not with xadmaster.library or lha xad client.

    Another problem i have when just tryied to unpack some 3mb size of .lha archive, which contain lots of small files/directories , and it bring me dopus requester with words: Opening file failed. And in the progress bar i have :

    Exracting ...

    MyDir
    /

    I attached few screenshots as well to show how it looks like.

     
    Last edit: kas1e 2014-05-14
  • kas1e

    kas1e - 2014-05-14

    @All

    Ok, problem #3 can be skiped: i just didn't have lha xadmaster client before (so it was just the same old story with newer .lha archives, when dopus4 shown in them garbage, and dopus5 just do that "skip" of them. Strange how it even woks before for other lha archives ..

    But problem #1 and #2 still remains.

     
  • xenic

    xenic - 2014-05-14

    @kas1e
    I tested some archives and can't see any problem. Since I don't see any numbers with the problems, I don't know what problem 1 and 2 are. Please tell me what the remaining problems are (1 and 2) so I can't test the problem.

    One thing you need to be careful about is the way Dopus5 changes source/destination listers. If you have lister 1 as source and lister 2 as destination, when you double-click an archive in lister 1 Dopus5 makes the new lister (3) the source and lister 1 becomes the destination. If you select something in the new lister (3) and copy then the copy goes to lister 1 when you think it's going to go to lister 2. You can even end up trying to copy an archive into itself. I don't like the way Dopus5 opens a new lister to show the contents of an archive.

     
  • kas1e

    kas1e - 2014-05-14

    @xenic
    just download attached test_cor.lha and try to unpack by dopus5 to some place. you will see that unpacked content will have additional halfword directory in the first unpacked directory. that about problem 2. as for first problem , you need to boot on debug kernel with munge.

     
  • xenic

    xenic - 2014-05-14

    @kas1e
    When I unpack test_cor.lha everything looks normal here. I unpacked it from ram: to a sub-directory in ram: and there is no halfword directory. Are you unpacking it on OS4?

     
  • kas1e

    kas1e - 2014-05-15

    @Xenic
    Yeah, that os4 version on os4. But i also tried the same on winuae with os3 version: the same bug. You probably just don't go to the Kickstart directory after unpacking ?

    To reproduce, you just unpack test_cor.lha like this:
    -- dbl-click on archive (so , new lister with content of test_cor spawns).
    -- choice any lister to where you will copy content from (does not matter, can be ram, or whatever you choice , bug always happens everywhere).
    -- mark content of that archive (those 2 directories).
    -- then drag&drop , or just "f5" , or press button "copy" of whole content of test_core (i.e. those 2 directories)
    -- now just go to unpacked directory called Kickstart. There you will find additional directory "lassic", which is not present in the archive, which will have content of the KickstartClassic directory.

    I attach you 2 screenshots to show problem , one from os4/os4 version, and one from winuae/os3 version. Bug for sure happens everywhere on all oses.

    See on them , directory "lassic" in the Kickstart directory are present after unpacking. Which is dbl-copy of the KickstartClassic directory, with the same content inside.

     
    Last edit: kas1e 2014-05-15
  • xenic

    xenic - 2014-05-15

    @kas1e
    I thought you were unpacking the whole archive. I now see that you are copying files from a lister where you are viewing the contents of an archive. I tested the SASC version of Dopus5 with the original 68k xadopus.module and the same problem is there too. It's not a new problem that we added. Copying directories from an archive displayed in a lister doesn't work in Dopus4 either; it skips directories and notifies the user that the object is of the wrong type. If nobody else could get copying directories from an archive lister to work then we should see if we can make the xadopus.module skip directories for a copy function.

     
1 2 > >> (Page 1 of 2)

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks