7-zip 4.09 beta and p7zip on a big endian mac

my space
2004-10-15
2012-12-08
  • my space

    my space - 2004-10-15

        I haved quickly port again the sources of 7za 4.09 beta to linux x86.

    The basic operations of 7za for linux seems to work.

    As you have said that 7-zip 4.x should work on a big endian machine,
    I have tested my port of 7za on a Solaris 8 workstation with a sparc CPU.
    Nothing seems to work :(

    7z409b_005.7z : my source compressed by 7-zip 3.13 (Windows version)
    abcd : a file

    On a Solaris, here is the errors :

    7za.exe l 7z409b_005.7z

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled

        Error: 7z409b_005.7z is not supported archive

        Errors: 1

      It seems that 7za.exe cannot understand the headers of the 7z format ?

    7za.exe t 7z409b_005.7z

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled

        Processing archive: 7z409b_005.7z

        Error: 7z409b_005.7z is not supported archive

        Archive Errors: 1

    7za.exe a abcd.7z abcd

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled

        Error: abcd.7z is not supported archive

        Error:

        System error:

     
      It seems that 7za.exe cannot find the right archive for the extension ".7z" !

    I have not the time and the means to debug the program on the Solaris workstation.

    Have you tested your code on a big endian machine ?

    As I have now little time for the maintenance of p7zip,  can we agreed with a
    rapid convergence of both of our sources ?

    I would like that you better maintain your codes that are in the "else" of "#ifdef WIN32".
    by "maintain", I mean at least a compilation (and link) of that code.

    To what machine and or OS have you easily access ?

    - the unix like environnement for Windows (www.cygwin.com)
      - pros : you stay in your Windows environnement
      - cons : the filesystem remains case-insensitive (you cannot detect that #include "QwZeRtY" does not work on linux)
               the unicode environement (i.e. UTF-8) does not seems to work

    - linux or FreeBSD
      - pros : you can test "case-sensitive filesystem" and unicode environment (UTF-8)
      - cons : need a new disk or partition

    - linux or FreeBSD in a virtual machine like www.wmware.com
      - pros : like "linux or FreeBSD"
               you can easily go and back between your virtual machine (Unix) and your real machine (Windows)
      - cons : wmware is not free
               your computer needs at least 512Mo of RAM and 4Go of free disk.

    Are you ready yo accept changes in your code that isolate windows stuff that
    are not in the directory "Windows" ?
    I think about :
    - HANDLE and the WaitForSingleObjet/WaitForMultipleObjects
    - ::CreateThread

    In conclusion, I would like to pass to a phase that :
    - you have the sources of 7za for Windows and Unix
    - you give it to me (or to other developpers) in order that sources are compiled
    and tested on linux/FreeBsd machine (and sometimes on a Sparc Solaris machine)
    - I (and other developpers) give to you the fixes
    - and so on as far as you think that you can do it by yourself.

     
    • Igor Pavlov

      Igor Pavlov - 2004-10-16

      Lets try to fix big-endian problem at first.
      1) Can it read empty .7z archive (without files)?
      if yes, Can it read/extract .7z archive with just one file or one folder?
      2) Can it read/extract from other formats: zip, gzip, tar, bz2?

       
      • my space

        my space - 2004-10-16
         
      • my space

        my space - 2004-10-16

        I have access to a Solaris workstation only during my job.
        So don't expect something before monday evening.

         
        • Igor Pavlov

          Igor Pavlov - 2004-10-16

          At sourceforge you can get access to PowerPC darwin/SSH/compile farm.

           
      • my space

        my space - 2004-10-18

        SourceForge 's Compile Farm is offline since saturday.
        Here is my tests of 7za for Solaris 8 with sparc CPU:

        7z:
        - OK : create(empty,onedir), list(empty, onedir,onefile),test(empty,onedir), extract(empty,onedir)
        - BAD: create(onefile), test(onefile), extract(onefile)
        gzip
        - OK : list, test, extract
        - BAD: create

        bzip2
        - OK : create,list,test,extract
        - BAD: -

        zip (onefile)
        - OK : create
        - BAD: list, test, extract

        Here is the "screencopy" of my tests :

        empty.7z : empty archive created with 7za 3.13 for Windows
        onedir.7z : archive with one directory (named "dir") created with 7za 3.13 for Windows
        onefile.7z : archive with one file (named "text.txt") created with 7za 3.13 for Windows

        >7za l empty.7z

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled

        Listing archive: empty.7z

           Date      Time    Attr         Size   Compressed  Name       
        ------------------- ----- ------------ ------------  ------------
        ------------------- ----- ------------ ------------  ------------
                                                             0 files    

        =============================================================================

        >7za t empty.7z

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled

        Processing archive: empty.7z

        No files to process

        =============================================================================

        >7za x empty.7z

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled

        Processing archive: empty.7z

        No files to process

        =============================================================================

        >7za l onedir.7z

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled

        Listing archive: onedir.7z

           Date      Time    Attr         Size   Compressed  Name       
        ------------------- ----- ------------ ------------  ------------
        2004-10-17 16:11:55 D....            0            0  dir        
        ------------------- ----- ------------ ------------  ------------
                                             0            0  1 files 

        =============================================================================
        >7za t onedir.7z

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled

        Processing archive: onedir.7z

        Testing     dir

        Everything is Ok

        =============================================================================

        >7za x onedir.7z

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled

        Processing archive: onedir.7z

        Extracting  dir

        Everything is Ok

        [the directory "dir" is really created by the command]

        =============================================================================

        >7za l onefile.7z

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled

        Listing archive: onefile.7z

           Date      Time    Attr         Size   Compressed  Name       
        ------------------- ----- ------------ ------------  ------------
        2004-10-17 16:11:38 ....A            5            9  text.txt   
        ------------------- ----- ------------ ------------  ------------
                                             5            9  1 files

        =============================================================================

        >7za t onefile.7z

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled

        Processing archive: onefile.7z

        Testing     text.txt     Data Error             <<<<<<<<<<<<<<<================ ERROR

        Everything is Ok

        Sub items Errors: 1             <<<<<<<<<<<<<<<================ ERROR

        =============================================================================
        >7za x onefile.7z

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled

        Processing archive: onefile.7z

        Extracting  text.txt     Data Error             <<<<<<<<<<<<<<<================ ERROR

        Everything is Ok

        Sub items Errors: 1             <<<<<<<<<<<<<<<================ ERROR

        [the file "text.txt" is created but has 0 bytes]

        >7za a -tgzip t.gz t

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled
        Scanning

        Creating archive t.gz

        Error:             <<<<<<<<<<<<<<<================ ERROR

        System error:
        GetLastError=0x80070057

        [ so t.gz was created with "gzip -9c t > t.gz" ]

        =============================================================================

        >7za l t.gz

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled

        Listing archive: t.gz

           Date      Time    Attr         Size   Compressed  Name       
        ------------------- ----- ------------ ------------  ------------
        2004-10-18 13:29:41           17395496      4042678  t          
        ------------------- ----- ------------ ------------  ------------
                                      17395496      4042678  1 files  

        =============================================================================

        >7za t t.gz

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled

        Processing archive: t.gz

        Testing     t

        Everything is Ok

        =============================================================================
        >7za x t.gz

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled

        Processing archive: t.gz

        Extracting  t

        Everything is Ok

        >diff -s t t.orig
        Files t and t.orig are identical

        [ this file "t" is the same as the original file "t"]

        >7za a -tbzip2 t.bz2 t

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled
        Scanning

        Creating archive t.bz2

        Compressing  t                                                           

        Everything is Ok

        =============================================================================
        >7za l  t.bz2

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled

        Listing archive: t.bz2

           Date      Time    Attr         Size   Compressed  Name       
        ------------------- ----- ------------ ------------  ------------
        26131-05-24 17:52:22                         2064808  t          
        ------------------- ----- ------------ ------------  ------------
                                                    2064808  1 files

        =============================================================================
        >7za t  t.bz2

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled

        Processing archive: t.bz2

        Testing     t

        Everything is Ok

        =============================================================================
        >7za x  t.bz2

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled

        Processing archive: t.bz2

        Extracting  t

        Everything is Ok

        [ this file "t" is the same as the original file "t"]

        > bzip2 -tv t.bz2
          t.bz2:   ok

        > bzip2 -d t.bz2
        [ this file "t" is the same as the original file "t"]

        Remark : 7za and bzip2 build the same file "t.bz2".

        =============================================================================

        >7za a -tzip t.zip t

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled
        Scanning

        Creating archive t.zip

        Compressing  t                                                           

        Everything is Ok

        =============================================================================

        >unzip -t t.zip
        Archive:  t.zip
            testing: t                        OK
        No errors detected in compressed data of t.zip

        =============================================================================

        >7za l  t.zip

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled

        Error: t.zip is not supported archive             <<<<<<<<<<<<<<<================ ERROR

        Errors: 1

        =============================================================================

        >7za t t.zip

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled

        Processing archive: t.zip

        Error: t.zip is not supported archive             <<<<<<<<<<<<<<<================ ERROR

        Archive Errors: 1

        =============================================================================
        >7za x  t.zip

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled

        Processing archive: t.zip

        Error: t.zip is not supported archive             <<<<<<<<<<<<<<<================ ERROR

        Archive Errors: 1

        =============================================================================

        >unzip t.zip
        Archive:  t.zip
          inflating: t

        >diff -s  t t.orig
        Files t and t.orig are identical

        =============================================================================
        >7za a local_empty.7z nothing

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled
        Scanning

        Creating archive local_empty.7z

        Everything is Ok

        =============================================================================

        >7za t local_empty.7z

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled

        Processing archive: local_empty.7z

        No files to process

        =============================================================================
        >7za a local_dir.7z dir

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled
        Scanning

        Creating archive local_dir.7z

        Everything is Ok

        7za l  local_dir.7z

        =============================================================================
        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled

        Listing archive: local_dir.7z

           Date      Time    Attr         Size   Compressed  Name       
        ------------------- ----- ------------ ------------  ------------
        42682-01-65289 65523:65477:65524 D....            0            0  dir        
        ------------------- ----- ------------ ------------  ------------
                                             0            0  1 files

        =============================================================================
        7za a local_file.7z t

        7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
        p7zip Version 0.92
        Support for files larger than 2GiB : Enabled
        Scanning

        Creating archive local_file.7z

        Error:             <<<<<<<<<<<<<<<================ ERROR

        System error:
        GetLastError=0x80004005

         
        • Igor Pavlov

          Igor Pavlov - 2004-10-18

          I think we must check "test" command for .7z for different archives:
          -m0=copy
          -m0=bzip2
          -m0=lzma
          -m0=deflate
          also you can simplify task: use file with length=1

           
          • my space

            my space - 2004-10-18

            The compile Farm is now online :)

            On Darwin ppc-osx2.cf.sourceforge.net,
            I cannot compile p7zip because "wchar.h" cannot be found ?!

            So I use the usf-cf-alpha-linux-1 2.2.20 (alpha Processor with linux)
            my first tests seems to indicate that 7za works on this target.
            It's difficult to pass all my tests because upload is slow
            and the compile farm is really slow and not easy to use (no GUI).
            I just realize that the alpha processor is a little endian CPU ...

            So It seems that it's the solaris and/or the sparc CPU that raises
            the problems with 7za.
            After some search, I realize that compile farm provides also a workstation
            with solaris 9 on a sparc CPU.
            After a very long period of compiling, here is the result :

            the ".7" archives were created with 7za 3.13 for Windows.

            7za t t_copy.7z : ERROR

            7za t t_bzip2.7z : ERROR

            7za t t_lzma.7z : ERROR

            7za t t_deflate.7z : ERROR

            Here is the error message :

            > 7za t t_deflate.7z

                7-Zip (A) 4.09 beta  Copyright (c) 1999-2004 Igor Pavlov  2004-10-05
                p7zip Version 0.92
                Support for files larger than 2GiB : Enabled

                Processing archive: t_deflate.7z

                Testing     t     Data Error

                Everything is Ok

                Sub items Errors: 1

            Any idea what sources/functions should I debug carefully ?

             
            • Igor Pavlov

              Igor Pavlov - 2004-10-19

              I also had wchar.h problem at MAC OS when I compiled SDK there.
              Now I don't remember how I solved it, but my String.h/String.cpp conatin such lines:

              #include <string.h>

              #ifdef WIN32
              #include "StringConvert.h"
              #else
              #include "ctype.h"
              #endif

              #include "String.h"

              About "Testing t Data Error" I'll answer later (I must think about it).

               
            • Igor Pavlov

              Igor Pavlov - 2004-10-19

              Lets check simplest case
              7za t t_copy.7z
              and use file with length=1.
              The question:
              Does is also give "Data Error"? Or it's "CRC error"?

               
              • my space

                my space - 2004-10-19

                "lza t  t1_copy.7z" gives "Data Error" (no CRC error is displayed)
                where t1_copy.7z contains the file t1 (length=1).

                Remark : my previous t_*.7z contain the file t (length=3).
                t is a 'a' followed by the 2 characters "newline for windows".

                 
            • Igor Pavlov

              Igor Pavlov - 2004-10-19

              In 7-zip now there is one_thread_mode for decompressing.
              It's still disabled, since it doesn't support some complex cases.
              But in simple cases (one method without filters) it works.
              So if the problem is in multi-thread code, you can try single thread version.

              To enable it you must edit two files:

              7zDecode.h:

              Add this:
              #define _ST_MODE
              before
              #ifdef _ST_MODE
              #include "../Common/CoderMixer2ST.h"
              #endif

              7zDecode.cpp:
              after:
                _multiThread = true;
                #ifdef _ST_MODE
                _multiThread = multiThread;
                #endif
              add this:
                _multiThread = false;

              2)
              And I also see date/time endian problem in bzip2 and 7z in you log.
              Note that in windows FILETIME is not UINT64, rather it's two UINT32:
                {
                  DWORD dwLowDateTime;
                  DWORD dwHighDateTime;
                }     FILETIME;
              So is it bug in my code or in your code?

               
        • Igor Pavlov

          Igor Pavlov - 2004-10-19

          I've just fixed some big-endian bugs in .zip .arj, and .rar
          So don't check them before next beta.
          .gz - it can be date/time problem. So we must solve that date/time problem before.

           
          • my space

            my space - 2004-10-19

            After some more debug, here is my result for 7za on a Sparc Solaris 8:

            - 7za can now create, test, list, extract ".7z" archive.
            (pthread_create returned -1 because "-lpthreads" was not in
            the link command ...)

            - 7za can now create, test, list, extract ".gz" archive.
            (my code for FILETIME was for a little-endian CPU)
            > .gz - it can be date/time problem. So we must solve that date/time problem before.
            yes that was the problem : because the date was wrong, your FileTimeToUnixTime
            function was returning "false".

            Sorry for disturbing you with false problems in you code.

            - 7za can now create ".zip" archive.
            But 7za cannot test, list, extract ".zip" archive.
            I think that the code in 7zip/archive/Zip/ZipIn.cpp
            is for a little-endian CPU.
            You use ReadBytes indeed of ReadUInt32 to read the signatures ...

            > I've just fixed some big-endian bugs in .zip .arj, and .rar
            > So don't check them before next beta.
            Ok.

            So, when do you plan to release the next beta ?

             
            • Igor Pavlov

              Igor Pavlov - 2004-10-19

              > So, when do you plan to release the next beta ?
              Probably in 2-3 days.
              What about MAC OS version problems?

               
              • my space

                my space - 2004-10-19

                The Mac OS X machine provided by the compile Farm are overloaded.

                I cannot connect to the two Mac OS X machines :(

                It seems that to have the wchar_t type you must include stddef.h instead of wchar.h.
                But the functions that deal with wchar_t are not provided ...

                I will try to connect another day because I think that a 7za for Mac OS X (from the Unix port) should be possible but without the support of true unicode filenames :(

                 
    • my space

      my space - 2004-10-16

      By "darwin" you mean Mac OS 10.x ?

      I haved opt-on an access to the compiler Farm.

      But a delay of 24H is needed before the access is granted ...

       

Log in to post a comment.