#2449 Files packed from STDIN have no attributes

[eugenesan]

On Linux, files in 7z archives packed from STDIN have attributes at all.

Expected behavior is to have default attributes: 664 (-rw-rw-r--)

P.S.
There a few bugs on GitHub but activity.
If issues there are ignored maybe issue tracked should be disabled there and redirected here.

[Igor Pavlov]

Expected behavior is to have default attributes: 664 (-rw-rw-r--)
Expected behavior is to have default attributes: 775 | rwxrwxr-x

So what is Expected behavior?
Can you test another programs in Linux fot same thing?
you can look attributes for another created archives so:

7zz l a.7z -slt
7zz l a.zip -slt

[eugenesan]

Thank you for responding so quickly.

Please find below the log of my testing.
I've included a few examples of expected behavior for reference.
Please note that only the file extracted from the archive created using STDIN has no attributes at all ---------- while in all other examples it has default attributes -rw-rw-r--.

$ 7z l -slt data.rar

Listing archive: data.rar

--
Path = data.rar
Type = Rar5
Physical Size = 82
Solid = -
Blocks = 1
Encrypted = -
Multivolume = -
Volumes = 1

----------
Path = 0.data
Folder = -
Size = 4096
Packed Size = 17
Modified = 2001-02-02 12:05:06
Created = 
Accessed = 
Attributes =  -rw-r--r--
Alternate Stream = -
Encrypted = -
Solid = -
Split Before = -
Split After = -
CRC = C71C0011
Host OS = Unix
Method = m3:17
Characteristics = UnixTime CRC
Symbolic Link = 
Hard Link = 
Copy Link = 
Volume Index = 
Checksum = 
NT Security = 

$ 7z x data.rar
$ ll 0.data

-rw-rw-r-- 1 username username 4096 Feb 2 2001 0.data

$ 7z a data.file.7z 0.data
$ 7z l -slt data.file.7z

Listing archive: data.file.7z

--
Path = data.file.7z
Type = 7z
Physical Size = 155
Headers Size = 122
Method = LZMA2:12
Solid = -
Blocks = 1

----------
Path = 0.data
Size = 4096
Packed Size = 33
Modified = 2001-02-02 12:05:06.0000000
Attributes = A -rw-rw-r--
CRC = C71C0011
Encrypted = -
Method = LZMA2:12
Block = 0

$ 7z x data.file.7z
$ ll 0.data
-rw-rw-r-- 1 username username 4096 Feb 2 2001 0.data

$ cat 0.data | 7z a -si 0.data.7z
$ 7z l -slt 0.data.7z

Listing archive: 0.data.7z

--
Path = 0.data.7z
Type = 7z
Physical Size = 123
Headers Size = 90
Method = LZMA2:24
Solid = -
Blocks = 1

----------
Path = 0.data
Size = 4096
Packed Size = 33
Modified = 2024-02-05 11:32:27.0558920
Attributes = RA ----------
CRC = C71C0011
Encrypted = -
Method = LZMA2:24
Block = 0

$ 7z x 0.data.7z
$ ll 0.data
---------- 1 username username 4096 Feb 5 11:32 0.data
$ 7z x data.file.7z
$ ll 0.data
-rw-rw-r-- 1 username username 4096 Feb 2 2001 0.data
$ cat 0.data | xz > 0.data.xz
$ 7z l -slt 0.data.xz

Listing archive: 0.data.xz

--
Path = 0.data.xz
Type = xz
Physical Size = 92
Method = LZMA2:23 CRC64
Streams = 1
Blocks = 1

----------
Size = 4096
Packed Size = 92
Method = LZMA2:23 CRC64

$ 7z x 0.data.xz
$ ll 0.data
-rw-rw-r-- 1 username username 4096 Feb 5 11:37 0.data

$ xz -d -f 0.data.xz
$ ll 0.data
-rw-rw-r-- 1 username username 4096 Feb 5 11:37 0.data

[Igor Pavlov]

the question is about programs like rar,tar,cpio,zip that can store attribute.
And if these programs can read input data from stdin, what exact properties they writeto archive?
So we must create more archives from stdin with different programs.
And then we can look "7z l -slt arc" for them to check what exact value they write.

We can write any of these:
777
775
755
666
664
644
what way is better?

[eugenesan]

There is an example of XZ in my log cat 0.data | xz > 0.data.xz.

I just retested with XZ, Bzip2, Gzip and ZSTD and they all do the same, they give the output file default permissions 644

[x] read by owner
[x] write by owner
[x] read by group
[x] write by group
[x] read by others

Tar, CPIO, rar zip etc are not compressors and are not expected to work with STDIN/STDOUT.
7Zip is an exception in that regard due to it's universality ;-)

[Igor Pavlov]

XZ, Bzip2, Gzip and ZSTD can not be used as example, because they do not store file "mode" attributes.
"7z l a.xz -slt" doesn't show any mode value.

TAR ZIP RAR CPIO XAR can be used, because they store some "mode" value.
"7z l a.zip -slt" shows mode value.

[eugenesan]

tar and cpio expect a file list and can not accept STDIN.
I could not analog for 7z's -"si" parameter for rar and zip.
Not sure about XAR.

7z is unique by supporting "-si".
Judging by behavior of all other compressors, 7z should assign default file permissions to files in archive that were "sourced" from STDIN.

[Igor Pavlov]

zip < 1.txt > 2.zip
rar a a.rar -si1.txt < 1.txt

zip writes as 777 mode:

Attributes =  -rwxrwxrwx

and empty attributes in rar as I suppose in rar.
But probably we can get more examples.
If we can't get more examples, maybe I can use 777 as in zip.

[eugenesan]

777 is the default for Windows oriented tools this is why Zip and Rar on Windows behave this way.
On Unix 777 makes the file executable and readable/writable by everybody which is not ideal.

Just tested Rar with -si, and it stores files in archive as 644 so ideally 7z -si should do the same.
Also 7z does the same when storing regular files so it is consistent with existing behavior.

[Igor Pavlov]

rar doesn't stores 644.
your tests were incorrect.
Note that there are different things:
1) the value stored in archive
2) the value that set in filesystem when archive is extracted.
These values are not same.

The question is what value to store to archive.
zip stores 777
rar stores nothing (probably).
p7zip stores nothing (probably).
7-zip stores 000. and it can be problem.
(000 != nothing)
We can store nothing or 777 or some other value.

[eugenesan]

On my system with command line you've provided for RAR, it behaves as I showed earlier.
Maybe different versions of RAR behave differently?

RAR 6.23   Copyright (c) 1993-2023 Alexander Roshal   1 Aug 2023
Trial version             Type 'rar -?' for help

Archive: a.rar
Details: RAR 5

 Attributes      Size     Date    Time   Name
----------- ---------  ---------- -----  ----
 -rw-r--r--      4096  2024-02-05 13:22  1.txt
----------- ---------  ---------- -----  ----
                 4096                    1

-rw-r--r-- 1 username username 4096 Feb  5 13:22 1.txt

[Sam Tansy]

Expected behavior is to have default attributes: 664 (-rw-rw-r--)

It's not.

$ rm -f a; echo a > a; ls -l a
-rw-r--r-- 1 user users 2 Feb  5 21:21 a
$ rm -f d; mkdir d; ls -l | grep d
drwxr-xr-x 2 user users 40 Feb  5 21:21 d

Default is 644 (-rw-r--r--) for files and 755 (drwxr-xr-x) for directories.
It is controlled by umask (1), which is 0022 by default.

You can either set it to 644 by default, or read default file mask and set permissions accordingly.

P7zip does not set in-archive attributes at all and newly created files use default mask.

$ echo x | 7z a -si x.7z && 7z x x.7z; ls -l x
-rw-r--r-- 1 user users 2 Feb 5 21:27 x

$ echo xx | zz a -si xx.7z && 7zz x xx.7z; ls -l xx
---------- 1 user users 3 Feb 5 21:28 xx

$ 7z l -slt x.7z | grep Attributes
Attributes =

$ 7z l -slt xx.7z | grep Attributes
Attributes = RA_ ----------

Maybe that's the way to solve it, to let the system 'decide' what is default mode. Or, as said before, to read default mask and apply it to get default mode.

---

Rar calls file 'stdin' and set mode to 0, which is the same as 7zz now.

$ echo y | rar a -si -inul y.rar && rar x y.rar; ls -l
---------- 1 user users 2 Feb 5 21:29 stdin
-rw-r--r-- 1 user users 87 Feb 5 21:29 y.rar

and info zip calls it '-' and does its onw thing.

$ echo z | zip -q z.zip - && unzip -q z.zip; ls -l
-rw------- 1 user users 2 Feb 5 21:30 -
-rw-r--r-- 1 user users 198 Feb 5 21:30 z.zip

[Igor Pavlov]

Extract command is not important for as now.
Create archive comand and "7z l -slt" are important only.
Extracting can change attributes.
Now we think about stored value only.

[Sam Tansy]

Rar does not store any attributes.

$ echo y | rar a -si -inul y.rar; 7z l -slt y.rar | grep Attributes
Attributes =

It does its own, IMO wrong, thing during unpacking and creating a file.

Zip does.

$ echo z | zip -q z.zip - ; 7z l -slt z.zip | grep Attributes
Attributes = _ prw-------

It (zip) probably has to as they are 'static' and left unset would be zero anyway.
It marks it as 'p', which I guess is 'pipe'. That could be a clue if 7zip had that attribute.

[eugenesan]

I see.
Rar probably displays default permissions for the host system when attributes are empty and sets default permissions on extraction.
I guess 7zip should do the same.

[Sam Tansy]

Rar probably displays default permissions for the host system when attributes are empty

It rather shows cleared attributes, name 0x000...

and sets default permissions on extraction.

Rather sets zero, not default, to newly created file. Default would be 644.

[eugenesan]

This what I see when testing with RAR 6.23 on Linux

$ cat 1.txt | rar -si a 2.rar
$ rar l 2.rar

RAR 6.23   Copyright (c) 1993-2023 Alexander Roshal   1 Aug 2023
Trial version             Type 'rar -?' for help

Archive: 2.rar
Details: RAR 5

 Attributes      Size     Date    Time   Name
----------- ---------  ---------- -----  ----
 -rw-r--r--      4096  2024-02-05 15:13  stdin
----------- ---------  ---------- -----  ----
                 4096                    1

$ rar x 2.rar
$ ll stdin

-rw-r--r-- 1 username username 4096 Feb  5 15:13 stdin

[Sam Tansy]

Different version of rar: v5.91.

And, as you can see, rar set whatever it stored, rather than create new one and not change the system defaults.

[eugenesan]

Indeed. Seems like Rar 6.x behaves more "Unixy" ;-)
$ rar l stdin.rar5

RAR 6.23   Copyright (c) 1993-2023 Alexander Roshal   1 Aug 2023
Trial version             Type 'rar -?' for help

Archive: stdin.rar5
Details: RAR 5

 Attributes      Size     Date    Time   Name
----------- ---------  ---------- -----  ----
 ----------         2  2024-02-05 15:40  stdin
----------- ---------  ---------- -----  ----
                    2                    1

[eugenesan]

And it seems like Rar deals with the problem when packing and not when extracting:

$ rar x stdin.rar5
$ $ ll stdin

---------- 1 username username 2 Feb  5 15:40 stdin

[Sam Tansy]

And it seems like Rar deals with the problem when packing and not when extracting:

Zip too.

P7zip, does not store anything and does not set anything on creation.

Also setting attributes while packing makes sense that file with zeroed attributes could not be added to archive.

$ echo y > y; chmod 000 y; rar a y000.rar y 

Creating archive y000.rar

Cannot open y
Permission denied
WARNING: Cannot open 1 file
WARNING: No files

Unless you're root/admin, then it could. But that's nonsense to assume that user is administrator.

[eugenesan]

Mimicking p7zip behavior sounds good, especially since majority of recent Linux distributions switched to 7zip 23.01 from p7zip 16.02.

Either approach will work as long as resulting files/dirs have sane permisions (644/755).
Choose the the method that is easier to implement.

FYI:
This is the script that is packaged with 7zip to mimics p7zip's behavior: https://salsa.debian.org/debian/7zip/-/blob/master/debian/scripts/p7zip.sh

[Igor Pavlov]

I have tested rar again.
it writes DT_UNKNOWN instead of DT_REG to bits 12-15 of properties:

0rw-r--r--

Probably I'll not write attribute properties to 7z archive, because it's possible to create 7z archive without attributes metadata.
But we still neeed some value for zip archive.
So we can write DT_REG and 777 to zip in high 16-bits of 32-bit value.

[Sam Tansy]

With zip you may do what zip does - mark it as pipe and write mode as would local umask apply.

$ echo z | zip -q z.zip - ; 7z l -slt z.zip | grep Attributes
Attributes = _ prw-------

But it should obey local umask not hardcoded 0066, that would produce mode 600.

IMO, It should look like this:

$ umask 0022
$ echo z | zip -q z.zip - ; 7z l -slt z.zip | grep Attributes
Attributes = _ prw-r--r--

$ umask 0002
$ echo z | zip -q z.zip - ; 7z l -slt z.zip | grep Attributes
Attributes = _ prw-rw-r--

$ umask 0066
$ echo z | zip -q z.zip - ; 7z l -slt z.zip | grep Attributes
Attributes = _ prw-------