#229 decompress after codesign not working on OSX

[Dirk-Willem van Gulik]

Since the #87 change - code signing after upx compress works splendidly - but the larger text size seems to have broken decompressing (which obviously breaks the signature). It does not recognized the signed upx ensamble to be upx compressed (using striptease -x suggest it gets fooled by the too large a text size).

[John Reiser]

What is the meaning of "seems to have broken decompression"? Does the signed compressed file run on OSX? If so then the upx stub succeeded in decompressing the program into memory. Or does the complaint refer to "upx -d" does not work?

Please upload (Add attachments) two files: the first being an unsigned but compressed executable, the second being the signed version of the first, and where the second file suffers the complaint. The program can be something like
int x[10000] = {1,2,3};
main(){return 0;}
where the array is big enough to make compression worthwhile.

[Alexander Blach]

I just had the same problem and created the requested files:

UPXDecompressionIssue_uncompressed - the uncompressed compiled program
UPXDecompressionIssue_compressed - the compressed file
UPXDecompressionIssue_signed - the signed compressed file

The compressed file was generated with this command:

upx -o UPXDecompressionIssue_compressed UPXDecompressionIssue_uncompressed

I then copied the compressed file to UPXDecompressionIssue_signed and codesigned it with this command:

codesign --verbose --force --sign "<My Identity>" UPXDecompressionIssue_signed

Now decompression with "upx -d UPXDecompressionIssue_signed" fails:

upx: UPXDecompressionIssue_signed: NotPackedException: not packed by UPX

Unpacked 0 files.

The signed compressed binary still runs fine though!

[John Reiser]

Thank you for uploading the UPXDecompressionIssue_* files. Specific examples really help a lot.

Here is my analysis of what is happening. "upx -d UPXDecompressionIssue_signed" correctly complains "NotPackedException: not packed by UPX". The _signed file is the output of Apple codesign, and was not generated by UPX. The _compressesd file is the output from UPX, and is more than 9000 bytes smaller than the _signed file. In order to uncompress the _signed file then first it should be "unsigned": the alterations that Apple codesign made should be undone. That is, there should be a "-u" or "--unsign" argument to the codesign utility.

Apparently Apple codesign does not have such an "unsign" option, and thus lacks the required functionality. As a workaround, if you know the actual length of the corresponding _compressed file, then the "dd" command can be used:
dd if=_signed of=new_file_name bs=length_of_compressed count=1
For example
dd if=UPXDecompressionIssue_signed of=foo bs=4096 count=1
upx -d foo
File size Ratio Format Name
-------------------- ------ ----------- -----------
45768 <- 4096 8.95% Mach/AMD64 foo
and foo now compares equal to UPXDecompressionIssue_uncompressed .

[yk001]

I have a similar issue, signed-compressed binary used to run fine, up to OSX 10.9. It crashes in OSX 10.10. compressed-signed binary still can be run, but is it possible to fix the issue, so that my signed-compressed binary keeps running like it used to in OSX 10.9.

[John Reiser]

For better understanding, please describe the situation using the notation of functions compress() and sign(), instead of a hyphenation "signed-compressed" or "compressed-signed". Using the notation "compress(sign(my_prog))" makes it clear that sign() is performed first.

Can you please contruct a test case, such as

     #include <stdio.h>
     int x[10000] = {2, 3, 4};  // x[3..9999] = 0; highly compressable
     int main(int argc, char *argv[])
     {
       fprintf(stderr, "x[1]= %d/n", x[1]);
       return 5;
     }

then upload BOTH compress(sign(test_case)) and sign(compress(test_case)). Here "upload" means "Add attachments" to this bug report. Please give the symptoms, too: what are the observable effects of "crashes in OSX 10.10"?

[yk001]

Oops My mistake! compress(sign(Application)) was crashing with SIG_TRAP, while sign(compress(Application)) was running fine, so I figured there might be some problem with compress(sign(Application)), there wasn't. Actual problem was with a third party tool that failed to link properly with link(compress(sign(Application))). Doing a compress(sign(link(Application))) fixed the problem. What I thought was sign(compress(Application)) was in fact sign(compress(link(Application)), thats why it was running fine. I hope this is not very confusing for the readers, and thanks a lot for a quick response.