Embedding LZMA with LzmaLib - Receiving Raw Binary over UART

[TRAyres]

Hi everyone,

I took a look through the source and found LzmaLib.c/LzmaLib.h, which appear to contain the compress/decompress functions I'd need.

I'm working on a Zynq FPGA's Processing Subsystem (So an ARM Cortex A9). I also took the LzmaUtils in lzma1604\C\Util\Lzma, and compiled it as an x86 project with both MSYS2 using make -f makefile.gcc and, separately, as an Eclipse project so I could easily debug what was going on easier.

It looks like header bytes 5,6, and 7 encode the final size, and I can easily have the command program determine the input size before the file is sent.

Is there anything else I need to do to successfully use LzmaCompress that I am missing? Is this a good plan?

Thanks!
-Travis

[TRAyres]

Well now I feel stupid, it's offset 5 to 8 and it's a 64 byte unsigned integer. Nevertheless, the question stands - do I need to do anything else to call Lzmauncompress? Do I even need to strip the headers?

Thanks!

[Igor Pavlov]

SRes LzmaDecode(Byte *dest, SizeT *destLen, const Byte *src, SizeT *srcLen,
    const Byte *propData, unsigned propSize, ELzmaFinishMode finishMode,
    ELzmaStatus *status, ISzAllocPtr alloc);

5 bytes - properties (you send it propData)
8 bytes - uncompressed size (if is not FFFFFFFFFFFFFFFF), you send it in destLen.
another data - compressed data.

[TRAyres]

Ok! So now I can receive, uncompress, and retransmit my data!

It's incredible! However, I had to hardcode the source and destination lengths - for some reason I was having trouble picking them out of the header.

I know this is currently very ugly (and hard-coded), but unpackSize was giving me a huge (incorrect) size, and when I hardcode the lengths it is working.

                int i;
              /* header: 5 bytes of LZMA properties and 8 bytes of uncompressed size */
              unsigned char header[LZMA_PROPS_SIZE + 8];
              UInt64 unpackSize;
              //Read LZMA_PROPS_SIZE+8 bytes into header!

              for(i=0; i < sizeof(header); i ++){
                  header [i] = XUartPs_RecvByte(UART_BASEADDR);
              }


              /*for (i = 0; i < 8; i++){
                unpackSize += (UInt64)header[LZMA_PROPS_SIZE + i] << (i * 8);
              } */
              unpackSize = 655360; //<- The above didn't work so I hardcoded the unpacked size
              unsigned char * src;
              src = malloc(1254-sizeof(header)); //<-Hardcoded src size as well


             for (i=0; i < 1254-sizeof(header); i ++){
                 src[i] = XUartPs_RecvByte(UART_BASEADDR);
             }

            unsigned char * dest;

            SizeT srcLen = 1254-sizeof(header);//<-Hardcoded src size as well

            //Copied Props from header
            unsigned char props[5];
            for (i = 0; i < 5; i++){
                props[i] = header[i];
            }

            dest = malloc(sizeof(unsigned char)*unpackSize);
            LzmaUncompress(dest, &unpackSize, src, &srcLen,props, sizeof(props));

            for(i = 0 ; i < unpackSize; i++){
                XUartPs_SendByte(UART_BASEADDR,dest[i]);

            }

[Igor Pavlov]

1)

              UInt64 unpackSize = 0;

2) you don't need copy data to props, just send header. Smaller code.

3) if you need compressed size, you can store it in file just before 13 bytes of lzma header. So no need that "hardcode".