#258 XzDec status is strange?

[ciel]

(In this ticket, patch is optional; need discussion first)

In decompression loop, I see the situation that "no decompression buffer nor compression buffer is consumed, but CODER_STATUS_NOT_FINISHED is set to status (by XzDec.c)".
I suppose in this case CODER_STATUS_NEEDS_MORE_INPUT should be set instead.
I'd like your opinion.

[Igor Pavlov]

I don't think that I need to fix it.
It returns CODER_STATUS_NEEDS_MORE_INPUT in another cases.

[ciel]

Then could you tell me what's wrong with this code?
In this code, shared buffer is 4MB and first 2MB is used for storing xz stream.
When I test, xz files larger than 2MB cannot be decompressed (without the "awful" line).

#include <stdio.h>
#include <stdlib.h>
#define BUFLEN (1<<22)
typedef unsigned char u8;
u8 buf[BUFLEN];

static void *SzAlloc(void *p, size_t size) { return malloc(size); }
static void SzFree(void *p, void *address) { free(address); }
static ISzAlloc g_Alloc = { SzAlloc, SzFree };

static int decode(CXzUnpacker *xz,FILE *fin, FILE *fout){
    ECoderStatus status=0;
    const size_t buflen=BUFLEN>>1;
    u8 *compbuf=buf;
    u8 *decompbuf=buf+buflen;
    size_t decomplen=buflen;
    int readlen=fread(compbuf,1,buflen,fin);
    size_t complen=readlen;
    size_t comppos=0;
    for(;status!=CODER_STATUS_FINISHED_WITH_MARK;){
        int complen_old=complen;
        if(XzUnpacker_Code(xz,
            decompbuf,&decomplen,
            compbuf+comppos,&complen,
            CODER_FINISH_ANY,&status))return status;
        if(!complen)status=CODER_STATUS_NEEDS_MORE_INPUT; //something awful happened...
        fwrite(decompbuf,1,decomplen,fout);
        decomplen=buflen;

        comppos+=complen;
        complen=complen_old-complen;
        if(status==CODER_STATUS_NEEDS_MORE_INPUT){
            if(complen)return -2;
            //memmove(compbuf,compbuf+comppos,complen);
            readlen=fread(compbuf+complen,1,buflen-complen,fin);
            if(readlen<0)return -3;
            comppos=0;
            complen+=readlen;
            if(readlen==0)break;
        }
    }
    for(;status!=CODER_STATUS_FINISHED_WITH_MARK;){
        if(XzUnpacker_Code(xz,
            decompbuf,&decomplen,
            compbuf+(readlen-complen),&complen,
            CODER_FINISH_END,&status))return status;

        fwrite(decompbuf,1,decomplen,fout);
        decomplen=buflen;
        if(XzUnpacker_IsStreamWasFinished(xz))break;
    }
    return 0;
}

int main(const int argc, const char **argv){
    if(isatty(fileno(stdin))&&isatty(fileno(stdout))){
        fprintf(stderr,"unxz <in.xz >out.bin\n");
        fprintf(stderr,"Both stdin and stdout have to be redirected\n");return -1;
    }
    CrcGenerateTable();Crc64GenerateTable();
    CXzUnpacker xz;
    XzUnpacker_Construct(&xz,&g_Alloc);
    int ret=decode(&xz,stdin,stdout);
    XzUnpacker_Free(&xz);
    return ret;
}

[Igor Pavlov]

Probably there were some reasons why I used such codes.

You can look XzHandler.cpp as example client code.
It reads input stream when it's possible.
And it uses CODER_STATUS_NEEDS_MORE_INPUT only to detect some error cases.

[ciel]

Thank you. I checked XzHandler and LzmaDecoder, and I now know the correct handling.
Hopefully these two are bundled in LZMA SDK, which means public domain (or "permissive")...