Work at SourceForge, help us to make it a better place! We have an immediate need for a Support Technician in our San Francisco or Denver office.

Close

Chunked Transfer-Encoding

2005-03-10
2013-05-02
  • Paul Noffke
    Paul Noffke
    2005-03-10

    I've been using CyberLink against the MusicMatch music server and found that it didn't support the chunked encoding method.

    Below is new version of HTTPPacket.set that does.

    Cheers
    Paul

        protected boolean set(InputStream in, boolean onlyHeaders)
        {
             try {
                BufferedReader reader = new BufferedReader(new InputStreamReader(in));
               
                String firstLine = reader.readLine();
                if (firstLine == null || firstLine.length() <= 0)
                    return false;
                setFirstLine(firstLine);
               
                // Thanks for Giordano Sassaroli <sassarol@cefriel.it> (09/03/03)
                HTTPStatus httpStatus = new HTTPStatus(firstLine);
                int statCode = httpStatus.getStatusCode();
                if (statCode == HTTPStatus.CONTINUE){
                    //ad hoc code for managing iis non-standard behaviour
                    //iis sends 100 code response and a 200 code response in the same
                    //stream, so the code should check the presence of the actual
                    //response in the stream.
                    //skip all header lines
                    String headerLine = reader.readLine();
                    while ((headerLine != null) && (0 < headerLine.length()) ) {
                        HTTPHeader header = new HTTPHeader(headerLine);
                        if (header.hasName() == true)
                            setHeader(header);
                        headerLine = reader.readLine();
                    }
                    //look forward another first line
                    String actualFirstLine = reader.readLine();
                    if ((actualFirstLine != null) && (0 < actualFirstLine.length()) ) {
                        //this is the actual first line
                        setFirstLine(actualFirstLine);
                    }else{
                        return true;
                    }
                }
               
                String headerLine = reader.readLine();
                while ((headerLine != null) && (0 < headerLine.length()) ) {
                    HTTPHeader header = new HTTPHeader(headerLine);
                    if (header.hasName() == true)
                        setHeader(header);
                    headerLine = reader.readLine();
                }
               
                if (onlyHeaders == true) {
                    setContent("", false);
                    return true;
                }
               
                long conLen = getContentLength();
                StringBuffer conBuf = new StringBuffer();
                if (0 < conLen) {
                    int chunkSize = HTTP.getChunkSize();
                    char readBuf[] = new char[chunkSize];
                    long readCnt = 0;
                    while (readCnt < conLen) {
                        try {
                            // Thanks for Mark Retallack (02/02/05)
                            long bufReadLen = conLen - readCnt;
                            if (chunkSize < bufReadLen)
                                bufReadLen = chunkSize;
                            int readLen = reader.read(readBuf, 0, (int)bufReadLen);
                            if (readLen < 0)
                                break;
                            conBuf.append(new String(readBuf, 0, readLen));
                            readCnt += readLen;
                        }
                        catch (Exception e)
                        {
                            Debug.warning(e);
                            break;
                        }
                    }
                }
                else
                {
                    HTTPHeader encodingHeader = getHeader("Transfer-Encoding");
                    if (encodingHeader != null && "chunked".equals(encodingHeader.getValue()))
                    {
                        char chunkBuf[] = new char[HTTP.getChunkSize()];
                        int chunkLen;
                        while (true)
                        {
                            // Get the next chunk's length
                            String chunkLenStr = reader.readLine();
                            try
                            {
                                chunkLen = Integer.parseInt(chunkLenStr, 16);
                            }
                            catch (NumberFormatException e)
                            {
                                Debug.warning("Invalid chunk length." +
                                        "  Value: " + chunkLenStr);
                                return false;
                            }

                            // A chunk length of 0 indicates the end
                            if (chunkLen == 0)
                            {
                                break;
                            }

                            // Resize the chunk buffer - shouldn't ever happen
                            if (chunkLen > chunkBuf.length)
                            {
                                chunkBuf = new char[chunkLen];
                            }

                            // Read the chunk
                            int chunkRead = 0;
                            while (chunkRead < chunkLen)
                            {
                                int readLen = reader.read(chunkBuf, chunkRead, chunkLen - chunkRead);
                                if (readLen <= 0)
                                {
                                    Debug.warning("Premature end of chunk." +
                                            "  Expected length: " + chunkLen +
                                            ", Actual Length: " + chunkRead);
                                    return false;
                                }
                                chunkRead += readLen;
                            }

                            // Append it to the buffer
                            conBuf.append(chunkBuf, 0, chunkLen);

                            // Each chunk is terminated by a CR/LF pair
                            chunkLenStr = reader.readLine();
                            if (chunkLenStr == null || chunkLenStr.length() > 0)
                            {
                                Debug.warning("Chunk not terminated with \\n\\r pair.");
                                return false;
                            }
                        }
                    }
                }
                if (conBuf.length() > 0)
                {
                    // Thanks for Ralf G. R. Bergs (02/09/04)
                    String conStr = conBuf.toString();
                    //String trimStr = conStr.trim();
                    //setContent(trimStr.getBytes());
                    setContent(conStr.getBytes(), false);
                }
            }
            catch (Exception e) {
                Debug.warning(e);
                return false;
            }
           
            return true;
        }

     
    • Satoshi Konno
      Satoshi Konno
      2005-03-11

      Hi Paul,

      Thanks for your useful patches about the chanked stream.

      I am adding the changed stream support too to CyberLink C++ and Java, and I am testing the function using some test cases.

      I will reffer your codes to implement this function :-)