Re: [cpp-netlib-devel] Finished Integrating HTTP Incremental Parser

[Dean M. B. <mik...@gm...>]

Hey Guys!

Just an update. I'm still looking for a job, so if you've got any C++
jobs lying around, referrals would be greatly appreciated. I'm already
talking with a firm now though, but still keeping my options open.
Hopefully by November I'll go decide on what job to take.

Anyway, more information in-lined below.

On Wed, Sep 29, 2010 at 5:46 PM, Dean Michael Berris
<mik...@gm...> wrote:
>
>  * The tags system now uses heavy metaprogramming with Boost.MPL.

This is still true, and I highly recommend using clang [0] for your
development on Linux/Mac now.

>  * The tags "http_async_8bit_(udp|tcp)_resolve" are now supported and
> implemented; there should be no changes to client code that wants to
> use the asynchronous (I should say "active") HTTP client
> implementation.
>

It should be noted that the http client and the response objects are a
little heavier now in the asynchronous version. That's because the
client encapsulates a lifetime thread, and the response object
encapsulates futures internally; this uses Boost.Thread
implementations of the thread and the futures.

> There are a number of things that have yet to be done for packaging
> and releasing 0.7-devel:
>
>  * Make the HTTP 1.0 and HTTP 1.1 tests pass. Right now only the HTTP
> 1.0 test includes the new asynchronous tags in the build. The HTTP 1.1
> chunked-encoding transfer is what I intend to work on next, which will
> also require a new implementation of the incremental parser to support
> just the chunked-encoding body.

I still don't have the chunked transfer encoding done, and I would
really like to know if people do rely on chunked transfer encoding
before I go on working on this implementation. In the meantime the raw
body is being stored by the client regardless of the transfer
encoding.

>  * Update and merge the documentation.

I'll work with Glyn on this one.

>  * Implement a callback-based HTTP Client API. The rationale for this
> is so that instead of returning a response object that has
> Boost.Thread futures embedded in it (and neatly abstracted away), the
> client can then just call the appropriate callback(s) for processing
> headers and the body stream as they become available.

I realize that this requires a worker thread pool implementation which
has a "life of its own" and lives on different threads that don't deal
with I/O. This may have to wait as I have to implement this thread
pool implementation as well.

>  * Implement a streaming HTTP Client API. This means returning a
> response object that has a stream for a body instead of a pre-built
> string, as well as taking a request object that has a stream for a
> body. This should help with file-uploads through PUT/POST to make it
> not require all the contents of the file be in memory, a shared_ptr<>
> to a stream should suffice.
>

Also, this means that the response type may have to change for the
tags that enable this interface. What has to happen is the response
objects will contain a shared iostream that's bound to the socket of
the HTTP request to the server, that starts with the body of the HTTP
Response Message. This is in the case that the callback would like to
deal with the response body as a stream. I have an implementation
somewhere that doesn't expose a stream and holds the contents of the
response in memory as a string, which should do for most cases that
the responses aren't huge.

> As of now I don't have access to a MSVC compiler yet -- I'll see if I
> can get my hands on a Windows machine or VM soon, so that I can
> continue development on Windows -- but I have been building and
> testing with Clang and GCC in Linux, against Boost 1.44.0. I need some
> help in this regard as well as with the identification of issues in
> case there are any that you come across in the implementation.
>

Now that the tests build and pass on Linux with GCC and Clang, can
someone help out with the Windows testing?

Thanks guys and I hope this helps.

PS. Please review the code and let me know if you find any glaring
bugs or really ugly things that need explanation. That would be
greatly appreciated. [1]

[0] clang -- http://clang.llvm.org/
[1] 0.7-devel -- http://github.com/mikhailberis/cpp-netlib/tree/0.7-devel

-- 
Dean Michael Berris
deanberris.com