Re: [cpp-netlib-devel] TODO's in http_integration

[Dean M. B. <mik...@gm...>]

On Wed, Aug 13, 2008 at 3:39 AM, Divye Kapoor <div...@gm...> wrote:
>
> On Tue, Aug 12, 2008 at 2:24 PM, Dean Michael Berris
> <mik...@gm...> wrote:
>>
>> About multi-part message posting, I have not seen any documentation
>> regarding the semantics for a multi-part POST message. If it has
>> something to do with MIME, then that library has yet to be written
>> external to the HTTP implementation. AFAICT, each HTTP POST Message
>> has a single body, and a single Content-Length and Content-Type
>> headers -- If the actual body is a multi-part message, then I think
>> the body should have to be formed accordingly, and the Content-Type
>> correctly as well to something like 'multi-part/mixed;
>> boundary="some_boundary"'. The HTTP Client will then be sufficient in
>> its current form to be able to do:
>
> The documentation for sending multiple files/form-fields/data via POST
> (multipart data) is in RFC 2388 (copy at
> http://www.faqs.org/rfcs/rfc2388.html ). A quick example of how the body is
> structured for a multipart POST request is located at
> http://www.codeproject.com/KB/cs/multipart_request_C_.aspx
>

Right, thanks.

> I think supporting this feature should be our responsibility. The user
> should not need to deal with the intricacies of structuring the data
> according to the rules of a multipart form. It will be a source of errors in
> the client application. The user should simply provide a content type and
> corresponding stream and we should draft a proper request out of it.
>

Like I implied, if it has something to do with MIME, we need to write
that library outside of HTTP. An imaginary MIME library would expose a
stream interface, and perhaps provide the following:

<code>

using namespace boost::network::mime;
omimestream multi_part_builder;
multi_part_builder
  << content(
    disposition(
      "form-data",
      make_tuple(
        make_pair("name", "param1")
        )
      ),
    data("param1Value")
    )
  << content(
    disposition(
      "file",
      make_tuple(
        make_pair("name", "file"),
        make_pair("filename", "filename1.txt")
        )
      ),
    type(
      "text/plain"
      ),
    data(file_stream)
  ;

...
client_.post(request, multi_part_builder.type(), multi_part_builder.body());

</code>

This allows us to use the MIME library outside of HTTP and be a usable
library in itself. In this case, the decoupling of the MIME handling
from the HTTP implementation will be a good thing.

I'm not saying it shouldn't be our responsibility -- I agree that we
should make it easier for users -- but how we're going to allow that
to happen has to fit within the goals of the project, which is
building a set of extensible components for building network-aware
applications in C++.

I can write that MIME builder interface with Boost.Proto or from
scratch, but that would require me to context-switch from the HTTP
implementation.

If anybody's interested in picking up that piece of the puzzle, or
would like to be part of the discussion about it, now would be a good
time to sound off. :D

>
>>
>> Another thing that may be required is to change the multi-map
>> underlying implementation for HTTP to use a list of std::pair<>'s to
>> preserve ordering. I've read something about the order of occurrence
>> and possible merging of repeating HTTP Headers.
>
> Some information about merging headers is located at
> http://httpd.apache.org/docs/2.2/mod/mod_headers.html
>

Thanks for the link. This is done on the server-side rather than the
client-side?

I think one thing we can do is to "un-merge" the merged headers
received from the server to make it consistent. In the case of merging
client-formed request headers, we can make it optional -- and should
be easy to do.

>> With multi-line headers processing, I'm not sure if this is ever going
>> to be a problem with HTTP -- I'm not sure of the HTTP spec requires
>> that headers be a single line -- but I am a little worried about our
>> lack of HTTP headers checking. Does anybody have experience with
>> special headers that may span multiple lines? Or am I imagining things
>> here? :D
>
> See section 4.2 - Message Headers of RFC 2616 (
> http://www.faqs.org/rfcs/rfc2388.html ).
> Headers can span multiple lines. This would require a change in the
> processing of the headers.
>

Oh bugger. Thanks for the link. Hmmm... Maybe a test would be good to
expose the bug? :D

>
>>
>> One last thing which needs some scrutiny: maybe we should define a
>> User-Agent: header to identify cpp-netlib? :D
>
> Maybe a default User-Agent header can be provided. Any other ideas?
>

A default should be provided unless it's overridden from the request,
I agree. Test, test, test... ;)

-- 
Dean Michael C. Berris
Software Engineer, Friendster, Inc.