Re: [cpp-netlib-devel] HTTP Code Conversion (WAS RE: A few questions)

[Michael D. <mi...@mi...>]

On Mar 23, 2008, at 7:52 PM, Dean Michael C. Berris wrote:

> ...
>> 1) Why does the http::request class not derive from
>> basic_message?  I
>> noticed that http::response, OTOH does
>>
>
> Good question. There's really no straight answer yet, but I was in the
> middle of making an http::request object contain a basic_message  
> inside,
> and somehow make it convertible to a basic_message instead of deriving
> it from a basic_message. The reasons are:
>
> 1. Constructing an http::request is different from constructing a
> basic_message -- if you notice, the constructors are heavy and perform
> parsing of the URI.
>
> 2. An http::request may contain metadata that does not fit within a
> basic_message -- that being said, converting an http::request into a
> basic_message should be possible, but constructing one from a
> basic_message may be impossible to do in a straight forward manner
> (consider how you'd add the metadata like method, http version, etc.).
>
> 3. The request object is more for the consumption of an http::client
> rather than as a message that should be streamed to/from endpoints.  
> That
> being said, I agree though that it should somehow look like or at one
> point "be" a basic_message one way or another.

I would like to do this hierarchy conversion right away (in the  
branch).  There are many common aspects for HTTP requests and  
responses, i.e. they both have name/value headers, they both have  
content bodies, and the parsing semantics are almost identical  
(unfortunately "almost", because things like the response status code  
influence response parsing....).   A lot of the parser code is also  
simplified by these two concepts sharing a common base.

This does raise the question though of whether or not an  
http_message<> class would make sense?  i.e.

http::message<> : public basic_message<>;   (or http_message<>?)
http::request<> : public http::message<>;
http::response<> : public http::message<>;

Perhaps this makes more sense as we add more members to the http  
message objects...?

>> 2) I'm new to the art of making a "header only" library and am not
>> familiar with the techniques and best practices required for this
>> yet.  This became an issue for me in converting over the "types"
>> code.  What is the best way to define static constant variables that
>> will never change (strings, numbers, etc.)?  How do I avoid
>> "duplicate
>> symbol" errors in my compiler?
>
> The best way is to do something like this:
>
> template <typename tag=tags::default_tag>
> struct constants {
>  static typename tag::int_type const A_CONST = 1;
> };
>
> And in places where you'd need A_CONST, you'd do this instead:
>
> 	constants<>::A_CONST
>
> I hope this helps. ;)

Interesting... so does this work around the issue with compilers that  
don't normally let you define constant values in headers.  i.e.

static const unsigned int A_CONST = 25;

would work in some compilers, but not in others.  Does this "type- 
trick" work around that problem?


On a totally unrelated note, I noticed that you use "typename tag" in  
the templates in some places.. shouldn't this be "typename Tag" (based  
on the Boost template parameter naming guidelines?)


>> 3) A bug we recently fixed in Pion was with handling of HTTP header
>> data.  I see that that message using a multimap of strings and
>> performs case-sensitive comparisons.  I believe that headers need to
>> be stored in a case-insensitive manner (please see my types::headers
>> typedef in the new types.hpp file).  I'm not so sure that
>> this is the
>> case for other protocols though.   Should I disregard the "headers"
>> container in message and just use a different one for the
>> http classes?
> There is a transformation layer that's currently being developed and
> tested -- one of the things I've done some time ago is to enable  
> string
> transformations to be performed relatively efficiently using message
> transformation functions. We have two choices here if we intend to  
> allow
> case insensitive storage in the HTTP processing:
>
> 1. We should normalize the headers before they're put into a
> basic_message -- I think the HTTP Spec says something about the
> recommended Camel-case or Camel-Case but I'm hazy on where exactly  
> that
> is.
>
> 2. We can specialize on basic_message<http::message_tag> and define
> different storage mechanisms for the headers. It can be as simple as a
> case-insensitive std::multi_map or as complex as an
> unordered_case_insensitive_multi_map -- which is (fortunately)  
> entirely
> up to us. ;)

We chose choice #2 in pion-net for a couple reasons:

a) it's much simpler & easier than having to write specialized  
accessor functions whenever you interface with the container (which  
performs the string conversion)

b) it's probably also much faster

c) it allows for constants to work properly (see the http::types  
strings constants).  i.e.

namespace http { namespace types { namespace { headers {
	static const std::string CONTENT_LENGTH("Content-Length");	//  
definition uses "recommended" case
};

std::cout << types::headers::CONTENT_LENGTH << ": " <<  
request.header(types::headers::CONTENT_LENGTH) << types::CRLF;

As for using an "unordered" container, that is my personal preference  
and what we are using in pion-net.  The challenge with this is with  
compiler support; not everyone seems to support tr1 yet, but those  
which do not support alternatives (i.e. stdext::hash_map in gcc).  We  
have a separate "PionHashMap.hpp" header in pion that uses #define's  
to work-around the compiler differences.

I left it as a "multimap" in my code for the sole purpose of starting  
out keeping everything as simple as possible.  I do like the idea of  
hashing for these headers, though since this would (in theory) help  
increase the performance of lookups (of course, in practical use it  
might make no difference..)  If you prefer that route too, I can  
convert over the #define stuff we have in PionHashMap, maybe by adding  
a "unordered_map.hpp" details file?

>> 4) I have a few other container types to add to the http message
>> implementations, namely for query string and cookie parameters (see
>> the "query_params" and "cookie_params" typedefs in types.hpp).  What
>> would be the best approach to incorporate these into http::request?
>>
>
> Ah, good question.
>
> If you notice, http::request is a template -- you can hijack the tag
> parameter and require these types to be passed to the tag, or use a
> traits (meta)function (read: type) to determine the correct type for
> these extra containers in an http::request. Example would be something
> like this (roughly from memory):
>
> namespace http {
>  template <typename tag, ... >
>  struct request {
>    typename query_params<tag>::type query_params;
>    typename cookie_params<tag>::type cookie_params;
>    ...
>  };
> }

Interesting.. so we could leave it up to the user to define the  
container type for these (i.e. multimap or unordered_multimap)?

Sort of makes me wonder though if we'd be giving people too much rope  
(to hang themselves with)?  Versus making it a specific type, that's  
protected and only allows accessors?

> That would be the simplest and most flexible way to do it. Another  
> way,
> is to make that part of the fusion sequence encapsulated inside of an
> http::request instead of being part of the actual request object. This
> is how we currently package the data within an http::request, which
> would make it consistent and easily iterated upon in a template
> metaprogramming level -- for example, we might want to do compile-time
> processing for some operations instead of doing it at runtime like
> enabling normalizations for elements in a header, requirements  
> checking,
> etc.
>> 5) In general, what is the best way to add additional member (like
>> status code, for example) variables to the http::request and
>> http::response classes?  It doesn't look like they were
>> intended to be
>> extended, but maybe that is b/c the implementation is just missing...
>>
>
> The best way would be to add it in the fusion map used inside the
> http::request and http::response types. Since we're doing this 'header
> only' style, we best use the header only packaging that fusion  
> allows us
> to have. :)
>
> I can go into more detail if you need more information. :)

Guess I need to read up on the fusion library =)

Would this just be for container types or does it apply to simple  
types (like ints) as well?

> I hope this helps Mike!
>

Absolutely, thanks!

-Mike