Re: [cpp-netlib-devel] Boost.CGI Merger (WAS RE: Nearly time to start merging?)

[Darren G. <dar...@gm...>]

Hi again Dean,

On 25/03/2008, Dean Michael C. Berris <dmb...@fr...> wrote:
>
>
> > -----Original Message-----
> > From: cpp...@li...
> > [mailto:cpp...@li...] On
> > Behalf Of Darren Garvey
>

> Perhaps we can open discussion about where things should and
> > shouldn't align and how best to do that? It'll be summer
> > before I have any real time to spend on this, but I'm really
> > keen that we can make some progress on this.
> >
>
> Like Glyn, I'll try looking into the code. However, I may very well be
> concentrating more in helping out Mike with the integration of
> pion-net's HTTP code into the library. It would be a dream come true if
> somehow the HTTP message code we're developing will be the same one the
> CGI library will be using! :)


This is the code in (SVN_ROOT)/branches/http_integration/ ? I've had a brief
look at it a couple of times, it has great potential. I think the first
steps towards integration of the CGI library is to start using the
basic_message<> class, which I think should be pretty straight-forward.

> P.S.
> >
> > Ideas:
> > * add a form_parser class (as part of the public API)
> >    - use Boost.Spirit
> >    - should read/parse multipart/form-data regardless of request type.
> >    - should allow incremental reading/parsing of individual form parts
>
> I like this. :)
>
> We can even do this using the TDD approach where we can write the unit
> test(s) independent of any subsystem, and develop the parser(s) to
> implement to make the unit tests pass. This way, the compliance to
> parsing multipart/form-data can be developed independently.


Exactly! All this requires is using Boost.Asio's (A)Sync(Read|Write)Stream
concepts, so shouldn't require too much isolated code.

> * add a param class
> >    - holds a boost::iterator_range to the data
> >    - convertible to std::string subject to a macro condition
> > (aka. taint mode)
> >    - allows easy case-insensitive comparison (possibly using
> > Boost.String_algo? Or the netlib transform meta-functions?)
> >    - allows compile-time checking of its source (eg.
> > env/GET/POST) to protect against XSS vulnerabilities.
> >    - allows easy lexical casting (eg. param::as<>)
> >
>
> I don't really understand what you mean by this, but if you mean 'param'
> in the query string parameter case, then this may be a good idea. I
> don't see though if using Boost.Any might make more sense for some
> people who want the dynamic-ness (if there's such a thing) of the value.


I was referring to a possible container for any/all parsed request data. The
idea is that data can be read directly into a basic_message<> and when it's
parsed, the name/values of each 'request variable' is stored in a container
that just *points* to the data, without copying it.

The  `param` (or `basic_param<>`) class would be little more than a
cheap-to-copy drop-in replacement for std::string that supplied minimal
networking-related member functions (eg. a case-insensitive operator==() and
nested typedefs like ::source_type).

I just wrote a load about how it might work, but it might be better if I
threw together an implementation, rather than rambling on about what it
could do!

[snip]

I wasn't too sure about the as<> member function myself either, but several
people requested it and the idea already has precedence in boost libraries,
although I forget which ones right now. Perhaps it would be better to have
an 'as<>-less' network::basic_param<> and then a cgi::basic_param<> which
derives from the former and adds the as<> member function..?

What do you think? *

> * add prepare/consume/commit member functions to
> > network::basic_message<>
> >   - functions like in asio::streambuf.
> >   - would allow reading data straight from a
> > (asio)(A)SyncReadStream (eg. a socket), without need for buffering
> >   - could help with incremental parsing and/or making
> > form_parser class request-agnostic
> >
>
> My suggestion instead of overloading the interface of basic_message<> is
> to define adaptors.


Ahh, interesting...

It should be feasible to have a
> 'streamed<basic_message<> >' or 'async<basic_message<> >' instance which
> would decorate the functionality of the basic_message encapsulated
> within the instance. That way it can still be converted into and is
> still interface compatible to basic_message<>, but adds functionality
> that's typically not inherent in the actual message.


... and supplies reusable components too. :)

streamed<basic_message<http::message_tag> > streamed_message;
> std::string body_line;
> getline(body(body_stream), body_line); // synchronously wait for the
> first body line to be streamed
>
> async<basic_message<http::message_tag> > async_message;
> future<std::string> body_line = body(async_message);
> // at some point in time
> cout << body_line.get();
>
> This keeps the basic_message<> interface intact and the usage semantics
> consistent. What do you think? :D


This sounds like a good idea. For a readable basic_message<> - ie. with the
only addition being member functions prepare/commit/consume - how about
storage<basic_message<>> ? This would allow direct reading/writing to/from
the message with Asio.

> P.P.S.
> >
> > The basic_message<> class has embedded _source/_destination
> > parameters. These aren't particularly relevant to
> > CGI/FastCGI/SCGI and they imply a close relation between the
> > payload of an HTTP message and its source/destination, which
> > might not always be a 1:1 relationship. I wonder if they
> > should be moved out of the basic_message<> class to some
> > higher-level abstraction?
>
> This may be a good idea, but it's possible to actually use SFINAE or a
> cgi::message_tag to specialize on the basic_message<>'s internal
> representation while keeping it consistent. For example, you can disable
> using SFINAE the interface to accessing (and presence of) the
> _source/_destination parameters.


Yeah, this is possible, but do you not think the basic_message<> class
should be as minimal as possible? In the HTTP case you could possibly use
something like:

template<>
class basic_message<http::message_tag>
  : basic_message<tags::default_> // this might be wrong
{
  std::string _source;
  std::string _destination;
public:
  std::string& source() const { return _source; }
  std::string& destination() const { return _destination; }
};

We could then build up a message-type 'stack' on top of this for the
HTTP-based protocols and other 'stacks' for any other ones.

What do you think Darren? I hope these make sense! :)


It all sounds very interesting. I wish I had some time to start on it now!

Cheers,
Darren

[*] From early discussion for the CGI library, some people talked about
embedded system support. They seem to prefer very tight controls on memory
allocation, so avoiding copying might make these use-cases easier to
support...