cpp-netlib-devel — Developers Mailing List

Re: [cpp-netlib-devel] authentication and project files

[Stjepan R. <st...@as...>]

Hi Dean, thanks for your response:

On 5/23/07, Dean Michael Berris <mik...@gm...> wrote:
> On 5/22/07, Stjepan Rajko <st...@as...> wrote:
> So if there's a protocol like SMTP where there's a need for a username
> and password, my suggestion would be that there should be a
> "boost::network::smtp::credentials" object which encapsulates that
> information.
>

OK, I like this. :-)  Keeps things consistent.

> > there be a separate layer that takes care of the authentication?  A
> > related issue is how lasting connections will be handled.
> >
>
> As for lasting connections, Boost.Asio already provides stuff that can
> be re-used and simply re-worked and packaged so that they're used in a
> more generic way (instead of having to implement the same keepalive
> implementation on all applications that need this functionality or
> something similar).
>

Sure.  Maybe I'm not getting your answer fully, but what I meant was
keeping track of the state of a long lasting session (you use a
session as an example in another e-mail which I quote below).  For
example, if I open an FTP connection, and the first thing I send is a
'credentials' message, the protocol should log in using the
credentials.  If instead, the first thing I send is a request for a
file, maybe the system should try to log me in as anonymous and then
get the file.  Similarly, if I send a 'credentials' message after
already being logged in (either as anonymous or via some other
credentials), there is a different sequence of actions needed than the
one used when credentials are sent at the beginning of the connection.

So, my question was, how do we keep track of a long lasting session
and its internal state, which determines how exactly different
messages are handled at the moment.  In your example, you have:

> { // open a scope for the smtp connection
>   smtp::session my_persistent_session(smtp::server("mail.google.com",
> 575, boost::network::tls), smtp::credentials("username", "password"));
>   boost::network::message m;
>   m << header("To", "mik...@gm...")
>     << header("From", "so...@th...mewhere.nw")
>     << destination("mik...@gm...")
>     << source('so...@th...mewhere.nw")
>     << header("Subject", "This is a test message.")
>     << body("The quick brown fox jumps over the lazy dog")
>   ;
>   for (unsigned int i = 0; i < 100; ++i) // send 100 times
>     smtp::send_message(my_persistent_session, m);
> } // close the scope for the smtp connection
>

, so this makes things a lot clearer for me (although, I think I would
prefer my_persistent_session.send_message(m);  since the session knows
what protocol to use.)  To add, I think it would be important to have
a set of generic templates for session classes, both to ease our own
development of many protocols, and to allow users to easily conjure up
their own protocols.  Something that covers cases like "authentication
must come at the beginning, then every other message is OK".  Perhaps
a generic session can be modeled as a map (state, message) -> handler.

> > Also, I've been working on the template-based generator for
> > boost/sandbox projects - once we've checked that it actually complies
> > with Boost guidelines, I can generate some IDE projects (for msvc and
> > xcode) and fill-in-the-blanks docs / doc-build-files for this project.
> >  Any problems with me adding that when it's ready?
> >
>
> I don't know what you mean, but when we do have the code to build in
> different platforms (I'm guessing we won't because I intend to keep
> this library "header only") then things like that would help. For now
> though, I'd rather stick with Boost.Build + Boost.Test for the
> testing, maybe some Quickbook + doxygen for the documentation, among
> other things.

The IDEs generated actually build everything (lib if applicable,
tests, examples) using bjam.  They just provide the utility of an IDE,
which I like.  If other people also use IDEs it might be useful to
have the IDE projects in the subversion, so everyone doesn't have to
make their own.

I can certainly add the build files for a qb+doxy documentation and a
template for docs themselves... soon :-)

Cheers,

Stjepan

Re: [cpp-netlib-devel] authentication and project files

[Dean M. B. <mik...@gm...>]

Hi Stjepan!

On 5/22/07, Stjepan Rajko <st...@as...> wrote:
> Hi all,
>
> I just took a quick glance at the code, and am wondering how
> authentication will be addressed in the library (since that's my main
> interest :-))
>

The short answer would be, that it should be handled on a per-protocol
basis. We can definitely use the boost::network::message to contain
the necessary information to be dealt with by the protocol
implementation.

So if there's a protocol like SMTP where there's a need for a username
and password, my suggestion would be that there should be a
"boost::network::smtp::credentials" object which encapsulates that
information.

> For HTTP and SMTP this might not be an issue, but looking ahead what
> are the thoughts on this?  Will the user just put together a message
> which contains the necessary information in the headers / body, and
> send it off through an appropriate authentication object, or will
> there be a separate layer that takes care of the authentication?  A
> related issue is how lasting connections will be handled.
>

As for lasting connections, Boost.Asio already provides stuff that can
be re-used and simply re-worked and packaged so that they're used in a
more generic way (instead of having to implement the same keepalive
implementation on all applications that need this functionality or
something similar).

> Also, I've been working on the template-based generator for
> boost/sandbox projects - once we've checked that it actually complies
> with Boost guidelines, I can generate some IDE projects (for msvc and
> xcode) and fill-in-the-blanks docs / doc-build-files for this project.
>  Any problems with me adding that when it's ready?
>

I don't know what you mean, but when we do have the code to build in
different platforms (I'm guessing we won't because I intend to keep
this library "header only") then things like that would help. For now
though, I'd rather stick with Boost.Build + Boost.Test for the
testing, maybe some Quickbook + doxygen for the documentation, among
other things.

We'll see if we can use the stuff you're working on too -- if it can
help us write better docs quickly, then I guess we should definitely
go for it. :)

Thanks much! :)

-- 
Dean Michael C. Berris
http://cplusplus-soup.blogspot.com/
mikhailberis AT gmail DOT com
+63 928 7291459

Re: [cpp-netlib-devel] Insights to the Message Class and Network Routines

[Dean M. B. <mik...@gm...>]

Hi Glyn!

On 5/23/07, Glyn Matthews <gly...@gm...> wrote:
>
> On 23 May 2007 00:49:24 +0200, Peter Simons < si...@cr...> wrote:
> > > Is it your intention to use this for boost.network too?
> >
> > No. My preference would be that logging is the responsibility of
> > the users of this library. With the exception of debugging, I
> > wouldn't know what kind of log messages a networking library
> > would need to write. Providing functions that pretty-print data
> > structures into strings, say an Apache-style access log entry, is
> > one thing, but actually writing these strings to some output
> > device is another thing. How to do that is a problem ::main()
> > should solve.
> >
> > Does that sound reasonable?
>
>
>  Yes.  I think more generally its important at this stage to determine
> clearly our policies w.r.t. things like logging, thread-safety, security
> because we have to ensure that we don't hinder boost.network users from
> creating what Dean described as "Custom Web Services".  Someone is always
> going to have, for example, complex logging requirements.  Do we just
> provide a minimal system and say "X, Y and Z is up to you" or is it possible
> to anticipate as far as possible what library users might need?  I'd prefer,
> if possible, the former and I think everyone agrees with this, but it does
> require that we be careful.
>

I agree with providing a minimal yet comprehensive network library
completely. Logging is not within the realm of network operations --
and I have absolutely no intention of getting into (say, even near)
the Boost.Log discussion, which I honestly think is going nowhere.
Let's let the users choose how they want to log things and just throw
meaningful exceptions when there are exceptional conditions met in the
code. :)

> Forgive the sometimes basic questions, I am not as experienced with C++
> networking as other people on this list and I'm trying to understand how
> this will work conceptually.
>

I don't know about the others, but I certainly appreciate well asked
questions. :) I don't necessarily love answering, but I do love
questions nonetheless. :D

So please, if you have questions, don't hesitate to ask away. :-)

-- 
Dean Michael C. Berris
http://cplusplus-soup.blogspot.com/
mikhailberis AT gmail DOT com
+63 928 7291459

Re: [cpp-netlib-devel] Insights to the Message Class and Network Routines

[Dean M. B. <mik...@gm...>]

Hi Peter!

On 23 May 2007 00:32:20 +0200, Peter Simons <si...@cr...> wrote:
>
>  > [The idea for a message class is that it can be used like so:]
>  >
>  > class my_handler  {
>  >   void handle(boost::network::message & msg) {
>  >     // manipulate the message however i want to
>  >   };
>  > };
>
> I wonder at what times during the HTTP transmission this handler
> function would be invoked? From the server's perspective, the
> relevant state changes occur here:
>

It's too early to tell when, but I'll try and put down my thoughts on
the suggestions you have below. I must admit, I was just thinking of
the simplest case in the example I gave in my other message's
implementation. :)

>  1) Complete HTTP header was received. Callback has a chance to
>     say "error response" without waiting for the body to be
>     transmitted. (Errors generally have "Connection: close" and
>     can be written early in the HTTP dialogue.)
>

This can be done, and implemented as a strategy part of the
http::server<>'s template parameters. A
"policies::header_complete_processor<...>" can be used as such:

namespace http {
  template <class HandlerBase,
    class HeaderCompleteProcessor = policies::header_complete_ignore<void>,
    class BodyCompleteProcessor = policies::body_complete_processor<HandlerBase>
    class StreamingProcessor = policies::streaming_processor<void>
  > struct server
  : forward_to_real_impl<HandlerBase,
    HeaderCompleteProcessor,
    BodyCompleteProcessor,
    StreamingProcessor>::type { };
}; // namespace http

>  2) Body data is received.
>
>     a) Body still incomplete? Then the handler has the chance to
>        do something with the data we already have. If possible,
>        the handler should take ownership of the memory because
>        bodies may be very large ("upload") and the HTTP doesn't
>        want to buffer all that in its entirety.
>
>     b) Body complete? Invoke the handler so that it has the
>        chance to put a response into the output buffer.
>

This behavior can be enabled in as much as policies above provide. It
should be noted that a different handler can be provided for when the
headers are complete, and when the body is already complete -- and yet
another when the body is being processed as the data is streaming in.

These different handlers could be differentiated by providing wrapper
types (for metaprogramming) like suggested above:

policies::header_complete_processor<my_header_processor>
policies::body_complete_processor<my_body_processor>
policies::streaming_processor<my_stream_validator>

These policy classes can either be inherited from (which isn't the way
I'd personally like it used) or encapsulated and used as instance
variables in the http::server type. This now all depends on how the
forwarding type behaves -- it could also be noted that this is
currently still entirely up to us. ;-)

>  3) Output buffer is empty.
>
>     a) Does the handler have more data that needs to be written?
>        Then go back to (3).
>
>     b) The transaction is complete.
>
> Does that make any sense?
>

#3 I don't quite understand (yet). Maybe a couple more hours sleep and
a good re-read would do me good. :)

But I think the callback mechanism is missing a
handle/reference/smart_ptr to the actual socket from which the message
came from. So I guess the handler's signature should look something
like:

struct my_handler {
  void operator() (boost::network::message const & message,
    http::socket_ptr_type socket) {
    // do whatever needs to be done
  }
};

> What is the situation regarding protocols other than HTTP? SMTP,
> for instance, has no real use for the boost::network::message
> class because the payload is opaque for the server. There even is
> an ESMTP extension CHUNKING (a.k.a "Binary MIME") that treats the
> payload as an 8-bit octet stream without further structure. An
> e-mail can be represented as a boost::network::message, but an
> SMTP server doesn't really it.
>

SMTP servers, I am still a bit hazy on the implementation. But what I
think is important (as well as in the case of HTTP) is to look at SMTP
processing from the client perspective.

The SMTP server can, and like as you suggest (and I think should),
create a boost::network::message which is a representation of the SMTP
session -- and have the body contain a serialized
boost::network::message still encapsulating the email message and/or
the attachments or whatnot.

In the same light, an SMTP client implementation would just require
one boost::network::message which represents the SMTP session, where
the body is a serialized boost::network::message. If we intend to send
many messages over a single link, we could enable that with the
following technique:

{ // open a scope for the smtp connection
  smtp::session my_persistent_session(smtp::server("mail.google.com",
575, boost::network::tls), smtp::credentials("username", "password"));
  boost::network::message m;
  m << header("To", "mik...@gm...")
    << header("From", "so...@th...mewhere.nw")
    << destination("mik...@gm...")
    << source('so...@th...mewhere.nw")
    << header("Subject", "This is a test message.")
    << body("The quick brown fox jumps over the lazy dog")
  ;
  for (unsigned int i = 0; i < 100; ++i) // send 100 times
    smtp::send_message(my_persistent_session, m);
} // close the scope for the smtp connection

> Applications that do need 'message', however, would be mail
> readers, news readers, and all kinds of mail-to-something-else
> converters. That is a fairly large market, given what kind of
> devices can deal with e-mail these days.
>

I agree completely, and the library should be able to work on these devices. .:)

>  >   http::server<my_handler> handler_instance;
>  >   boost::thread http_thread(handler_instance);
>  >   http_thread.join();
>
> How would the http::server<T> class mentioned here be used in a
> single-threaded environment? Not every device that knows how to
> speak IP also knows how to do multi-threading. In my humble
> opinion, multi-threading should be an option, but no requirement.
>

I agree. Threading can be "yet another policy", and I tend to err on
the side of caution than anything else.

I'd like to explicitly say that the library is multi-threading -- so
that we support the larger (I think) more dominant target which are
desktop/infrastructure developers -- then provide mechanisms for
single threaded operations.

> How do others feel about this subject?

I'm no expert on this, and would defer this to the community at large
-- those evaluating and using the library. If there's a significant
size of the user base looking for single threaded modes for the
servers/clients, I would be inclined to consider providing single
threaded implementations.

However, I would jump the gun and say that it shouldn't be impossible
to provide single-threaded implementations of the servers/client. :-)

>  >
>  > Chris Kohlhoff has proposed a way of using future<> and promise<>
>  > objects to implement asynchronous DNS lookups using a Singleton Active
>  > Object DNS resolver.
>
> Exactly, a DNS server invokes the proper handler function every
> time a response is received, and because of the stateless nature
> of the DNS protocol, there is no need for having more than one
> resolver per process. What I don't understand is how these
> insights get anyone any closer to implementing the DNS protocol.
>

Remember though that Chris already has a resolver in Boost.Asio... So
I don't think we need to implement the DNS protocol -- unless you're
suggesting that we actually have a dns::server<> template, which I
think really is beyond the scope of the library ;-).

> Neither Adns nor C-Ares care what I/O main loop drives them; both
> libraries are concerned with interpreting DNS datagrams. My
> personal opinion is that it would be an incredible waste of time
> to re-invent that wheel. I noticed, though, that Chris has a
> knack for re-inventing perfectly good wheels every know and then.
> I guess it's actually quite likely that he'll end up
> re-implementing DNS. I'll let myself be surprised.
>

I think you'll like to look at Boost.Asio's name resolver... :D

>
>  >   try {
>  >     timeout_read(socket_ptr, buffer, boost::posix_time::seconds(30));
>  >   } catch (boost::network::read::timeout & e) {
>  >     // the read has timed out.
>  >   };
>
> I don't think that function can exist in a non-blocking I/O
> model. Unless, of course, setjmp() / longjmp() counts as a
> solution. ;-)
>

It's meant to be a "blocking read" (referring to timeout_read(...) ).
For non-blocking I/O, I don't think we need to deviate from how
Boost.Asio does the asynchronous read stuff. :D

>
>  > msg["SOME_HEADER"] = "SOME_CONTENT" might work if the internal
>  > representation of a message is a std::map<...>, I'm not
>  > entirely sure this makes sense for a multimap (like what we're
>  > currently using).
>
> It's difficult to say whether a map or a multi-map is the best
> choice. The problem is that some protocols require a multi-map
> and some protocols don't. An e-mail may have any number of
> "Received:" lines. In HTTP, however, headers are generally
> unique.
>

Actually, the reason I chose a multimap is precisely the reason you
cite: because some protocols need non-unique headers, while some don't
-- and since a multimap can work for either case (where the
transformations/validations will just deal with the details), I chose
to use a multimap for all cases. :-) One of my partners at
http://software.orangeandbronze.com/ and I were talking about this in
another project, where I chose to use a map instead of a multi-map.
Needless to say, his case makes more sense to use a multimap because
it can do pretty much anything a map can and then some. :)

> Anyway, this is a stimulating conversation. It's fun to think and
> talk about these topics.
>

Definitely. :)

> Best regards,
> Peter
>

Thanks for the insights! It definitely helps me make things clearer in
my head, and write down the thoughts (somewhere) albeit in an
"informal" manner. :)

-- 
Dean Michael C. Berris
http://cplusplus-soup.blogspot.com/
mikhailberis AT gmail DOT com
+63 928 7291459

Re: [cpp-netlib-devel] Insights to the Message Class and Network Routines

[Glyn M. <gly...@gm...>]

Peter,

On 23 May 2007 00:49:24 +0200, Peter Simons <si...@cr...> wrote:

> > Is it your intention to use this for boost.network too?
>
> No. My preference would be that logging is the responsibility of
> the users of this library. With the exception of debugging, I
> wouldn't know what kind of log messages a networking library
> would need to write. Providing functions that pretty-print data
> structures into strings, say an Apache-style access log entry, is
> one thing, but actually writing these strings to some output
> device is another thing. How to do that is a problem ::main()
> should solve.
>
> Does that sound reasonable?



 Yes.  I think more generally its important at this stage to determine
clearly our policies w.r.t. things like logging, thread-safety, security
because we have to ensure that we don't hinder boost.network users from
creating what Dean described as "Custom Web Services".  Someone is always
going to have, for example, complex logging requirements.  Do we just
provide a minimal system and say "X, Y and Z is up to you" or is it possible
to anticipate as far as possible what library users might need?  I'd prefer,
if possible, the former and I think everyone agrees with this, but it does
require that we be careful.

Forgive the sometimes basic questions, I am not as experienced with C++
networking as other people on this list and I'm trying to understand how
this will work conceptually.

G

Re: [cpp-netlib-devel] Insights to the Message Class and Network Routines

[Glyn M. <gly...@gm...>]

Guys,

On 23 May 2007 00:32:20 +0200, Peter Simons <si...@cr...> wrote:

>
> How would the http::server<T> class mentioned here be used in a
> single-threaded environment? Not every device that knows how to
> speak IP also knows how to do multi-threading. In my humble
> opinion, multi-threading should be an option, but no requirement.
>
> How do others feel about this subject?


I think it could be possible to implement multi-threading as a policy.  I
certainly agree with Peter that this is optional.


> Anyway, this is a stimulating conversation. It's fun to think and
> talk about these topics.


It certainly is.  I'm getting new insights into network programming.  It
will be interesting to see what we can produce.
G

Re: [cpp-netlib-devel] Insights to the Message Class and Network Routines

[Peter S. <si...@cr...>]

Hi Glyn,

you wondered:

 > I do notice [mini-httpd uses] John Torjo's logging library. I
 > may have missed something but will this be present in a future
 > version of boost?

As far as I know, the original Boost.Log library was rejected
during the review. A new proposal recently showed up on the
developer's mailing list. I don't know much about it, though.

Personally, I access logging functions only through macros in my
programs to make sure the source code doesn't depend on any
concrete library. I hardly ever need sophisticated features. On
Unix systems, all I really need is syslog(3). :-)


 > Is it your intention to use this for boost.network too?

No. My preference would be that logging is the responsibility of
the users of this library. With the exception of debugging, I
wouldn't know what kind of log messages a networking library
would need to write. Providing functions that pretty-print data
structures into strings, say an Apache-style access log entry, is
one thing, but actually writing these strings to some output
device is another thing. How to do that is a problem ::main()
should solve.

Does that sound reasonable?

Best regards,
Peter

Re: [cpp-netlib-devel] Insights to the Message Class and Network Routines

[Peter S. <si...@cr...>]

Hi guys!

 > [The idea for a message class is that it can be used like so:]
 >
 > class my_handler  {
 >   void handle(boost::network::message & msg) {
 >     // manipulate the message however i want to
 >   };
 > };

I wonder at what times during the HTTP transmission this handler
function would be invoked? From the server's perspective, the
relevant state changes occur here:

 1) Complete HTTP header was received. Callback has a chance to
    say "error response" without waiting for the body to be
    transmitted. (Errors generally have "Connection: close" and
    can be written early in the HTTP dialogue.)

 2) Body data is received.

    a) Body still incomplete? Then the handler has the chance to
       do something with the data we already have. If possible,
       the handler should take ownership of the memory because
       bodies may be very large ("upload") and the HTTP doesn't
       want to buffer all that in its entirety.

    b) Body complete? Invoke the handler so that it has the
       chance to put a response into the output buffer.

 3) Output buffer is empty.

    a) Does the handler have more data that needs to be written?
       Then go back to (3).

    b) The transaction is complete.

Does that make any sense?

What is the situation regarding protocols other than HTTP? SMTP,
for instance, has no real use for the boost::network::message
class because the payload is opaque for the server. There even is
an ESMTP extension CHUNKING (a.k.a "Binary MIME") that treats the
payload as an 8-bit octet stream without further structure. An
e-mail can be represented as a boost::network::message, but an
SMTP server doesn't really it.

Applications that do need 'message', however, would be mail
readers, news readers, and all kinds of mail-to-something-else
converters. That is a fairly large market, given what kind of
devices can deal with e-mail these days.

 >   http::server<my_handler> handler_instance;
 >   boost::thread http_thread(handler_instance);
 >   http_thread.join();

How would the http::server<T> class mentioned here be used in a
single-threaded environment? Not every device that knows how to
speak IP also knows how to do multi-threading. In my humble
opinion, multi-threading should be an option, but no requirement.

How do others feel about this subject?


 >> An asynchronous DNS resolver is badly missed. Implementing
 >> the DNS protocol from scratch is, however, a significant
 >> effort. It might be wise to base our efforts on one of the
 >> existing libraries, e.g.:
 >>
 >> http://www.chiark.greenend.org.uk/~ian/adns/
 >> http://daniel.haxx.se/projects/c-ares/
 >
 > Chris Kohlhoff has proposed a way of using future<> and promise<>
 > objects to implement asynchronous DNS lookups using a Singleton Active
 > Object DNS resolver.

Exactly, a DNS server invokes the proper handler function every
time a response is received, and because of the stateless nature
of the DNS protocol, there is no need for having more than one
resolver per process. What I don't understand is how these
insights get anyone any closer to implementing the DNS protocol.

Neither Adns nor C-Ares care what I/O main loop drives them; both
libraries are concerned with interpreting DNS datagrams. My
personal opinion is that it would be an incredible waste of time
to re-invent that wheel. I noticed, though, that Chris has a
knack for re-inventing perfectly good wheels every know and then.
I guess it's actually quite likely that he'll end up
re-implementing DNS. I'll let myself be surprised.


 >   try {
 >     timeout_read(socket_ptr, buffer, boost::posix_time::seconds(30));
 >   } catch (boost::network::read::timeout & e) {
 >     // the read has timed out.
 >   };

I don't think that function can exist in a non-blocking I/O
model. Unless, of course, setjmp() / longjmp() counts as a
solution. ;-)


 > msg["SOME_HEADER"] = "SOME_CONTENT" might work if the internal
 > representation of a message is a std::map<...>, I'm not
 > entirely sure this makes sense for a multimap (like what we're
 > currently using).

It's difficult to say whether a map or a multi-map is the best
choice. The problem is that some protocols require a multi-map
and some protocols don't. An e-mail may have any number of
"Received:" lines. In HTTP, however, headers are generally
unique.

Anyway, this is a stimulating conversation. It's fun to think and
talk about these topics.

Best regards,
Peter

[cpp-netlib-devel] authentication and project files

[Stjepan R. <st...@as...>]

Hi all,

I just took a quick glance at the code, and am wondering how
authentication will be addressed in the library (since that's my main
interest :-))

For HTTP and SMTP this might not be an issue, but looking ahead what
are the thoughts on this?  Will the user just put together a message
which contains the necessary information in the headers / body, and
send it off through an appropriate authentication object, or will
there be a separate layer that takes care of the authentication?  A
related issue is how lasting connections will be handled.

Also, I've been working on the template-based generator for
boost/sandbox projects - once we've checked that it actually complies
with Boost guidelines, I can generate some IDE projects (for msvc and
xcode) and fill-in-the-blanks docs / doc-build-files for this project.
 Any problems with me adding that when it's ready?

Regards,

Stjepan

Re: [cpp-netlib-devel] Insights to the Message Class and Network Routines (was Re: Some code already up in subversion)

[Glyn M. <gly...@gm...>]

Guys,

On 21/05/07, Dean Michael Berris <mik...@gm...> wrote:
>
> Hi Peter!
>
> On 21 May 2007 20:02:02 +0200, Peter Simons <si...@cr...> wrote:
> >
> > No, I think rfc2822 is a perfect name for that hierarchy.

Ah, now it makes more sense. :) I agree, this would be a good
> collection which I think merits itself worthy (I think) of inclusion
> into Boost either as part of Spirit or as a package in itself. :)


FWIW I'm inclined to agree.


> >  > [scope of boost.network]
> >
> > Alright, thank you for sharing your ideas. From what I can
> > tell, the things we ought to develop in this project are
> > general purpose HTTP and SMTP drivers that can be re-used in
> > clients and servers. I have implemented a (very rough) HTTP
> > server here:
> >
> >   http://git.cryp.to/?p=mini-httpd
> >


I've taken a brief look at the code, but I'll have to look at it in more
depth to make more comments.  But I do notice you're using John Torjo's
logging library.  I may have missed something but will this be present in a
future version of boost?  Is it your intention to use this for boost.networktoo?

<snip code>
>
> So the goal would be to allow people to make "Custom HTTP Services"
> without having to worry too much about the details of an HTTP Server
> implementation (which of course under the hood requires Boost.Asio).


That's an admirable goal.

I'd take a look when I get the time... For the week I'm being taken
> over by the day job... So I doubt I can do that within this week.
>
> Maybe others can give Peter a hand at evaluating/profiling/testing the
> daemon? :-)


Yes I intend to, but I will be rather busy this week and i'll be abroad over
the weekend so I can't promise too much at the moment.

Regards,
Glyn

[cpp-netlib-devel] Insights to the Message Class and Network Routines (was Re: Some code already up in subversion)

[Dean M. B. <mik...@gm...>]

Hi Peter!

On 21 May 2007 20:02:02 +0200, Peter Simons <si...@cr...> wrote:
>
> No, I think rfc2822 is a perfect name for that hierarchy. My
> problem is with the "network" part, because the code is about
> text parsing, not about networks. The location I'd prefer
> would be more along the lines of "boost/spirit/rfc2822",
> "boost/parsers/rfc2822", or even "/boost/ietf/rfc/2822/".
> Whatever. The point is that putting this code into a network
> hierarchy doesn't feel right, which is why I suggested a
> different organization that offers more granularity.
>

Ah, now it makes more sense. :) I agree, this would be a good
collection which I think merits itself worthy (I think) of inclusion
into Boost either as part of Spirit or as a package in itself. :)

> I guess there would be code inside of "boost/network" that
> would depend on those parsers, but the parsers don't depend on
> "boost/network". So I feel they shouldn't necessarily be a
> part of it.
>

I agree completely. :)

>
>  > [scope of boost.network]
>
> Alright, thank you for sharing your ideas. From what I can
> tell, the things we ought to develop in this project are
> general purpose HTTP and SMTP drivers that can be re-used in
> clients and servers. I have implemented a (very rough) HTTP
> server here:
>
>   http://git.cryp.to/?p=mini-httpd
>

Yes, you're right. :)

I'll go take a look in a while, but basically the idea I'm playing
around with in my head is something to the effect of having a class
like:

namespace http {
template <
  class Base,
  class ProcessingPolicy=policies::async_processing,
  class ThreadingPolicy=policies::default_threading
>
struct server : Base, ProcessingPolicy, ThreadingPolicy {
  // all bolerplate, and basic HTTP server specific code
  private:

  // ... require a handle function from the parent class
  // which processes a boost::network::message
  void _call_handle(boost::network::message message_) {
    ThreadingPolicy::scoped_lock _(*this);
    Base::handle(message_);
  };

};

}; // namespace http

Which would be used like so:

class my_handler  {
  void handle(boost::network::message const & msg) {
    // manipulate the message however i want to
  };
};

{
  http::server<my_handler> handler_instance;
  boost::thread http_thread(handler_instance);
  http_thread.join();
}

So the goal would be to allow people to make "Custom HTTP Services"
without having to worry too much about the details of an HTTP Server
implementation (which of course under the hood requires Boost.Asio).

> I would very much appreciate feedback concerning that daemon,
> by the way. The latest "snapshot" comes with a
> Autoconf/Automake build system that should -- in theory --
> work on most Unix machines. My understanding is, however, that
> there is at least one more project (libhttpd?) that tries to
> do the same thing.
>

I'd take a look when I get the time... For the week I'm being taken
over by the day job... So I doubt I can do that within this week.

Maybe others can give Peter a hand at evaluating/profiling/testing the
daemon? :-)

> Concerning SMTP, I have no working implementation, but I do
> have a strong interest in the subject:
>
>   http://postmaster.cryp.to/
>
> That daemon is written in Haskell. That was great fun, but I'd
> like to turn that design into working C++ code, eventually.
>

Nice! Pretty much the above sample is what I'd use to do something
like this in C++ (using the Parametric Base Class Pattern or PBCP, as
how Joel de Guzman puts it in the documentation of Phoenix actors).

>
>  > Components we anticipate are (aside from the message class):
>  >
>  >   - asynchronous name resolver
>
> Yes, an asynchronous DNS resolver is badly missed.
> Implementing the DNS protocol from scratch is, however, a
> significant effort. It might be wise to base our efforts on
> one of the existing libraries, e.g.:
>
>   http://www.chiark.greenend.org.uk/~ian/adns/
>   http://daniel.haxx.se/projects/c-ares/
>

Chris Kohlhoff has proposed a way of using future<> and promise<>
objects to implement asynchronous DNS lookups using a Singleton Active
Object DNS resolver -- of course, using Boost.Asio still under the
hood. So I intend to use that as the (simple) implementation of the
asynchronous DNS resolver.

>
>  >   - "synchronous timeout reader/writer"
>
> I'm not quite sure what your description refers to. I've found
> that covenience suggests another layer on top of Asio in order
> to further separation of application code and i/o code. My
> best bet into that direction so far is visible here:
>
>   http://git.cryp.to/?p=mini-httpd;a=blob;f=io.hpp;hb=master
>   http://git.cryp.to/?p=mini-httpd;a=blob;f=io-driver.cpp;hb=master
>

The idea is to be able to do a:

  try {
    timeout_read(socket_ptr, buffer, boost::posix_time::seconds(30));
  } catch (boost::network::read::timeout & e) {
    // the read has timed out.
  };

Which is sorely missed in Boost.Asio. I've implemented something
similar based on Chris Kohlhoff's advice as well, which I intend to
make part of the library. :)

>
> Concerning the message class, I have to admit to being
> reserved about the use of operator<<() to construct headers.
> In my humble opinion, the statement
>
>     msg << header("SOME_HEADER", "SOME_CONTENT") ;
>
> should rather read:
>
>     msg["SOME_HEADER"]  = "SOME_CONTENT";
>     msg["SOME_HEADER"] += "SOME_MORE_CONTENT";
>

It's pretty trivial to make your suggestion above happen, and the only
reason I use operator<<() is that it allows me to "pass a directive to
a message". In the case of assignment of values as you have described
above, this does make sense and I agree we should support that usage
as well. However, the particular use case below makes it possible to
do pretty funky things with message construction:

  msg << header("SOME_HEADER", "SOME_CONTENT")
    << transform(match_header("SOME_HEADER"),
            boost::bind(boost::to_lower(_1))) // using bind
    << mutate(body(), arg1 = "Hello, World!") // using phoenix as well
    ;

So yes, though msg["SOME_HEADER"] = "SOME_CONTENT"; might work if the
internal representation of a message is a std::map<...>, I'm not
entirely sure this makes sense for a multimap (like what we're
currently using).

The current recommended method of accessing the headers of a message
are as described in the test case:

  headers_range<message>::type range = headers(msg)["SOME_HEADER"];

which is why I'm apprehensive with allowing msg["HEADER"] -- because
it doesn't convey that you're manipulating/accessing the headers of
the message, rather attributes of the message. Besides, msg["HEADER"]
doesn't make sense for a multimap as described above. :)

> Use of operator<<() for constructing bodys is fine though. One
> thing I'm not quite sure about yet is whether that message API
> holds up to incremental construction and consumption, e.g. in
> a non-blocking I/O model.
>

Right... But this is all keeping in mind that we'll be using the
construction routines in a spirit action:

  body_parser_p[ var(msg) << body(construct<std::string>(arg1, arg2))
]; // using phoenix

I'm not sure though about how asynchronous construction will be
affected though... In the traditional Boost.Asio asynchronous classes,
you typically have handlers which are scheduled/called asynchronously.
That means if you have a method "handle_1" and "handle_2",
constructing a message asynchronously based on network protocols would
be trivial:

  void handle_1(std::string some_string, message & m) const {
    m << header("1", some_string);
  }

  void handle_2(std::string some_string, message & m) const {
    m << header("2", some_string);
  }

Now locking would have to be involved somewhere here, which can be
taken care of by boost::asio::strand's for handlers that perform
message manipulations.

> I also feel that usage of the type
>
>   std::basic_string<char, std::char_traits<char>, std::allocator>
>
> shouldn't be hard-coded in messages. I guess it's fine to
> commit to 'char' as a character type (although unnecessary),
> but people will definitely want to choose their own
> allocators. Specifying  different trait classes per message
> type might also be useful. RFC headers typically are
> case-insensitive, for example. The trait class attached below
> might be useful in that context.
>

Yup, the next stages of refactoring should address this. I guess it
should be template parameters to the basic_message class, where
reasonable defaults are set. That being said, the other
types/directives will have to support the type change to the message
class. :D

> Best regards,
> Peter
>

Thank you very much for the insights! They definitely open my eyes
wide to other issues I've tend to overlook. :)

-- 
Dean Michael C. Berris
http://cplusplus-soup.blogspot.com/
mikhailberis AT gmail DOT com
+63 928 7291459

Re: [cpp-netlib-devel] Some code already up in subversion

[Christian H. <chh...@gm...>]

On 21 May 2007 20:02:02 +0200, Peter Simons <si...@cr...> wrote:
> Hi Dean,
>
>  > I agree that boost/network/rfc2822 doesn't feel natural.
>  > Would you have a different name from rfc2822 which might
>  > make more sense and be more descriptive?
>
> No, I think rfc2822 is a perfect name for that hierarchy. My
> problem is with the "network" part, because the code is about
> text parsing, not about networks. The location I'd prefer
> would be more along the lines of "boost/spirit/rfc2822",
> "boost/parsers/rfc2822", or even "/boost/ietf/rfc/2822/".
> Whatever. The point is that putting this code into a network
> hierarchy doesn't feel right, which is why I suggested a
> different organization that offers more granularity.

For the time being we can just add them to "/boost/ietf/rfc/2822/". I
think that would make to most sense. More important is to get the
software design right.

>  > [scope of boost.network]
>
> Alright, thank you for sharing your ideas. From what I can
> tell, the things we ought to develop in this project are
> general purpose HTTP and SMTP drivers that can be re-used in
> clients and servers. I have implemented a (very rough) HTTP
> server here:
>
>   http://git.cryp.to/?p=mini-httpd
>
> I would very much appreciate feedback concerning that daemon,
> by the way. The latest "snapshot" comes with a
> Autoconf/Automake build system that should -- in theory --
> work on most Unix machines. My understanding is, however, that
> there is at least one more project (libhttpd?) that tries to
> do the same thing.

Another very basic implementation is provided by asio.

http://asio.sourceforge.net/boost_asio_0_3_7/libs/asio/doc/examples/index.html

But there is also a next version in the cvs trunk:

http://asio.cvs.sourceforge.net/asio/asio/src/examples/http/server2/

How would you guess to envision the handling of the status codes and
the request methods like GET, POST, etc. For the status codes I would
not vote for enums. I think small structs should encode them.

Christian

Re: [cpp-netlib-devel] Some code already up in subversion

[Dean M. B. <mik...@gm...>]

Hi Christian!

On 5/21/07, Christian Henning <chh...@gm...> wrote:
>
> Why not have the parser in boost/network/parsers/. Since some parsers
> are used in more than one protocol I don't think we should order them
> based on protocols. I think we should split them as much as we can,
> like in the rfc2822 lib. Although here I would change the namespace.
>

Good idea. :) This sounds good to me. :)

-- 
Dean Michael C. Berris
http://cplusplus-soup.blogspot.com/
mikhailberis AT gmail DOT com
+63 928 7291459

Re: [cpp-netlib-devel] Some code already up in subversion

[Christian H. <chh...@gm...>]

Hi guys,

>
> I agree that boost/network/rfc2822 doesn't feel natural. Would you
> have a different name from rfc2822 which might make more sense and be
> more descriptive? It might also even make sense that we can put it in
> boost/network/smtp/parser/ and have a boost/network/smtp/parsers.hpp
> file (just in case I understand that the parsers are for SMTP
> parsing).
>

Why not have the parser in boost/network/parsers/. Since some parsers
are used in more than one protocol I don't think we should order them
based on protocols. I think we should split them as much as we can,
like in the rfc2822 lib. Although here I would change the namespace.

Christian

Re: [cpp-netlib-devel] Some code already up in subversion

[Dean M. B. <mik...@gm...>]

Hi Peter!

On 21 May 2007 06:22:27 +0200, Peter Simons <si...@cr...> wrote:
>
> Yes, for an Uri class that's perfectly reasonable, but for other
> classes things are not that obvious. The rfc2822 parsers, for
> example, are, well, parsers. They will typically be applied to
> data that comes from the network, but the need to parse an e-mail
> address can just as well arise in a spreadsheet program. Adding
> those parsers into a boost/network/rfc2822/ hierarchy does make
> sense to a degree, but it doesn't feel like the natural choice --
> particularly considering the fact that those parsers will never
> depend on anything else in the network hierarchy.
>

I agree that boost/network/rfc2822 doesn't feel natural. Would you
have a different name from rfc2822 which might make more sense and be
more descriptive? It might also even make sense that we can put it in
boost/network/smtp/parser/ and have a boost/network/smtp/parsers.hpp
file (just in case I understand that the parsers are for SMTP
parsing).

> I still can't quite imagine what the scope of this whole endeavor
> is. Right now, I have a hard time seeing how my code fits in.
>

Okay, let me try and do a bird's eye view:

We have a message class, which every network protocol implementation
will use (meaning, create/return/manipulate/process). Of the network
protocol implementations, there will be parsers which will accept
string input and validate then generate message objects. These parsers
will generally be based on Boost.Spirit.

That being said, protocol implementations for SMTP processing will
definitely be used to generate message objects which the SMTP handlers
will use to perform the necessary actions. For example, for mail
sending the parsers will have to parse string responses from the SMTP
servers. Another example is for SMTP servers where incoming text
messages will be processed with a parser to generate the appropriate
message object, which then can be passed on to application specific
handlers. A POP client will also benefit from parsers to create
message objects on a per-email basis.

I hope the above descriptions/narrative makes things clearer... I
admit am having a hard time writing down these things that are pretty
much still all in my head. I will do what I can though to be able to
document all this before and during the development process.

>
>  >> What are the components we anticipate to have in cpp-netlib?
>  >
>  > Definitely a URI class would be a good idea.
>
> Right, an URI class would be great to have.
>
> Are there any other components we anticipate or would like to
> implement? What is the goal for cpp-netlib? When it's all done
> and accepted into boost, what does it contain?
>

One question at a time. ;-)

Components we anticipate are (aside from the message class):

  - asynchronous name resolver
  - "synchronous timeout reader/writer"
  - protocol implementations (of course)
  - server templates (for PBCP and CRTP servers)

The goal for cpp-netlib is to:

 * provide a high-quality, generic, extensible collection of
network-related routines and popular protocol implementations using
the C++ programming language
 * become the Boost.Network library
 * enable generic network protocol implementations
 * lower the barrier to entry for network-aware C++ applications

When it's gotten into Boost, it should contain at least the following:

 * Common message type
 * Transformations/Manipulations for the common message type instances
 * Minimal HTTP implementation

If we can even make an SMTP implementation using the parsers you've
written, then we could include that as well in the list above.

> I'm sorry for bringing up those very general questions instead of
> committing code, but it feels important to develop some sort of
> shared vision. I have my code in a publicly available repository
> already; so it doesn't feel like I'm gaining a lot by checking it
> into this repository instead. I would do so because I'd hope that
> the code might become part of a team effort. At the moment, I
> can't quite say where exactly that team effort is headed. It
> would be nice to develop that notion.
>

I'm open to suggestions as well. :) I'm also thankful that you bring
up these questions early, so that we can go discuss the direction at
this point, then be able to plan for the concrete actions that we'd
like to take.

I'd really like to know your thoughts on the things I've written above.

I hope this makes things clear to some extent. Please ask away and
suggest away about any matter regarding the development process and or
direction of the project.

(I'm pretty new at this open source development thing as well, and I
do have some trouble writing down ideas so please bear with me on
this... I'll try my best to articulate in terms of
English/HumanLanguage what I think -- and if I can't I'll write code
instead. ;) ).

> Best regards,
> Peter
>

Thanks again Peter!

-- 
Dean Michael C. Berris
http://cplusplus-soup.blogspot.com/
mikhailberis AT gmail DOT com
+63 928 7291459

Re: [cpp-netlib-devel] Some code already up in subversion

[Peter S. <si...@cr...>]

Hi Dean,

 > A suggestion would be to have that class under the
 > boost/network/ directory. This would yield something like a
 > 'boost/network/uri.hpp' header which can be used on its own.

Yes, for an Uri class that's perfectly reasonable, but for other
classes things are not that obvious. The rfc2822 parsers, for
example, are, well, parsers. They will typically be applied to
data that comes from the network, but the need to parse an e-mail
address can just as well arise in a spreadsheet program. Adding
those parsers into a boost/network/rfc2822/ hierarchy does make
sense to a degree, but it doesn't feel like the natural choice --
particularly considering the fact that those parsers will never
depend on anything else in the network hierarchy.

I still can't quite imagine what the scope of this whole endeavor
is. Right now, I have a hard time seeing how my code fits in.


 >> What are the components we anticipate to have in cpp-netlib?
 >
 > Definitely a URI class would be a good idea.

Right, an URI class would be great to have.

Are there any other components we anticipate or would like to
implement? What is the goal for cpp-netlib? When it's all done
and accepted into boost, what does it contain?

I'm sorry for bringing up those very general questions instead of
committing code, but it feels important to develop some sort of
shared vision. I have my code in a publicly available repository
already; so it doesn't feel like I'm gaining a lot by checking it
into this repository instead. I would do so because I'd hope that
the code might become part of a team effort. At the moment, I
can't quite say where exactly that team effort is headed. It
would be nice to develop that notion.

Best regards,
Peter

Re: [cpp-netlib-devel] Some code already up in subversion

[Dean M. B. <mik...@gm...>]

Hi Christian!

On 5/20/07, Christian Henning <chh...@gm...> wrote:
>
> Hi Dean, I have successfully compiled and run the code using VC7.1 .
> I'm using boost 1.34. I tried boost 1.33.1 but then the test code
> doesn't compile. So, I think we should make a statement of using 1.34.


I actually failed to mention that I've been using Boost CVS HEAD all this
time for testing. I think the problems we'll be encountering with 1.33.1 is
the interface changes to Boost.Test which are not backward compatible. The
version of Boost.Test in CVS is much closer to 1.34 which explains why it
would build with that release.

The recommendation would be that we should use Boost CVS HEAD all the time
while developing and try to make the library/test build when using Boost
1.34.

One reason for me to join this effort was to learn more about
> networking application. So, please excuse my obvious lack of
> knowledge.


No problem, we're all in it for the learning as well I think. ;-)

I was taking a look at asio's http example. How would you
> construct your network::message to generate a client request. In this
> example the client does the following:
>
>     std::ostream request_stream(&request_);
>     request_stream << "GET " << path << " HTTP/1.0\r\n";
>     request_stream << "Host: " << server << "\r\n";
>     request_stream << "Accept: */*\r\n";
>     request_stream << "Connection: close\r\n\r\n";
>
> as an example the server might be "www.boost.org" and the path is
> "/LICENSE_1_0.txt".


The recommended usage is something in this sort:

using namespace boost::network ;
message request ;
request << header("GET", std::string(path) + " HTTP/1.0")
    << header("Host", server)
    << header("Accept", "*/*")
    << header("Connection", "close")
    << body("")
;

Which can then be encapsulated in (when we already have it) an http request
object like such:

using namespace boost::network;
http::request boost_http_request(request, http::url("
http://boost.org/LICENSE_1_0.txt"));
copy (istream_iterator<std::string>(boost_http_request),
    istream_iterator<std::string>(),
    ostream_iterator<std::string>(cout, "\n"));

So what would be the source identifier, destination identifier,
> header, and body?


The source can be ignored, since http requests don't really need a "source",
and the destination is usually just part of the message headers. Considering
that the message is generic, it just holds the information which is then
converted/dealt with appropriately by the 'transformer' and 'protocol
implementation' -- in the above case an http::request is a network specific
translator.

An even simpler example (which wouldn't use a boost::network::message) would
be the following:

using namespace boost::network::http;
request boost_website(url("http://boost.org/LICENSE_1_0.txt"),
method("GET"), protocol::version<1.1>());
cout << boost_website;

:-)

Regards,
> Christian


I certainly hope the explanation above makes sense. :)

If not, we can discuss this further. :)

-- 
Dean Michael C. Berris
http://cplusplus-soup.blogspot.com/
mikhailberis AT gmail DOT com
+63 928 7291459

Re: [cpp-netlib-devel] Some code already up in subversion

[Dean M. B. <mik...@gm...>]

Hi Peter!

On 20 May 2007 21:27:15 +0200, Peter Simons <si...@cr...> wrote:
> Hi guys,
>
> it took a while to remember my sf.net password, but in the end I
> managed to commit. :-)
>

Great! I'll check out your changes soon as I can. :)

> I have a question. In the boost-sandbox repository, everyone is
> encouraged to add a top-level directory per project. An imaginary
> foo library is organized in SVN as follows:
>
>   foo / boost / foo.hpp
>   foo / boost / foo / ...
>   foo / libs  / foo / src  / ...
>   foo / libs  / foo / doc  / ...
>   foo / libs  / foo / test / ...
>
> How would we like to handle that? What is the granularity we aim
> for in this project? Let's say we come up with a very good class
> for representing URIs. Do we try to make that URI library a
> stand-alone component (that can be submitted to boost on its
> own), or is URI something that's tightly integrated into
> cpp-netlib?
>

A suggestion would be to have that class under the boost/network/
directory. This would yield something like a 'boost/network/uri.hpp'
header which can be used on its own.

cpp-netlib supposedly follows the above structure, replacing 'trunk'
with something like 'network' so the structure looks like

    network / boost / network
    network / libs / network

> Tight integration often leads to more efficient source code
> because chances for re-use are higher. If things are integrated
> too tightly, however, it is hard to use any one component of the
> library without using everything else too.
>

Agreed.

> What are the components we anticipate to have in cpp-netlib? A
> general purpose class for things that look like an "Internet
> Message" is one of topics we need to address, but what else would
> we like to see in Boost?
>

Definitely a URI class would be a good idea, preferrably something
that is convertible to a std::string by default. We can even use the
same approach we have for creating message objects with directives and
operator overloads to make the usage be a bit more expressive.

> Best regards,
> Peter
>

Thanks Peter!

I hope the above explanation makes sense. :)

-- 
Dean Michael C. Berris
http://cplusplus-soup.blogspot.com/
mikhailberis AT gmail DOT com
+63 928 7291459

[cpp-netlib-devel] Committing Changes (Was Re: Some code already up in subversion)

[Dean M. B. <mik...@gm...>]

Hi Glyn!

On 5/20/07, Glyn Matthews <gly...@gm...> wrote:
>
> On 20/05/07, Dean Michael Berris <mik...@gm...> wrote:
> > Hi Glyn!
> >
> > On 5/20/07, Glyn Matthews <gly...@gm...> wrote:
> >
> > Please update any changes as long as the tests are green. Again, if
> > we're going to be introducing new functionality/code, we need a unit
> > test for it first. :D
>
> Do I need a username/password to commit changes?

Yes, you need to use your sourceforge username and password to commit. :)

There's documentation on the sourceforge site on how to use their
subversion services. :)

-- 
Dean Michael C. Berris
http://cplusplus-soup.blogspot.com/
mikhailberis AT gmail DOT com
+63 928 7291459

Re: [cpp-netlib-devel] Some code already up in subversion

[Christian H. <chh...@gm...>]

Hi Dean, I have successfully compiled and run the code using VC7.1 .
I'm using boost 1.34. I tried boost 1.33.1 but then the test code
doesn't compile. So, I think we should make a statement of using 1.34.

One reason for me to join this effort was to learn more about
networking application. So, please excuse my obvious lack of
knowledge. I was taking a look at asio's http example. How would you
construct your network::message to generate a client request. In this
example the client does the following:

    std::ostream request_stream(&request_);
    request_stream << "GET " << path << " HTTP/1.0\r\n";
    request_stream << "Host: " << server << "\r\n";
    request_stream << "Accept: */*\r\n";
    request_stream << "Connection: close\r\n\r\n";

as an example the server might be "www.boost.org" and the path is
"/LICENSE_1_0.txt".

So what would be the source identifier, destination identifier,
header, and body?

Regards,
Christian

On 5/20/07, Dean Michael Berris <mik...@gm...> wrote:
> Hi Everyone!
>
> If and when you have the time, I've been working on the message type
> and I've got code already up in the subversion repository.
>
> I'm developing this using the Test Driven Development approach, where
> I write the tests first (in libs/network/test) then make the tests
> "run green" by adding new functionality and/or refactoring.
>
> I've tried to put as much documentation into the code as I can, and if
> there are some things not clear form the get go please feel free to
> email questions/suggestions/clarifications to the list so we can
> discuss it.
>
> Thanks in advance!
>
> --
> Dean Michael C. Berris
> http://cplusplus-soup.blogspot.com/
> mikhailberis AT gmail DOT com
> +63 928 7291459
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> _______________________________________________
> Cpp-netlib-devel mailing list
> Cpp...@li...
> https://lists.sourceforge.net/lists/listinfo/cpp-netlib-devel
>

Re: [cpp-netlib-devel] Some code already up in subversion

[Peter S. <si...@cr...>]

Hi guys, 

it took a while to remember my sf.net password, but in the end I
managed to commit. :-)

I have a question. In the boost-sandbox repository, everyone is
encouraged to add a top-level directory per project. An imaginary
foo library is organized in SVN as follows:

  foo / boost / foo.hpp
  foo / boost / foo / ...
  foo / libs  / foo / src  / ...
  foo / libs  / foo / doc  / ...
  foo / libs  / foo / test / ...

How would we like to handle that? What is the granularity we aim
for in this project? Let's say we come up with a very good class
for representing URIs. Do we try to make that URI library a
stand-alone component (that can be submitted to boost on its
own), or is URI something that's tightly integrated into
cpp-netlib?

Tight integration often leads to more efficient source code
because chances for re-use are higher. If things are integrated
too tightly, however, it is hard to use any one component of the
library without using everything else too. 

What are the components we anticipate to have in cpp-netlib? A
general purpose class for things that look like an "Internet
Message" is one of topics we need to address, but what else would
we like to see in Boost?

Best regards,
Peter

Re: [cpp-netlib-devel] Some code already up in subversion

[Glyn M. <gly...@gm...>]

On 20/05/07, Dean Michael Berris <mik...@gm...> wrote:
>
> Hi Glyn!
>
> On 5/20/07, Glyn Matthews <gly...@gm...> wrote:
>
> Please update any changes as long as the tests are green. Again, if
> we're going to be introducing new functionality/code, we need a unit
> test for it first. :D


Do I need a username/password to commit changes?
G

Re: [cpp-netlib-devel] Some code already up in subversion

[Dean M. B. <mik...@gm...>]

Hi Glyn!

On 5/20/07, Glyn Matthews <gly...@gm...> wrote:
> Dean,
>
> On 20/05/07, Dean Michael Berris <mik...@gm...> wrote:
> > I'm developing this using the Test Driven Development approach, where
> > I write the tests first (in libs/network/test) then make the tests
> > "run green" by adding new functionality and/or refactoring.
>
>
> That's a good approach, I agree.  But I ran the tests immediately and they
> didn't run.  I used gcc 4.1.2 and intel icc 9.1 on suse linux 10.2.  I was
> able to run the tests by changing the following lines to:
>
> network/message/wrappers/headers.hpp:35
> return
> headers_wrapper<Tag>::_message.headers().equal_range(key)
>
> network/message/wrappers/headers.hpp:39
> return headers_wrapper<Tag>::_message.headers().count(key)
>
> And I also replaced the local class "tags" in the first test case.
>

Good catch! I haven't tested on Linux just yet, been running the tests
in Windows using MSVC 8. :)

> For the tests it might be better to include individual headers (e.g.
> message.hpp instead of network.hpp).  Also, I'm a bit worried that the
> header guards could potentially clash (for example, you have used
> __NETWORK_HPP__ and I've seen that defined in other headers that I've come
> across).  These are minor things, however.
>

Yeah, somehow I'd have to rethink the include guard names... I've
found myself doing that from habit, naming the include guard based on
the file name. :D We could change this by adding some uniqe prefix or
suffix... Should be simple enough to do though. :)

> How do you intend to structure the tests?  Will you test the different
> layers that you have described distinctly?  Have you given any thought to
> how to test between layers?
>

Actually I use the unit tests for specifying interface and behavior.
The approach is to do testing at various levels, and this is the
lowest.

The next level of tests would be integration tests, where multi-layer
operations are used and verified to be working according to behavioral
expectations. These would go into a different subdirectory inside of
libs/network/test -- something like libs/network/test/integration.

Then there'd have to be some sort of mock testing at the protocol
level. I think we can write testing harnesses that other people can
use, because I know I've done this before in another project. ;)

We can structure the tests in a different way if this approach seems
cluttered and unorganized... I just tend to put unit tests into a
single place and name them differently to distinguish what I want them
called. :)

> I have to say I'm new to boost development, should I update any changes
> myself or do I have to pass them by you first?
>

Please update any changes as long as the tests are green. Again, if
we're going to be introducing new functionality/code, we need a unit
test for it first. :D

Any "major" changes would have to be discussed among ourselves though. :)

> Kind regards,
> Glyn
>

Thank you very much Glyn! :)

-- 
Dean Michael C. Berris
http://cplusplus-soup.blogspot.com/
mikhailberis AT gmail DOT com
+63 928 7291459

Re: [cpp-netlib-devel] Some code already up in subversion

[Glyn M. <gly...@gm...>]

Dean,

On 20/05/07, Dean Michael Berris <mik...@gm...> wrote:

> I'm developing this using the Test Driven Development approach, where
> I write the tests first (in libs/network/test) then make the tests
> "run green" by adding new functionality and/or refactoring.


That's a good approach, I agree.  But I ran the tests immediately and they
didn't run.  I used gcc 4.1.2 and intel icc 9.1 on suse linux 10.2.  I was
able to run the tests by changing the following lines to:

network/message/wrappers/headers.hpp:35
return headers_wrapper<Tag>::_message.headers().equal_range(key)

network/message/wrappers/headers.hpp:39
return headers_wrapper<Tag>::_message.headers().count(key)

And I also replaced the local class "tags" in the first test case.

For the tests it might be better to include individual headers (e.g.
message.hpp instead of network.hpp).  Also, I'm a bit worried that the
header guards could potentially clash (for example, you have used
__NETWORK_HPP__ and I've seen that defined in other headers that I've come
across).  These are minor things, however.

How do you intend to structure the tests?  Will you test the different
layers that you have described distinctly?  Have you given any thought to
how to test between layers?

I have to say I'm new to boost development, should I update any changes
myself or do I have to pass them by you first?

Kind regards,
Glyn

[cpp-netlib-devel] Some code already up in subversion

[Dean M. B. <mik...@gm...>]

Hi Everyone!

If and when you have the time, I've been working on the message type
and I've got code already up in the subversion repository.

I'm developing this using the Test Driven Development approach, where
I write the tests first (in libs/network/test) then make the tests
"run green" by adding new functionality and/or refactoring.

I've tried to put as much documentation into the code as I can, and if
there are some things not clear form the get go please feel free to
email questions/suggestions/clarifications to the list so we can
discuss it.

Thanks in advance!

-- 
Dean Michael C. Berris
http://cplusplus-soup.blogspot.com/
mikhailberis AT gmail DOT com
+63 928 7291459