cpp-netlib-devel

[cpp-netlib-devel] Some code already up in subversion

[Dean M. B. <mik...@gm...>]

Hi Everyone!

If and when you have the time, I've been working on the message type
and I've got code already up in the subversion repository.

I'm developing this using the Test Driven Development approach, where
I write the tests first (in libs/network/test) then make the tests
"run green" by adding new functionality and/or refactoring.

I've tried to put as much documentation into the code as I can, and if
there are some things not clear form the get go please feel free to
email questions/suggestions/clarifications to the list so we can
discuss it.

Thanks in advance!

-- 
Dean Michael C. Berris
http://cplusplus-soup.blogspot.com/
mikhailberis AT gmail DOT com
+63 928 7291459

Re: [cpp-netlib-devel] Some code already up in subversion

[Glyn M. <gly...@gm...>]

Dean,

On 20/05/07, Dean Michael Berris <mik...@gm...> wrote:

> I'm developing this using the Test Driven Development approach, where
> I write the tests first (in libs/network/test) then make the tests
> "run green" by adding new functionality and/or refactoring.


That's a good approach, I agree.  But I ran the tests immediately and they
didn't run.  I used gcc 4.1.2 and intel icc 9.1 on suse linux 10.2.  I was
able to run the tests by changing the following lines to:

network/message/wrappers/headers.hpp:35
return headers_wrapper<Tag>::_message.headers().equal_range(key)

network/message/wrappers/headers.hpp:39
return headers_wrapper<Tag>::_message.headers().count(key)

And I also replaced the local class "tags" in the first test case.

For the tests it might be better to include individual headers (e.g.
message.hpp instead of network.hpp).  Also, I'm a bit worried that the
header guards could potentially clash (for example, you have used
__NETWORK_HPP__ and I've seen that defined in other headers that I've come
across).  These are minor things, however.

How do you intend to structure the tests?  Will you test the different
layers that you have described distinctly?  Have you given any thought to
how to test between layers?

I have to say I'm new to boost development, should I update any changes
myself or do I have to pass them by you first?

Kind regards,
Glyn

Re: [cpp-netlib-devel] Some code already up in subversion

[Dean M. B. <mik...@gm...>]

Hi Glyn!

On 5/20/07, Glyn Matthews <gly...@gm...> wrote:
> Dean,
>
> On 20/05/07, Dean Michael Berris <mik...@gm...> wrote:
> > I'm developing this using the Test Driven Development approach, where
> > I write the tests first (in libs/network/test) then make the tests
> > "run green" by adding new functionality and/or refactoring.
>
>
> That's a good approach, I agree.  But I ran the tests immediately and they
> didn't run.  I used gcc 4.1.2 and intel icc 9.1 on suse linux 10.2.  I was
> able to run the tests by changing the following lines to:
>
> network/message/wrappers/headers.hpp:35
> return
> headers_wrapper<Tag>::_message.headers().equal_range(key)
>
> network/message/wrappers/headers.hpp:39
> return headers_wrapper<Tag>::_message.headers().count(key)
>
> And I also replaced the local class "tags" in the first test case.
>

Good catch! I haven't tested on Linux just yet, been running the tests
in Windows using MSVC 8. :)

> For the tests it might be better to include individual headers (e.g.
> message.hpp instead of network.hpp).  Also, I'm a bit worried that the
> header guards could potentially clash (for example, you have used
> __NETWORK_HPP__ and I've seen that defined in other headers that I've come
> across).  These are minor things, however.
>

Yeah, somehow I'd have to rethink the include guard names... I've
found myself doing that from habit, naming the include guard based on
the file name. :D We could change this by adding some uniqe prefix or
suffix... Should be simple enough to do though. :)

> How do you intend to structure the tests?  Will you test the different
> layers that you have described distinctly?  Have you given any thought to
> how to test between layers?
>

Actually I use the unit tests for specifying interface and behavior.
The approach is to do testing at various levels, and this is the
lowest.

The next level of tests would be integration tests, where multi-layer
operations are used and verified to be working according to behavioral
expectations. These would go into a different subdirectory inside of
libs/network/test -- something like libs/network/test/integration.

Then there'd have to be some sort of mock testing at the protocol
level. I think we can write testing harnesses that other people can
use, because I know I've done this before in another project. ;)

We can structure the tests in a different way if this approach seems
cluttered and unorganized... I just tend to put unit tests into a
single place and name them differently to distinguish what I want them
called. :)

> I have to say I'm new to boost development, should I update any changes
> myself or do I have to pass them by you first?
>

Please update any changes as long as the tests are green. Again, if
we're going to be introducing new functionality/code, we need a unit
test for it first. :D

Any "major" changes would have to be discussed among ourselves though. :)

> Kind regards,
> Glyn
>

Thank you very much Glyn! :)

-- 
Dean Michael C. Berris
http://cplusplus-soup.blogspot.com/
mikhailberis AT gmail DOT com
+63 928 7291459

Re: [cpp-netlib-devel] Some code already up in subversion

[Glyn M. <gly...@gm...>]

On 20/05/07, Dean Michael Berris <mik...@gm...> wrote:
>
> Hi Glyn!
>
> On 5/20/07, Glyn Matthews <gly...@gm...> wrote:
>
> Please update any changes as long as the tests are green. Again, if
> we're going to be introducing new functionality/code, we need a unit
> test for it first. :D


Do I need a username/password to commit changes?
G

Re: [cpp-netlib-devel] Some code already up in subversion

[Peter S. <si...@cr...>]

Hi guys, 

it took a while to remember my sf.net password, but in the end I
managed to commit. :-)

I have a question. In the boost-sandbox repository, everyone is
encouraged to add a top-level directory per project. An imaginary
foo library is organized in SVN as follows:

  foo / boost / foo.hpp
  foo / boost / foo / ...
  foo / libs  / foo / src  / ...
  foo / libs  / foo / doc  / ...
  foo / libs  / foo / test / ...

How would we like to handle that? What is the granularity we aim
for in this project? Let's say we come up with a very good class
for representing URIs. Do we try to make that URI library a
stand-alone component (that can be submitted to boost on its
own), or is URI something that's tightly integrated into
cpp-netlib?

Tight integration often leads to more efficient source code
because chances for re-use are higher. If things are integrated
too tightly, however, it is hard to use any one component of the
library without using everything else too. 

What are the components we anticipate to have in cpp-netlib? A
general purpose class for things that look like an "Internet
Message" is one of topics we need to address, but what else would
we like to see in Boost?

Best regards,
Peter

Re: [cpp-netlib-devel] Some code already up in subversion

[Christian H. <chh...@gm...>]

Hi Dean, I have successfully compiled and run the code using VC7.1 .
I'm using boost 1.34. I tried boost 1.33.1 but then the test code
doesn't compile. So, I think we should make a statement of using 1.34.

One reason for me to join this effort was to learn more about
networking application. So, please excuse my obvious lack of
knowledge. I was taking a look at asio's http example. How would you
construct your network::message to generate a client request. In this
example the client does the following:

    std::ostream request_stream(&request_);
    request_stream << "GET " << path << " HTTP/1.0\r\n";
    request_stream << "Host: " << server << "\r\n";
    request_stream << "Accept: */*\r\n";
    request_stream << "Connection: close\r\n\r\n";

as an example the server might be "www.boost.org" and the path is
"/LICENSE_1_0.txt".

So what would be the source identifier, destination identifier,
header, and body?

Regards,
Christian

On 5/20/07, Dean Michael Berris <mik...@gm...> wrote:
> Hi Everyone!
>
> If and when you have the time, I've been working on the message type
> and I've got code already up in the subversion repository.
>
> I'm developing this using the Test Driven Development approach, where
> I write the tests first (in libs/network/test) then make the tests
> "run green" by adding new functionality and/or refactoring.
>
> I've tried to put as much documentation into the code as I can, and if
> there are some things not clear form the get go please feel free to
> email questions/suggestions/clarifications to the list so we can
> discuss it.
>
> Thanks in advance!
>
> --
> Dean Michael C. Berris
> http://cplusplus-soup.blogspot.com/
> mikhailberis AT gmail DOT com
> +63 928 7291459
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> _______________________________________________
> Cpp-netlib-devel mailing list
> Cpp...@li...
> https://lists.sourceforge.net/lists/listinfo/cpp-netlib-devel
>

Re: [cpp-netlib-devel] Some code already up in subversion

[Dean M. B. <mik...@gm...>]

Hi Christian!

On 5/20/07, Christian Henning <chh...@gm...> wrote:
>
> Hi Dean, I have successfully compiled and run the code using VC7.1 .
> I'm using boost 1.34. I tried boost 1.33.1 but then the test code
> doesn't compile. So, I think we should make a statement of using 1.34.


I actually failed to mention that I've been using Boost CVS HEAD all this
time for testing. I think the problems we'll be encountering with 1.33.1 is
the interface changes to Boost.Test which are not backward compatible. The
version of Boost.Test in CVS is much closer to 1.34 which explains why it
would build with that release.

The recommendation would be that we should use Boost CVS HEAD all the time
while developing and try to make the library/test build when using Boost
1.34.

One reason for me to join this effort was to learn more about
> networking application. So, please excuse my obvious lack of
> knowledge.


No problem, we're all in it for the learning as well I think. ;-)

I was taking a look at asio's http example. How would you
> construct your network::message to generate a client request. In this
> example the client does the following:
>
>     std::ostream request_stream(&request_);
>     request_stream << "GET " << path << " HTTP/1.0\r\n";
>     request_stream << "Host: " << server << "\r\n";
>     request_stream << "Accept: */*\r\n";
>     request_stream << "Connection: close\r\n\r\n";
>
> as an example the server might be "www.boost.org" and the path is
> "/LICENSE_1_0.txt".


The recommended usage is something in this sort:

using namespace boost::network ;
message request ;
request << header("GET", std::string(path) + " HTTP/1.0")
    << header("Host", server)
    << header("Accept", "*/*")
    << header("Connection", "close")
    << body("")
;

Which can then be encapsulated in (when we already have it) an http request
object like such:

using namespace boost::network;
http::request boost_http_request(request, http::url("
http://boost.org/LICENSE_1_0.txt"));
copy (istream_iterator<std::string>(boost_http_request),
    istream_iterator<std::string>(),
    ostream_iterator<std::string>(cout, "\n"));

So what would be the source identifier, destination identifier,
> header, and body?


The source can be ignored, since http requests don't really need a "source",
and the destination is usually just part of the message headers. Considering
that the message is generic, it just holds the information which is then
converted/dealt with appropriately by the 'transformer' and 'protocol
implementation' -- in the above case an http::request is a network specific
translator.

An even simpler example (which wouldn't use a boost::network::message) would
be the following:

using namespace boost::network::http;
request boost_website(url("http://boost.org/LICENSE_1_0.txt"),
method("GET"), protocol::version<1.1>());
cout << boost_website;

:-)

Regards,
> Christian


I certainly hope the explanation above makes sense. :)

If not, we can discuss this further. :)

-- 
Dean Michael C. Berris
http://cplusplus-soup.blogspot.com/
mikhailberis AT gmail DOT com
+63 928 7291459

Re: [cpp-netlib-devel] Some code already up in subversion

[Dean M. B. <mik...@gm...>]

Hi Peter!

On 20 May 2007 21:27:15 +0200, Peter Simons <si...@cr...> wrote:
> Hi guys,
>
> it took a while to remember my sf.net password, but in the end I
> managed to commit. :-)
>

Great! I'll check out your changes soon as I can. :)

> I have a question. In the boost-sandbox repository, everyone is
> encouraged to add a top-level directory per project. An imaginary
> foo library is organized in SVN as follows:
>
>   foo / boost / foo.hpp
>   foo / boost / foo / ...
>   foo / libs  / foo / src  / ...
>   foo / libs  / foo / doc  / ...
>   foo / libs  / foo / test / ...
>
> How would we like to handle that? What is the granularity we aim
> for in this project? Let's say we come up with a very good class
> for representing URIs. Do we try to make that URI library a
> stand-alone component (that can be submitted to boost on its
> own), or is URI something that's tightly integrated into
> cpp-netlib?
>

A suggestion would be to have that class under the boost/network/
directory. This would yield something like a 'boost/network/uri.hpp'
header which can be used on its own.

cpp-netlib supposedly follows the above structure, replacing 'trunk'
with something like 'network' so the structure looks like

    network / boost / network
    network / libs / network

> Tight integration often leads to more efficient source code
> because chances for re-use are higher. If things are integrated
> too tightly, however, it is hard to use any one component of the
> library without using everything else too.
>

Agreed.

> What are the components we anticipate to have in cpp-netlib? A
> general purpose class for things that look like an "Internet
> Message" is one of topics we need to address, but what else would
> we like to see in Boost?
>

Definitely a URI class would be a good idea, preferrably something
that is convertible to a std::string by default. We can even use the
same approach we have for creating message objects with directives and
operator overloads to make the usage be a bit more expressive.

> Best regards,
> Peter
>

Thanks Peter!

I hope the above explanation makes sense. :)

-- 
Dean Michael C. Berris
http://cplusplus-soup.blogspot.com/
mikhailberis AT gmail DOT com
+63 928 7291459

Re: [cpp-netlib-devel] Some code already up in subversion

[Peter S. <si...@cr...>]

Hi Dean,

 > A suggestion would be to have that class under the
 > boost/network/ directory. This would yield something like a
 > 'boost/network/uri.hpp' header which can be used on its own.

Yes, for an Uri class that's perfectly reasonable, but for other
classes things are not that obvious. The rfc2822 parsers, for
example, are, well, parsers. They will typically be applied to
data that comes from the network, but the need to parse an e-mail
address can just as well arise in a spreadsheet program. Adding
those parsers into a boost/network/rfc2822/ hierarchy does make
sense to a degree, but it doesn't feel like the natural choice --
particularly considering the fact that those parsers will never
depend on anything else in the network hierarchy.

I still can't quite imagine what the scope of this whole endeavor
is. Right now, I have a hard time seeing how my code fits in.


 >> What are the components we anticipate to have in cpp-netlib?
 >
 > Definitely a URI class would be a good idea.

Right, an URI class would be great to have.

Are there any other components we anticipate or would like to
implement? What is the goal for cpp-netlib? When it's all done
and accepted into boost, what does it contain?

I'm sorry for bringing up those very general questions instead of
committing code, but it feels important to develop some sort of
shared vision. I have my code in a publicly available repository
already; so it doesn't feel like I'm gaining a lot by checking it
into this repository instead. I would do so because I'd hope that
the code might become part of a team effort. At the moment, I
can't quite say where exactly that team effort is headed. It
would be nice to develop that notion.

Best regards,
Peter

Re: [cpp-netlib-devel] Some code already up in subversion

[Dean M. B. <mik...@gm...>]

Hi Peter!

On 21 May 2007 06:22:27 +0200, Peter Simons <si...@cr...> wrote:
>
> Yes, for an Uri class that's perfectly reasonable, but for other
> classes things are not that obvious. The rfc2822 parsers, for
> example, are, well, parsers. They will typically be applied to
> data that comes from the network, but the need to parse an e-mail
> address can just as well arise in a spreadsheet program. Adding
> those parsers into a boost/network/rfc2822/ hierarchy does make
> sense to a degree, but it doesn't feel like the natural choice --
> particularly considering the fact that those parsers will never
> depend on anything else in the network hierarchy.
>

I agree that boost/network/rfc2822 doesn't feel natural. Would you
have a different name from rfc2822 which might make more sense and be
more descriptive? It might also even make sense that we can put it in
boost/network/smtp/parser/ and have a boost/network/smtp/parsers.hpp
file (just in case I understand that the parsers are for SMTP
parsing).

> I still can't quite imagine what the scope of this whole endeavor
> is. Right now, I have a hard time seeing how my code fits in.
>

Okay, let me try and do a bird's eye view:

We have a message class, which every network protocol implementation
will use (meaning, create/return/manipulate/process). Of the network
protocol implementations, there will be parsers which will accept
string input and validate then generate message objects. These parsers
will generally be based on Boost.Spirit.

That being said, protocol implementations for SMTP processing will
definitely be used to generate message objects which the SMTP handlers
will use to perform the necessary actions. For example, for mail
sending the parsers will have to parse string responses from the SMTP
servers. Another example is for SMTP servers where incoming text
messages will be processed with a parser to generate the appropriate
message object, which then can be passed on to application specific
handlers. A POP client will also benefit from parsers to create
message objects on a per-email basis.

I hope the above descriptions/narrative makes things clearer... I
admit am having a hard time writing down these things that are pretty
much still all in my head. I will do what I can though to be able to
document all this before and during the development process.

>
>  >> What are the components we anticipate to have in cpp-netlib?
>  >
>  > Definitely a URI class would be a good idea.
>
> Right, an URI class would be great to have.
>
> Are there any other components we anticipate or would like to
> implement? What is the goal for cpp-netlib? When it's all done
> and accepted into boost, what does it contain?
>

One question at a time. ;-)

Components we anticipate are (aside from the message class):

  - asynchronous name resolver
  - "synchronous timeout reader/writer"
  - protocol implementations (of course)
  - server templates (for PBCP and CRTP servers)

The goal for cpp-netlib is to:

 * provide a high-quality, generic, extensible collection of
network-related routines and popular protocol implementations using
the C++ programming language
 * become the Boost.Network library
 * enable generic network protocol implementations
 * lower the barrier to entry for network-aware C++ applications

When it's gotten into Boost, it should contain at least the following:

 * Common message type
 * Transformations/Manipulations for the common message type instances
 * Minimal HTTP implementation

If we can even make an SMTP implementation using the parsers you've
written, then we could include that as well in the list above.

> I'm sorry for bringing up those very general questions instead of
> committing code, but it feels important to develop some sort of
> shared vision. I have my code in a publicly available repository
> already; so it doesn't feel like I'm gaining a lot by checking it
> into this repository instead. I would do so because I'd hope that
> the code might become part of a team effort. At the moment, I
> can't quite say where exactly that team effort is headed. It
> would be nice to develop that notion.
>

I'm open to suggestions as well. :) I'm also thankful that you bring
up these questions early, so that we can go discuss the direction at
this point, then be able to plan for the concrete actions that we'd
like to take.

I'd really like to know your thoughts on the things I've written above.

I hope this makes things clear to some extent. Please ask away and
suggest away about any matter regarding the development process and or
direction of the project.

(I'm pretty new at this open source development thing as well, and I
do have some trouble writing down ideas so please bear with me on
this... I'll try my best to articulate in terms of
English/HumanLanguage what I think -- and if I can't I'll write code
instead. ;) ).

> Best regards,
> Peter
>

Thanks again Peter!

-- 
Dean Michael C. Berris
http://cplusplus-soup.blogspot.com/
mikhailberis AT gmail DOT com
+63 928 7291459

Re: [cpp-netlib-devel] Some code already up in subversion

[Christian H. <chh...@gm...>]

Hi guys,

>
> I agree that boost/network/rfc2822 doesn't feel natural. Would you
> have a different name from rfc2822 which might make more sense and be
> more descriptive? It might also even make sense that we can put it in
> boost/network/smtp/parser/ and have a boost/network/smtp/parsers.hpp
> file (just in case I understand that the parsers are for SMTP
> parsing).
>

Why not have the parser in boost/network/parsers/. Since some parsers
are used in more than one protocol I don't think we should order them
based on protocols. I think we should split them as much as we can,
like in the rfc2822 lib. Although here I would change the namespace.

Christian

Re: [cpp-netlib-devel] Some code already up in subversion

[Dean M. B. <mik...@gm...>]

Hi Christian!

On 5/21/07, Christian Henning <chh...@gm...> wrote:
>
> Why not have the parser in boost/network/parsers/. Since some parsers
> are used in more than one protocol I don't think we should order them
> based on protocols. I think we should split them as much as we can,
> like in the rfc2822 lib. Although here I would change the namespace.
>

Good idea. :) This sounds good to me. :)

-- 
Dean Michael C. Berris
http://cplusplus-soup.blogspot.com/
mikhailberis AT gmail DOT com
+63 928 7291459