cpp-netlib-devel — Developers Mailing List

[cpp-netlib-devel] [network] Release 0.9.0

[Dean M. B. <mik...@gm...>]

Good day everyone, apologies for the cross-post.

I just wanted to point everyone interested in the development of the
cpp-netlib project to the 0.9.0 release announcement over at C++ Soup:
http://cplusplus-soup.com/2011/04/18/cpp-netlib-0-9-0-released/ --
more information about what happened in the library between the beta
and the official release can be found there.

As earlier mentioned, this version is the pre-1.0 snapshot that is
being officially submitted to the Boost C++ Library as Boost.Network.
I would like to ask the review wizards to please accept this
announcement as a request to put Boost.Network/cpp-netlib into the
review queue pending a review manager. I would greatly appreciate
volunteers who would like to be the review manager(s) for the
Boost.Network review.

Have a good one and I definitely hope this helps.

Cheers from down under!

-- 
Dean Michael Berris
http://about.me/deanberris

Re: [cpp-netlib-devel] Working with Google, Future of cpp-netlib

[Dean M. B. <mik...@gm...>]

On Tue, Apr 19, 2011 at 7:19 AM, Glyn Matthews <gly...@gm...> wrote:
> Hi Dean,
>
> On 17 April 2011 16:10, Dean Michael Berris <mik...@gm...> wrote:
>>
>> Hi Everyone!
>>
>> I just wanted to let you all know that last February I accepted an
>> offer to work with Google Australia -- between then and now I've been
>> concerned with getting my paper work in order and tying up loose ends
>> in my consulting work before this fateful day. In a few hours, I will
>> be officially starting with Google Australia.
>>
>
> Excellent news, congratulations!
>

Thanks Glyn!

>>
>> I would just like to reiterate how I'm still committed to developing
>> and supporting the cpp-netlib effort -- and I encourage everyone who
>> has ideas as to what would be good to have and what would be good to
>> do with the library to sound off and make yourselves heard. I cannot
>> tell you enough how much hearing people are using the library makes me
>> feel -- and how it makes other contributors feel. Even if I'm not
>> making money off of cpp-netlib at the moment, the idea that people
>> like it and are thinking of using it in their projects (and those that
>> are already using it in their projects) is currency enough for me.
>>
>
> I'd still like to contribute, but I've been through some changes recently
> myself which has affected my commitment.  I've still got an old XMPP
> implementation that I'd like to see developed further.
>

That's cool, I understand exactly what you mean. :)

Make that two of us who'd like to see that implementation developed
further. Maybe we should schedule a 1.0-devel branch-and-merge (like
how we did last time circa 0.7 with the documentation overhaul) to
make that code available to more people and closer to the "main line"
development soon?

>>
>> Thanks again everyone and I do look forward to a more fruitful
>> cpp-netlib effort along with this new chapter of my life.
>>
>> Cheers from down under!
>
> Cheers, and I hope it goes well for you there!

Thanks Glyn! You and I both. :)

-- 
Dean Michael Berris
http://about.me/deanberris

Re: [cpp-netlib-devel] HTTPS server support

[Dean M. B. <mik...@gm...>]

On Tue, Apr 19, 2011 at 11:33 AM, Simon Collins <sim...@gm...> wrote:
> Hi Everyone,

Hi Simon,

> I was wondering if anyone was working on implementing SSL support server
> side? I think this has come up a few times in the past on the mailing list,
> I could take a look at it myself I suppose, but the reason I'm using this
> library is because I have less experience with network programming, so I
> would think it better if a more experienced contributor was able to look at
> the issue.

If nobody would pick this up, I probably might have to do it as well. :/

> The next step would be creating an issue in github, but I wanted to put this
> out there in case someone had a rough idea on how to approach the problem -
> would make a good introduction in the issue description.

+1 -- I would love to get others' insights on how else to do this in a
user-friendly manner.

> Thanks,
> Simon

Thank you Simon, I do hope others would respond to the question as well.

Cheers!

-- 
Dean Michael Berris
http://about.me/deanberris

[cpp-netlib-devel] HTTPS server support

[Simon C. <sim...@gm...>]

Hi Everyone,

I was wondering if anyone was working on implementing SSL support server
side? I think this has come up a few times in the past on the mailing list,
I could take a look at it myself I suppose, but the reason I'm using this
library is because I have less experience with network programming, so I
would think it better if a more experienced contributor was able to look at
the issue.

The next step would be creating an issue in github, but I wanted to put this
out there in case someone had a rough idea on how to approach the problem -
would make a good introduction in the issue description.

Thanks,

Simon

Re: [cpp-netlib-devel] Working with Google, Future of cpp-netlib

[Glyn M. <gly...@gm...>]

Hi Dean,

On 17 April 2011 16:10, Dean Michael Berris <mik...@gm...> wrote:

> Hi Everyone!
>
> I just wanted to let you all know that last February I accepted an
> offer to work with Google Australia -- between then and now I've been
> concerned with getting my paper work in order and tying up loose ends
> in my consulting work before this fateful day. In a few hours, I will
> be officially starting with Google Australia.
>
>
Excellent news, congratulations!


> I would just like to reiterate how I'm still committed to developing
> and supporting the cpp-netlib effort -- and I encourage everyone who
> has ideas as to what would be good to have and what would be good to
> do with the library to sound off and make yourselves heard. I cannot
> tell you enough how much hearing people are using the library makes me
> feel -- and how it makes other contributors feel. Even if I'm not
> making money off of cpp-netlib at the moment, the idea that people
> like it and are thinking of using it in their projects (and those that
> are already using it in their projects) is currency enough for me.
>
>
I'd still like to contribute, but I've been through some changes recently
myself which has affected my commitment.  I've still got an old XMPP
implementation that I'd like to see developed further.


> Thanks again everyone and I do look forward to a more fruitful
> cpp-netlib effort along with this new chapter of my life.
>
> Cheers from down under!
>

Cheers, and I hope it goes well for you there!
G

[cpp-netlib-devel] Working with Google, Future of cpp-netlib

[Dean M. B. <mik...@gm...>]

Hi Everyone!

I just wanted to let you all know that last February I accepted an
offer to work with Google Australia -- between then and now I've been
concerned with getting my paper work in order and tying up loose ends
in my consulting work before this fateful day. In a few hours, I will
be officially starting with Google Australia.

That said though I will still keep working on The C++ Network Library
and I do hope others will be encouraged to contribute to the library
as we're nearing the 1.0 milestone we're trying to hit for eventual
review submission (and hopefully inclusion) to the Boost C++ Library.
At this time I would like to thank everyone who has contributed in
ways small and large to the project and bringing it up to where it is
at now.

In the next few days I should be releasing 0.9.0 which should be the
snapshot version for inclusion into the Boost C++ Library -- I don't
have any illusions that that version will make it into Boost without
any major modifications, but I wanted to get that ball rolling for me
to better see where the library should go. Hopefully by the time the
review comes (which has yet to be scheduled, I still need to contact a
review manager who will take that task on) 1.0 would be suitable for a
larger production deployment base.

I would just like to reiterate how I'm still committed to developing
and supporting the cpp-netlib effort -- and I encourage everyone who
has ideas as to what would be good to have and what would be good to
do with the library to sound off and make yourselves heard. I cannot
tell you enough how much hearing people are using the library makes me
feel -- and how it makes other contributors feel. Even if I'm not
making money off of cpp-netlib at the moment, the idea that people
like it and are thinking of using it in their projects (and those that
are already using it in their projects) is currency enough for me.

Thanks again everyone and I do look forward to a more fruitful
cpp-netlib effort along with this new chapter of my life.

Cheers from down under!

-- 
Dean Michael Berris
http://about.me/deanberris

Re: [cpp-netlib-devel] [Proxy] The great forgotten feature :)

[Dean M. B. <mik...@gm...>]

Hi Amoriello! First off I'd like to thank you for taking the time and
detailing your thoughts. The least I can do is thank you. :D

That said, please see my thoughts in-lined below:

On Sat, Apr 16, 2011 at 7:38 PM, Amoriello Hutti
<amo...@gm...> wrote:
> Hi Dean, Contributors,
>
>
> First of all, I would like to thanks you and the contributors of the
> cpp-netlib for their awsome work!

Thank you, I'm glad you think it's awesome. :)

> I would like to use you library, but it misses a key feature for the
> use I would like to make of this lib.
>
> This feature is described as "Use O/S described proxy facilites"
> https://github.com/cpp-netlib/cpp-netlib/issues/2
>
> I am a cpp fan, but not as experienced as you and the contributors, so
> I can't submit any pull request yet.
>
> But I have red the docs (html, slide, paper and even the boostcon talk
> :) and I finaly came to open the code.
> All I can share (now) to help this request to be done, is to share
> some points I saw while reading the code.
>

Cool! Thanks for going through all the effort. ;)

> Actualy, I tried to imagine where to start if I wanted to implement a
> "simple" http-proxy configuration : proxy name and port (no
> authentication) in the sync_client impl.
> (and because this is the only thing I need yet :)
>
>
> * as you said in the request comment, I also think the best place to
> configure the proxy information is the client ctor.
> ------------------------------------------------------------------------------------------------
>
> So we can imagine we add have a
>
> sync_client::sync_client (bool cache_resolved,
>                                     bool follow_redirect,
>                                     optional<string_type> const&
> certificate_path = optional<string_type>(),
>                                     optional<string_type> const&
> verify_path = optional<string_type>(),
>                                     optional<string_type> const&
> proxy_host = optional<string_type>(),
>                                     optional<string_type> const&
> proxy_port = optional<string_type>())
>
>

Well... actually you'd like to look at the Boost.Parameter based
constructors that the server and clients already support in 0.9-beta
(soon to be 0.9.0). That would make it a lot easier to extend the
constructor options that would be supported -- along with adding
Boost.Parameter options to the other API calls.

>
> * Then we have to change the sync_client:request_skeleton<>(...)
> ------------------------------------------------------------------------------------------------
>
> In this function, you get the connection_object from the
> connection_base::get_connection.
> Here, we have to pass to the get_connection function the proxy_host
> and proxy_port optionals string. That would lead to the following
> prototype :
>
> connection_ptr get_connection(resolver_type& resolver,
>                                             basic_request<Tag> const& request_,
>                                             optional<string_type>
> const & certificate_file = optional<string_type>(),
>                                             optional<string_type>
> const & verify_file = optional<string_type>(),
>                                             optional<string_type>
> const & proxy_host = optional<string_type>(),
>                                             optional<string_type>
> const & proxy_port = optional<string_type>())
>
>
> and then, if (proxy_host.size != 0), then I initialize the
> connection_impl not on the request.host, and port, but on the
> proxy_host and port.
>

You can also use the Boost.Parameters based approach here, or if
you're looking at hard-coding it as an internal detail (like how the
options for the SSL-specific parts are handled) then it would look
like this is the way to go.

> ----- I don't really think this "if" is what you expect in a "modern"
> library design.
>

Well, it being a runtime option, we can't really do anything about
that. The 'if' statement is still part of the language the last time I
checked, no reason for us not to use it. :D

>
>
>
> * Then the connection_base::send_request
> ------------------------------------------------------------------------------------------------
>
> basic_response<Tag>
> send_request(string_type const & method, basic_request<Tag> request_,
> bool get_body)
>
> Here, we loose the trace of the proxy_host and proxy_port optional
> string (maybe they should be added to the connection_policy's private
> members).
>
> I think, there, we are close to the end of the "simple proxy support
> implementation".
>

Yeah, actually what should really happen is that there should be a way
to isolate the factory for connections and have the factories be
constructed based on runtime options passed into the constructors.
This will be an internal detail of the library which will make it
easier to extend the implementation in the future, without having to
burden the user with dependency injection mumbo jumbo from the
interface.

I'll implement this and write a paper about it when I get the time to
do that -- between me moving to Sydney, Australia and starting work at
Google Australia in a few hours, there's not a lot of time for me to
get work done. But that's another issue that needs to be addressed in
a different email (following shortly). :D

>
> * Last, but not least. The linearize algorithm.
> ------------------------------------------------------------------------------------------------
>
> When we connect to a proxy, (using GET for example), the GET parameter
> MUST be the entire uri, not only the uri.path().
>
> if (request.path().empty() || request.path()[0] != consts::slash_char())
>       *oi = consts::slash_char();
> boost::copy(request.path(), oi);
>
>
> But there, we completely loose the trace of the proxy_host and
> proxy_port information, so there is no way to know if it is the
> complete uri that should be written, or only the uri.path().
>
> I am not really friendly with boost::concept yet, so the linearize.hpp
> code is quite obfuscated to me.
> I don't know how to help the linearize algorithm to be aware of the
> fact that it is behind a proxy or not.
>

This can be done with a predicate or function object. We can have a
path transformer function object that takes a uri and turns it into
the appropriate linearized request. This also means we can then use
the linearize algorithm for things like body compression, encoding,
etc. by providing points of customization, which would be similar to
how the STL algorithms do it -- with function objects passed as
parameters to the call to the linearize algorithm.

Aside from the tag dispatch, we can then set up a set of default
transformers and implementations that "do nothing" basically behaving
as they do at the moment. At the point where we'd call the linearize
algorithm, we then determine externally whether the path transformer
needs to know about the proxy stuff along with whether the headers
transformer would need to add more details to the headers being
linearized.

There's a lot of leeway for the algorithm to evolve, and it being an
implementation detail that just so happens to be generic enough to be
lifted, we're in good shape with changing the implementation and
interface. :)

> Maybe a new tag "proxy" and some meta like "is_behind_proxy" to
> provide a static dispatch based on the tag and avoid the ugly "if" in
> the connection object, and help the linearize algo?
>

If you want to encode that information statically then that would be
an option. However I don't think there is any reason to make that a
static option when it can very well be just a runtime configuration
option.

> Thanks for this lib, I hope futur boost reviewer will see the
> potential of this piece of master code and design, and will accept it
> to be integrated in the boost trunk.
> Sorry if I can't submit any pull request yet, and sorry for my bad english.
>

No problem and thanks for the encouragement. I'll try to see if I can
get things going to meet some of the requirements for the proxy
implementation, but in the meantime please feel free to just play
around with the code and submit any (and al) pull requests that you
might want to share with me.

Have a good one and I definitely do hope I can attend to this in the
coming days/weeks.

Cheers!

-- 
Dean Michael Berris
http://about.me/deanberris

[cpp-netlib-devel] [Proxy] The great forgotten feature :)

[Amoriello H. <amo...@gm...>]

Hi Dean, Contributors,


First of all, I would like to thanks you and the contributors of the
cpp-netlib for their awsome work!
I would like to use you library, but it misses a key feature for the
use I would like to make of this lib.

This feature is described as "Use O/S described proxy facilites"
https://github.com/cpp-netlib/cpp-netlib/issues/2

I am a cpp fan, but not as experienced as you and the contributors, so
I can't submit any pull request yet.

But I have red the docs (html, slide, paper and even the boostcon talk
:) and I finaly came to open the code.
All I can share (now) to help this request to be done, is to share
some points I saw while reading the code.

Actualy, I tried to imagine where to start if I wanted to implement a
"simple" http-proxy configuration : proxy name and port (no
authentication) in the sync_client impl.
(and because this is the only thing I need yet :)


* as you said in the request comment, I also think the best place to
configure the proxy information is the client ctor.
------------------------------------------------------------------------------------------------

So we can imagine we add have a

sync_client::sync_client (bool cache_resolved,
                                     bool follow_redirect,
                                     optional<string_type> const&
certificate_path = optional<string_type>(),
                                     optional<string_type> const&
verify_path = optional<string_type>(),
                                     optional<string_type> const&
proxy_host = optional<string_type>(),
                                     optional<string_type> const&
proxy_port = optional<string_type>())



* Then we have to change the sync_client:request_skeleton<>(...)
------------------------------------------------------------------------------------------------

In this function, you get the connection_object from the
connection_base::get_connection.
Here, we have to pass to the get_connection function the proxy_host
and proxy_port optionals string. That would lead to the following
prototype :

connection_ptr get_connection(resolver_type& resolver,
                                             basic_request<Tag> const& request_,
                                             optional<string_type>
const & certificate_file = optional<string_type>(),
                                             optional<string_type>
const & verify_file = optional<string_type>(),
                                             optional<string_type>
const & proxy_host = optional<string_type>(),
                                             optional<string_type>
const & proxy_port = optional<string_type>())


and then, if (proxy_host.size != 0), then I initialize the
connection_impl not on the request.host, and port, but on the
proxy_host and port.

----- I don't really think this "if" is what you expect in a "modern"
library design.




* Then the connection_base::send_request
------------------------------------------------------------------------------------------------

basic_response<Tag>
send_request(string_type const & method, basic_request<Tag> request_,
bool get_body)

Here, we loose the trace of the proxy_host and proxy_port optional
string (maybe they should be added to the connection_policy's private
members).

I think, there, we are close to the end of the "simple proxy support
implementation".




* Last, but not least. The linearize algorithm.
------------------------------------------------------------------------------------------------

When we connect to a proxy, (using GET for example), the GET parameter
MUST be the entire uri, not only the uri.path().

if (request.path().empty() || request.path()[0] != consts::slash_char())
       *oi = consts::slash_char();
boost::copy(request.path(), oi);


But there, we completely loose the trace of the proxy_host and
proxy_port information, so there is no way to know if it is the
complete uri that should be written, or only the uri.path().

I am not really friendly with boost::concept yet, so the linearize.hpp
code is quite obfuscated to me.
I don't know how to help the linearize algorithm to be aware of the
fact that it is behind a proxy or not.

Maybe a new tag "proxy" and some meta like "is_behind_proxy" to
provide a static dispatch based on the tag and avoid the ugly "if" in
the connection object, and help the linearize algo?

Thanks for this lib, I hope futur boost reviewer will see the
potential of this piece of master code and design, and will accept it
to be integrated in the boost trunk.
Sorry if I can't submit any pull request yet, and sorry for my bad english.

-- 
Amoriello Hutti

[cpp-netlib-devel] [network] cpp-netlib v0.9-beta out now!

[Dean M. B. <mik...@gm...>]

Hi Everyone,

Apologies for the cross-post, but I just want to announce that after a
couple months delay because of some outstanding bugs and TODO items
that needed to get dealt with, I am now ready to package a 0.9 release
and with that in mind I've just uploaded the 0.9-beta release of
cpp-netlib. When the beta period (1 week) is over and all outstanding
bugs are addressed, the 0.9 release will be the basis for submission
for review to the Boost C++ Library. You can get the beta from:

  https://github.com/cpp-netlib/cpp-netlib/downloads

What's New in 0.9 (taken from the documentation):
- IMPORTANT BREAKING CHANGE: By default all compile-time heavy parser
implementations are now compiled to external static libraries. In
order to use cpp-netlib in header-only mode, users must define the
preprocessor macro BOOST_NETWORK_NO_LIB before including any
cpp-netlib header. This breaks code that relied on the version 0.8.x
line where the library is strictly header-only.
- More consistent message API for client and server messages (request
and response objects).
- Refactoring of internal implementations to allow better separation
of concerns and more manageable coding/documentation.
- Client and Server constructors that support Boost.Parameter named parameters.
- Client and Server cosntructors now take in an optional reference to
a Boost.Asio io_service to use internally.
- Documentation updates to reflect new APIs.

This Beta has been tested mainly on Linux (GCC 4.4, 4.5) and Mac OS X
(10.6.7, Xcode 3.2.x w/ GCC 4.2.1) builds against Boost trunk and
Boost 1.45. Note that cpp-netlib does NOT work with Boost 1.46.1
because of some breaking changes in the Boost.Spirit customization
points have caused cpp-netlib to cease to play well with it. That
said, the issues have been addressed in Boost trunk and should be
fixed in Boost.147.

Thanks to everyone who contributed to making this Beta possible,
namely Emre Turkay, Oleg Malashenko, and other users that filed issues
on the Github issue tracker.

Have a good day everyone and I do look forward to feedback!

-- 
Dean Michael Berris
http://about.me/deanberris

[cpp-netlib-devel] Ready for 0.9 Beta

[Dean M. B. <mik...@gm...>]

Hi Everyone,

In the past few days I've been trying to track down a nasty bug which
I think I have squashed successfully offline. Now I'm just running the
final tests and am looking to tag a beta of the 0.9 branch for a
one-week testing period. The 0.9 beta will not work with Boost 1.46.1,
but will work against the latest from Boost's SVN trunk and with Boost
1.47 when it comes out, as well as Boost 1.45.

If anybody with access to multiple different platforms can test the
beta that will be tagged in a few minutes, feedback would be most
appreciated.

Thanks in advance guys and I look forward to a 0.9 release soon!

-- 
Dean Michael Berris
http://about.me/deanberris

[cpp-netlib-devel] Trouble with Mac OS X

[Dean M. B. <mik...@gm...>]

Hi Everyone,

As some of you might already know I've gotten myself a new development
machine and I really like it. Needless to say though I've delayed the
release of cpp-netlib 0.9 for a while mostly because of some nagging
bugs that I see popping up in the tests. Now though I've run into
something pretty serious and it involves Mac OS X and Boost.Asio.

Just to make sure that it's not just my machine, can anybody try out
the latest in 0.9-devel building and run the tests on Linux against
Boost's trunk? Windows builds and tests would be most appreciated as
well.

I'm trying to isolate an issue which might be Mac specific with
Boost.Asio, but I need to be certain that it's not a bug somewhere
else. As it stands I don't have access to a Linux nor Windows machine
at the moment, and feedback on build issues and test failures would be
most appreciated.

Thanks in advance and I do look forward to hearing from you soon!

-- 
Dean Michael Berris
http://about.me/deanberris

Re: [cpp-netlib-devel] netlib 0.8.1 with boost 1.46 (visual studio 2010)

[Dean M. B. <mik...@gm...>]

On Fri, Apr 1, 2011 at 2:55 AM, Sébastien Taylor <me...@st...> wrote:
> I never got a chance to test the boost svn version to see if the issue
> was resolved but Dean felt that the fix was in there so that might be
> a good place to start.
>

I have been developing against Boost's SVN trunk but I don't see the
fix in Spirit there yet. The problem is really caused by the re-do of
the attribute system in Spirit, which unfortunately breaks the code
that used to rely on some extension points in Spirit.

I have a choice between fixing the grammar in the URI implementation
and having the Spirit guys look at and fix the Spirit implementation.
I'm leaning towards fixing the grammar more than having Spirit fix the
breakage, but I am coordinating with them too.

> On Thu, Mar 31, 2011 at 12:44 PM, Andy <joi...@ho...> wrote:
>> I am having the same issue with boost 1.46 and netlib-8.1.  I am trying to
>> compile hello_world_client.cpp with instruction provided on your website.  Is
>> there workaround it till the problem is fixed?
>>

For the MPL stuff, yes there's workaround -- you can try `#include
<boost/mpl/or.hpp>` before any cpp-netlib includes. For the Spirit
stuff, unfortunately no workaround for that yet.

HTH

-- 
Dean Michael Berris
http://about.me/deanberris

Re: [cpp-netlib-devel] netlib 0.8.1 with boost 1.46 (visual studio 2010)

[Sébastien T. <me...@st...>]

I never got a chance to test the boost svn version to see if the issue
was resolved but Dean felt that the fix was in there so that might be
a good place to start.

On Thu, Mar 31, 2011 at 12:44 PM, Andy <joi...@ho...> wrote:
> I am having the same issue with boost 1.46 and netlib-8.1.  I am trying to
> compile hello_world_client.cpp with instruction provided on your website.  Is
> there workaround it till the problem is fixed?
>
>
>
> ------------------------------------------------------------------------------
> Create and publish websites with WebMatrix
> Use the most popular FREE web apps or write code yourself;
> WebMatrix provides all the features you need to develop and
> publish your website. http://p.sf.net/sfu/ms-webmatrix-sf
> _______________________________________________
> Cpp-netlib-devel mailing list
> Cpp...@li...
> https://lists.sourceforge.net/lists/listinfo/cpp-netlib-devel
>

Re: [cpp-netlib-devel] netlib 0.8.1 with boost 1.46 (visual studio 2010)

[Andy <joi...@ho...>]

I am having the same issue with boost 1.46 and netlib-8.1.  I am trying to
compile hello_world_client.cpp with instruction provided on your website.  Is
there workaround it till the problem is fixed?

Re: [cpp-netlib-devel] BOOST_NO_EXCEPTIONS (and -fno-exceptions)

[Dean M. B. <mik...@gm...>]

On Thu, Mar 31, 2011 at 6:11 AM, Hochhaus, Andrew
<aho...@sa...> wrote:
> Hi Dean,
>
> On Mon, Mar 14, 2011 at 12:13 AM, Dean Michael Berris
> <mik...@gm...> wrote:
>>> On Sun, Mar 13, 2011 at 7:17 PM, Dean Michael Berris
>>> <mik...@gm...> wrote:
>>> The only other thing that I can think of is to "#ifndef
>>> BOOST_NO_EXCEPTIONS" comment out all "try {" clauses and "} catch  {
>>> ... }" blocks in cpp-netlib. This should be a safe assumption as the
>>> user supplied throw_exception is never allowed to return (so no need
>>> to handle error conditions).
>>>
>>> An example of commenting out the "try ... catch" stuff can be seen in
>>> this patch to the thread library.
>>>
>>> https://svn.boost.org/trac/boost/attachment/ticket/2100/boost_thread_BOOST_NO_EXCEPTIONS_20110306.diff
>>>
>>> With those two changes, I believe that compilation should succeed with
>>> -fno-exceptions.
>>>
>>
>> Interesting. I think I need to think about that a little. There are a
>> few places where the exception mechanism is relied upon to convey
>> information to the user of the library -- like when an HTTP GET
>> actually fails, etc. I think it ought to be able to design interfaces
>> that take an lvalue reference to an optional error, or maybe just a
>> boost::system::error_code (maybe it's time that I implement a
>> boost::network::error_code that plays well with Boost.System much like
>> how Boost.Asio has the same).
>>
>> It might not be as simple as that too as with the asynchonous client
>> implementation relies on portable Boost.Exception types that are
>> encapsulated in Boost.Thread futures.
>
> I just learned about BOOST_{TRY,CATCH,CATCH_END,RETHREOW} from:
>
> third-party/boost/boost/detail/no_exceptions_support.hpp
>
> I think using these (combined with BOOST_THROW_EXCEPTION) is
> preferable to #ifndef BOOST_NO_EXCEPTIONS guarding that I previously
> suggested. Just thought I would mention them as they may be helpful.
>

Cool!

If you can give me a pull request that would allow me to integrate
your changes in the library that would be *awesome*.

Thanks Andy!

-- 
Dean Michael Berris
http://about.me/deanberris

Re: [cpp-netlib-devel] BOOST_NO_EXCEPTIONS (and -fno-exceptions)

[Hochhaus, A. <aho...@sa...>]

Hi Dean,

On Mon, Mar 14, 2011 at 12:13 AM, Dean Michael Berris
<mik...@gm...> wrote:
>> On Sun, Mar 13, 2011 at 7:17 PM, Dean Michael Berris
>> <mik...@gm...> wrote:
>> The only other thing that I can think of is to "#ifndef
>> BOOST_NO_EXCEPTIONS" comment out all "try {" clauses and "} catch  {
>> ... }" blocks in cpp-netlib. This should be a safe assumption as the
>> user supplied throw_exception is never allowed to return (so no need
>> to handle error conditions).
>>
>> An example of commenting out the "try ... catch" stuff can be seen in
>> this patch to the thread library.
>>
>> https://svn.boost.org/trac/boost/attachment/ticket/2100/boost_thread_BOOST_NO_EXCEPTIONS_20110306.diff
>>
>> With those two changes, I believe that compilation should succeed with
>> -fno-exceptions.
>>
>
> Interesting. I think I need to think about that a little. There are a
> few places where the exception mechanism is relied upon to convey
> information to the user of the library -- like when an HTTP GET
> actually fails, etc. I think it ought to be able to design interfaces
> that take an lvalue reference to an optional error, or maybe just a
> boost::system::error_code (maybe it's time that I implement a
> boost::network::error_code that plays well with Boost.System much like
> how Boost.Asio has the same).
>
> It might not be as simple as that too as with the asynchonous client
> implementation relies on portable Boost.Exception types that are
> encapsulated in Boost.Thread futures.

I just learned about BOOST_{TRY,CATCH,CATCH_END,RETHREOW} from:

third-party/boost/boost/detail/no_exceptions_support.hpp

I think using these (combined with BOOST_THROW_EXCEPTION) is
preferable to #ifndef BOOST_NO_EXCEPTIONS guarding that I previously
suggested. Just thought I would mention them as they may be helpful.

Thanks,
-Andy

Re: [cpp-netlib-devel] VS 2010 and cpp-netlib 0.8.1 errors

[Dean M. B. <mik...@gm...>]

Hi Donald,

On Tue, Mar 29, 2011 at 4:28 PM, Donald Carpenter <do...@xm...> wrote:
> Came across cpp-netlib via Pion. Looks promising. However, in trying to get
> up and running, I tried compiling the following simple no-brainer code using
> VS2010 together with Boost 1.46.1 (I also tested again Boost Subversion
> repos only to get the same errors). In both cases, I had copied
> cpp-netlib-0.8.1/{boost,lib} into the boost hierarchy.
>

What version of cpp-netlib are you using? Does this happen in 0.8.1?

>
> For whatever reason, I added the global netlib header to see if that made
> any difference. When I uncommented the #include <boost/network.hpp> line I
> get different errors:
>
> Error 1    error C2752:
> 'boost::spirit::traits::is_weak_substitute<T,Expected>' : more than one
> partial specialization matches the template argument list
> \projects\boost\boost\mpl\aux_\preprocessed\plain\or.hpp 51
>
> Error 2    error C2752:
> 'boost::spirit::traits::is_weak_substitute<T,Expected>' : more than one
> partial specialization matches the template argument list
> \projects\boost\boost\mpl\aux_\preprocessed\plain\or.hpp 51
>

Yeah, this one looks like it broke in 1.46.1 -- I'm going to try and
address this soon as I can't move on developing 0.9-devel to work with
1.46.1 nor the trunk. I can say though that this doesn't happen with
Boost 1.45 -- can you try with that instead?

At any rate I'll keep everyone posted on my progress with this issue.

>
> Can someone shed some light on this ?
>

Hopefully I can simplify the grammar of the URI parser to not use the
Spirit extension points to avoid the issues with Spirit. About the or_
thing, I would tell you to try and use 0.8.1 instead.

Thanks and I look forward to hearing from you soon!

BTW, you can get the latest from
https://github.com/cpp-netlib/cpp-netlib/downloads -- have a good one
and I hope this helps!

-- 
Dean Michael Berris
http://about.me/deanberris

[cpp-netlib-devel] VS 2010 and cpp-netlib 0.8.1 errors

[Donald C. <do...@xm...>]

Came across cpp-netlib via Pion. Looks promising. However, in trying to get
up and running, I tried compiling the following simple no-brainer code using
VS2010 together with Boost 1.46.1 (I also tested again Boost Subversion
repos only to get the same errors). In both cases, I had copied
cpp-netlib-0.8.1/{boost,lib} into the boost hierarchy.

 

//#include <boost/network.hpp>

#include <boost/network/include/http/server.hpp>

 

int main(int argc, char* argv[]) { return 0; }

 

Compiling this gave the following errors in VS2010:

 

Error 1 error C2065: 'or_' : undeclared identifier
\projects\boost\boost\network\protocol\http\message\traits\version.hpp    28

Error 2    error C2275: 'boost::network::is_sync<Message::tag>' : illegal
use of this type as an expression
\projects\boost\boost\network\protocol\http\message\traits\version.hpp    29

Error 3    error C2974: 'boost::mpl::if_' : invalid template argument for
'T1', type expected
\projects\boost\boost\network\protocol\http\message\traits\version.hpp    32

Error 4    error C2977: 'boost::mpl::if_' : too many template arguments
\projects\boost\boost\network\protocol\http\message\traits\version.hpp    35

Error 5    error C2143: syntax error : missing ',' before '>'
\projects\boost\boost\network\protocol\http\message\traits\version.hpp    36

 

For whatever reason, I added the global netlib header to see if that made
any difference. When I uncommented the #include <boost/network.hpp> line I
get different errors:

Error 1    error C2752:
'boost::spirit::traits::is_weak_substitute<T,Expected>' : more than one
partial specialization matches the template argument list
\projects\boost\boost\mpl\aux_\preprocessed\plain\or.hpp 51

Error 2    error C2752:
'boost::spirit::traits::is_weak_substitute<T,Expected>' : more than one
partial specialization matches the template argument list
\projects\boost\boost\mpl\aux_\preprocessed\plain\or.hpp 51

 

 

Can someone shed some light on this ?

 

--

DonC.

 

 

 

Re: [cpp-netlib-devel] load a certificate

[Dean M. B. <mik...@gm...>]

On Sat, Mar 26, 2011 at 10:09 AM, Dean Michael Berris
<mik...@gm...> wrote:
> On Fri, Mar 25, 2011 at 11:34 PM, Benjamin Weibel <we...@tr...> wrote:
>>
>> I've already had a look at the source code and saw that the method
>> "https_sync_connection" takes a string-argument "certificate_filename",
>> but I saw no way to pass such a filename at the high-level-api.
>> Is this parameter solely used to load system-default CA-certificates?
>> If not, could you please explain to me how I can load a certificate from
>> file or memory?
>>
>
> Oops, that filename should actually be a list of filenames; also there
> should be a way in the server's constructor to pass the parameter
> properly into the connection objects. That has still yet to be done,
> but can actually be patched easily.
>

And because I thought it was worth the effort, I made changes in the
0.9-devel branch that makes this possible. Documentation about using
the new constructor parameters (_openssl_certificate and
_openssl_verify_path) pending, but the idea is simple:

  using namespace boost::network::http;
  client c(_openssl_certificate="/tmp/my-cert");
  client::request req("https://some.server.com/");
  client::response res = c.get(req);

I've just pushed this to 0.9-devel so you should be able to start
using this in your application.

HTH!

-- 
Dean Michael Berris
http://about.me/deanberris

Re: [cpp-netlib-devel] load a certificate

[Dean M. B. <mik...@gm...>]

On Fri, Mar 25, 2011 at 11:34 PM, Benjamin Weibel <we...@tr...> wrote:
> Hi all,
>
> first I want to thank you for the great work you did writing this
> library! It really helps me a lot!

You're welcome. :)

> I have been working on integrating openssl and writing an own
> minimalistic dummy-HTTP-client when I discovered this awesome projekt
> just in time.
>
> I work on a cross-platform library and that library has to communicate
> with one specific HTTPS server.
> To authenticate the server, we receive (in advance) an X509-certificate
> to build into our software.
>
> I would like to ask you if it is possible to load one or more specific
> CA-certificates for a HTTPS client connection (doesn't matter if from
> file or from memory).
>
> I've already had a look at the source code and saw that the method
> "https_sync_connection" takes a string-argument "certificate_filename",
> but I saw no way to pass such a filename at the high-level-api.
> Is this parameter solely used to load system-default CA-certificates?
> If not, could you please explain to me how I can load a certificate from
> file or memory?
>

Oops, that filename should actually be a list of filenames; also there
should be a way in the server's constructor to pass the parameter
properly into the connection objects. That has still yet to be done,
but can actually be patched easily.

I was in the middle of adding support for this when I moved my focus
to the server side of the HTTP implementation. At any rate I'll see
whether I can make that happen as soon as I get a better development
machine than the one I have right now.

> Thank you very much in advance and good luck with this project!
>

Thank you for the feedback too -- I'll look at adding this capability
from the client constructor to allow for adding a range of filenames
or even just areas in memory that point to the contents of the
certificates.

If you can, please file an issue over at
https://github.com/cpp-netlib/cpp-netlib/issues so that I can keep
track of this feature.

> Greetings from Switzerland,
>

Greetings from the Philippines!

Thanks Benjamin!

-- 
Dean Michael Berris
http://about.me/deanberris

[cpp-netlib-devel] load a certificate

[Benjamin W. <we...@tr...>]

Hi all,

first I want to thank you for the great work you did writing this 
library! It really helps me a lot!
I have been working on integrating openssl and writing an own 
minimalistic dummy-HTTP-client when I discovered this awesome projekt 
just in time.

I work on a cross-platform library and that library has to communicate 
with one specific HTTPS server.
To authenticate the server, we receive (in advance) an X509-certificate 
to build into our software.

I would like to ask you if it is possible to load one or more specific 
CA-certificates for a HTTPS client connection (doesn't matter if from 
file or from memory).

I've already had a look at the source code and saw that the method 
"https_sync_connection" takes a string-argument "certificate_filename",
but I saw no way to pass such a filename at the high-level-api.
Is this parameter solely used to load system-default CA-certificates?
If not, could you please explain to me how I can load a certificate from 
file or memory?

Thank you very much in advance and good luck with this project!

Greetings from Switzerland,

Benjamin Weibel

Re: [cpp-netlib-devel] Proxy support for http::client?

[Dean M. B. <mik...@gm...>]

On Fri, Mar 25, 2011 at 7:10 PM, Stanimir Mladenov
<sta...@gm...> wrote:
> I saw a question in the mailing list archive about proxy support for
> the client back to year 2008.
> Is it possible now to setup a proxy address and port for the client?
>

At the moment, not yet.

It should be easy to implement but I don't have enough time and
resources to test this just yet. Maybe others might want to send in
patches to make this happen on the 0.9-devel branch too.

HTH

-- 
Dean Michael Berris
http://about.me/deanberris

[cpp-netlib-devel] Proxy support for http::client?

[Stanimir M. <sta...@gm...>]

I saw a question in the mailing list archive about proxy support for
the client back to year 2008.
Is it possible now to setup a proxy address and port for the client?

Re: [cpp-netlib-devel] async keepalive and other things

[Dean M. B. <mik...@gm...>]

On Tue, Mar 22, 2011 at 2:59 AM, Rick Richardson
<ric...@gm...> wrote:
>
>
> On Mon, Mar 21, 2011 at 10:08 AM, Dean Michael Berris
> <mik...@gm...> wrote:
>>
>> Hi Rick!
>>
>> ...
>>
>> Okay, but how do you deal with the parsing? Do you reset the parser
>> state per connection?
>>
> Yes. That is my current approach. Oddly enough, I have it partially working.
> I have a test which pipelines two requests. Currently it gets the first
> request and responds correctly. The 2nd request is read and processed by
> handle_read_data but is not correctly parsed, and I'm not sure why yet. I
> just need to spend a bit more time with it.
>

There is a parser type that does the incremental parsing of incoming
data. Maybe all that needs to be done is to check to see if the
parser's state is reset properly every time once a GET request is
parsed appropriately. There may be other complications but I was
thinking of a way to make it happen leveraging object lifetimes, which
may or may not work -- I still have to explore that possibility.

That said I'll wait to see your pull request(s) soon. :)

>
>>
>> Asynchronous server handlers can call connection.write(...) multiple
>> times per request. The calls to connection.read(...) in the case of a
>> handler that wants to read up data from the body of a POST/PUT request
>> will also have to deal with the boundary issues -- granted that the
>> HTTP/1.1 spec doesn't allow pipelining for POST/PUT requests, we'd
>> want to be able to let the handler recover or at least deal with
>> invalid data appropriately.
>
> Yea, this is a tricky point because we're basically relying on the top layer
> of the stack to complete parsing for us.
> It is good that the spec disallows pipelining posts and puts, but either way
> I would like to actually process the entire body in the Connection object
> because there is *tons* of complicating parsing logic I would like to put in
> for POSTS:
> There is a new type of socket exhaustion attack out which correctly
> (according to the HTTP spec) slowly dribbles in data in a multi-part post.
>  I would like to put counter-measures to defend against such things as well
> as other DOS attacks
> (see http://www.ecl-labs.org/2011/03/17/roboo-http-mitigator.html )
>

Right. Implementing the POST/PUT body parser was something I wasn't
thinking of incorporating in the connection, opting to delegate those
to helper/utility types/functions. However I see that having those in
the connection's implementation would do a lot of good to a lot of
people so I definitely hope to see how you (and others) would take
this task on. :)

>>
>> Have you handled this? (Granted, I haven't looked at your patch just yet).
>
> I don't handle this well enough yet, apparently.

That's fine. :)

>>
>> That sounds alright, although we might have to guard for a race
>> somehow there when the handler is actually reading from the same
>> connection (in the case of a POST/PUT handler). Basically we have to
>> know whether the timeout happened while the handler is waiting for a
>> POST/PUT body or whether the timeout happened while at the boundaries
>> of the GET requests.
>
> If we process the multipart POST body in the Connection object, we can
> re-set the keepalive timer for every packet of data we receive, (if the
> client waits more than TIMEOUT seconds between each packet, I would like to
> kick them off anyways)

Yes, I agree. Although that would mean making the Connection object's
implementation a lot hairier than it already is. Just thinking about
incremental parsing support for multi-part MIME on the *server* side
is making me want to block off a good week and just hole up with a
powerful machine to get that figured out. :D

>>
>> > I haven't bothered to fork cpp-netlib on github so I don't have a
>> > checkin to
>> > make, but I did create a 'git diff' from  boost/   I have attached it
>> > here
>> > for your perusal.
>> > Please let me know your preferred approach for reviewing and accepting
>> > patches (I am assuming a pull request, which I can get around to)
>> >
>>
>> You assume right that I would much rather prefer looking at diffs over
>> at Github, so a pull request is going to be most appreciated.
>>
> I will do that.. That patch is already outdated anyways. I made a bunch of
> changes.
>

Cool, I'll look forward to the pull requests then. :)

> Flush is being called. I did find one of the problems.  Most clients expect
> that the header arrive in a single packet. The problem in AsyncConnection is
> that it was sending the first line as a single packet, then following up
> with the rest of the header.  I altered the code (in a later patch than what
> I sent) to add the first line straight into the header buffer. It fixed
> things.
>

Interesting. So how does it change the interface though? And how do
you intend to support sending '100 Continue' responses from the
handler with this change?

>>
>> As far as the spec goes, the server may send a "Connection: close" at
>> any time to signal to the client that it will initiate a close to
>> complete the transfer. Also if you're using Transfer-Encoding:
>> Chunked, I think there's a special sequence to indicate the end of the
>> response -- I need to cite that part, although an "empty" chunk
>> usually signals the end of the message.
>>
>> If you just blindly keep the connection alive though, then the answer
>> is the last request from the client side *should* have a "Connection:
>> close" to let the server know that this is the last request in the
>> pipeline. If it doesn't, the client may have to wait indefinitely for
>> the server to close the connection while the server is stuck in a
>> state where it would assume that there might be more requests that may
>> come into the connection.
>
> Ah. That last bit is a really useful piece of information. If all clients
> conform to that, then it will make this job much easier.

It's recommended by the spec that this be the mode of operation, but
if you think about it though just having a 'Content-Length' field in
the HTTP response should allow the client to not have to wait for the
connection to be closed regardless of whether requests were pipelined
or not. Does that make sense?

>>
>> Thank you too and I definitely hope to hear from you soon!
>>
>
> Thanks. I would like to experiment with adding request body processing, even
> if it doesn't end up in the AsyncConnection, there is a lot of work to be
> done here.   I am not looking forward to CHUNK processing yet, but I will
> get to that this weekend (My day job and 2 kids keep me rather occupied
> during the week)
>

Cool. :)

I know how that goes -- with family and having a day job. In my case
though lack of a powerful enough machine to do template-heavy
development on is killing my productivity a lot. Hopefully I can
remedy that soon now that the wife has agreed to invest on a more
modern machine now. :D

Have a good one Rick and I look forward to your pull requests soon!

-- 
Dean Michael Berris
http://about.me/deanberris

Re: [cpp-netlib-devel] async keepalive and other things

[Rick R. <ric...@gm...>]

On Mon, Mar 21, 2011 at 10:08 AM, Dean Michael Berris <
mik...@gm...> wrote:

> Hi Rick!
>
> ...
>
> Okay, but how do you deal with the parsing? Do you reset the parser
> state per connection?
>
> Yes. That is my current approach. Oddly enough, I have it partially
working. I have a test which pipelines two requests. Currently it gets the
first request and responds correctly. The 2nd request is read and processed
by handle_read_data but is not correctly parsed, and I'm not sure why yet. I
just need to spend a bit more time with it.



> Asynchronous server handlers can call connection.write(...) multiple
> times per request. The calls to connection.read(...) in the case of a
> handler that wants to read up data from the body of a POST/PUT request
> will also have to deal with the boundary issues -- granted that the
> HTTP/1.1 spec doesn't allow pipelining for POST/PUT requests, we'd
> want to be able to let the handler recover or at least deal with
> invalid data appropriately.
>

Yea, this is a tricky point because we're basically relying on the top layer
of the stack to complete parsing for us.

It is good that the spec disallows pipelining posts and puts, but either way
I would like to actually process the entire body in the Connection object
because there is *tons* of complicating parsing logic I would like to put in
for POSTS:
There is a new type of socket exhaustion attack out which correctly
(according to the HTTP spec) slowly dribbles in data in a multi-part post.
 I would like to put counter-measures to defend against such things as well
as other DOS attacks (see
http://www.ecl-labs.org/2011/03/17/roboo-http-mitigator.html )



>
> Have you handled this? (Granted, I haven't looked at your patch just yet).
>

I don't handle this well enough yet, apparently.

>
> > If there are no pending additional requests, then the async_read_some
> will
> > block indefinitely.
> > The second phase of the plan, then, is to register a deadline timer at
> start
> > of the new connection. Every new request that gets read will kick the
> > deadline timer back up to its max timeout time.  When the timer does go
> off,
> > it cancels any operations that are pending on the Connection's socket.
> > This does two things, first and foremost, it keeps a hold of the
> > Connection's shared ptr, so that it is guaranteed not to be destroyed
> while
> > the keepalive is in effect.  Secondly, it quite obviously will free the
> > async_read_some from its wait.
>
> That sounds alright, although we might have to guard for a race
> somehow there when the handler is actually reading from the same
> connection (in the case of a POST/PUT handler). Basically we have to
> know whether the timeout happened while the handler is waiting for a
> POST/PUT body or whether the timeout happened while at the boundaries
> of the GET requests.
>

If we process the multipart POST body in the Connection object, we can
re-set the keepalive timer for every packet of data we receive, (if the
client waits more than TIMEOUT seconds between each packet, I would like to
kick them off anyways)


> > I haven't bothered to fork cpp-netlib on github so I don't have a checkin
> to
> > make, but I did create a 'git diff' from  boost/   I have attached it
> here
> > for your perusal.
> > Please let me know your preferred approach for reviewing and accepting
> > patches (I am assuming a pull request, which I can get around to)
> >
>
> You assume right that I would much rather prefer looking at diffs over
> at Github, so a pull request is going to be most appreciated.
>
> I will do that.. That patch is already outdated anyways. I made a bunch of
changes.



> > 3.  In attempting to benchmark my changes.  I did note that yes,
> keepalive
> > works, and also that it breaks httperf.  My problem is that httperf
> doesn't
> > seem to recognize a completed response unless the server closes the
> > connection.  It claims to be a 1.1 compliant tester, and when it sends
> > requests it doesn't mention a connection: close in its headers, so I'm
> > assuming it should expect the connection to remain open.
> >
>
> I think you need to flush the data from the server end. I forget if
> TCP_NODELAY is turned on by default and whether httperf (I really
> don't know what that is, you have to pardon me I use Apache's ab for
> performance testing) actually has a short timeout setting for data
> transfer or whether the buffers are small enough to avoid buffer
> starvation. How much data are you sending from the server side during
> the benchmarks?
>
> Flush is being called. I did find one of the problems.  Most clients expect
that the header arrive in a single packet. The problem in AsyncConnection is
that it was sending the first line as a single packet, then following up
with the rest of the header.  I altered the code (in a later patch than what
I sent) to add the first line straight into the header buffer. It fixed
things.


> > Does anyone know the correct procedure here, or why it might be thinking
> > that the responses have not completed?  My concern here is that there is
> > some nuance of the HTTP 1.1 spec that I may have missed which allows
> servers
> > to delineate between responses since, with pipelining, multiple responses
> > may come all at once over the same connection.
> >
>
> As far as the spec goes, the server may send a "Connection: close" at
> any time to signal to the client that it will initiate a close to
> complete the transfer. Also if you're using Transfer-Encoding:
> Chunked, I think there's a special sequence to indicate the end of the
> response -- I need to cite that part, although an "empty" chunk
> usually signals the end of the message.
>
> If you just blindly keep the connection alive though, then the answer
> is the last request from the client side *should* have a "Connection:
> close" to let the server know that this is the last request in the
> pipeline. If it doesn't, the client may have to wait indefinitely for
> the server to close the connection while the server is stuck in a
> state where it would assume that there might be more requests that may
> come into the connection.
>

Ah. That last bit is a really useful piece of information. If all clients
conform to that, then it will make this job much easier.

>
> Thank you too and I definitely hope to hear from you soon!
>
>
Thanks. I would like to experiment with adding request body processing, even
if it doesn't end up in the AsyncConnection, there is a lot of work to be
done here.   I am not looking forward to CHUNK processing yet, but I will
get to that this weekend (My day job and 2 kids keep me rather occupied
during the week)