Menu
▾
▴
cpp-netlib-devel
|
From: Rodrigo M. <rod...@gm...> - 2008-10-30 21:14:16
|
Hello there, I just found cpp-netlib while searching for an Application Layer library for the excellent Boost ASIO library. I have been trying to ask about this on the Boost Users mailing list, but I still have no answer on the issue of protocol Muxing (encoding and decoding). I see that you have an HTTP protocol parser example (and subsystem inside the library) but what I'm looking for is a generic parser class hierarchy and/or templates that would allow me to treat binary protocols. So simply put: What are your plans for this? Is this already possible with the current version? Thank you for your input and congratulations on your initiative. Kind regards, Rodrigo Madera |
|
From: Divye K. <div...@gm...> - 2008-10-31 02:14:30
|
Hello Rodrigo, On Fri, Oct 31, 2008 at 2:44 AM, Rodrigo Madera <rod...@gm...>wrote: > Hello there, > > I just found cpp-netlib while searching for an Application Layer library > for the excellent Boost ASIO library. > > I have been trying to ask about this on the Boost Users mailing list, but I > still have no answer on the issue of protocol Muxing (encoding and > decoding). > Could you provide a link to that message? > > I see that you have an HTTP protocol parser example (and subsystem inside > the library) but what I'm looking for is a generic parser class hierarchy > and/or templates that would allow me to treat binary protocols. > The http protocol support is a kind of subsystem in the project. There is nothing preventing you from rolling out your own protocol support. The base classe: basic_message provides a basic framework which you can specialize on the tag type to support any kind of protocol. Currently, the http module is using that approach. There are many directives, traits, etc. that operate on the basic_message directly, thus allowing you to approach protocol support speedily. > > So simply put: What are your plans for this? Is this already possible with > the current version? > > Thank you for your input and congratulations on your initiative. > > Kind regards, > Rodrigo Madera > > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's > challenge > Build the coolest Linux based applications with Moblin SDK & win great > prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > Cpp-netlib-devel mailing list > Cpp...@li... > https://lists.sourceforge.net/lists/listinfo/cpp-netlib-devel > > -- An idealist is one who, on noticing that a rose smells better than a cabbage, concludes that it will also make better soup. H. L. Mencken (1880 - 1956) My official web site: http://people.iitr.ernet.in/shp/061305/ Webmaster: http://www.drkapoorsclinic.com Blog: http://divyekapoor.blogspot.com |
|
From: Rodrigo M. <rod...@gm...> - 2008-10-31 11:51:59
|
Thank you for your reply. However, I don't need a whole set of libraries. I already have Boost's ASIO. I'll do the code using what you proposed and we can go from there. There will be no other way, unfortunately. Here is the link you asked for: http://article.gmane.org/gmane.comp.lib.boost.user/41209 Also, please note that if you ever needed to deal with a binary protocol using ASIO, you should have needed this kind of implementation code. Regards, Rodrigo On Fri, Oct 31, 2008 at 12:14 AM, Divye Kapoor <div...@gm...>wrote: > Hello Rodrigo, > > On Fri, Oct 31, 2008 at 2:44 AM, Rodrigo Madera <rod...@gm...>wrote: > >> Hello there, >> >> I just found cpp-netlib while searching for an Application Layer library >> for the excellent Boost ASIO library. >> >> I have been trying to ask about this on the Boost Users mailing list, but >> I still have no answer on the issue of protocol Muxing (encoding and >> decoding). >> > > Could you provide a link to that message? > > >> >> I see that you have an HTTP protocol parser example (and subsystem inside >> the library) but what I'm looking for is a generic parser class hierarchy >> and/or templates that would allow me to treat binary protocols. >> > > The http protocol support is a kind of subsystem in the project. There is > nothing preventing you from rolling out your own protocol support. The base > classe: basic_message provides a basic framework which you can specialize on > the tag type to support any kind of protocol. Currently, the http module is > using that approach. There are many directives, traits, etc. that operate on > the basic_message directly, thus allowing you to approach protocol support > speedily. > > > >> >> So simply put: What are your plans for this? Is this already possible with >> the current version? >> >> Thank you for your input and congratulations on your initiative. >> >> Kind regards, >> Rodrigo Madera >> >> >> ------------------------------------------------------------------------- >> This SF.Net email is sponsored by the Moblin Your Move Developer's >> challenge >> Build the coolest Linux based applications with Moblin SDK & win great >> prizes >> Grand prize is a trip for two to an Open Source event anywhere in the >> world >> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >> _______________________________________________ >> Cpp-netlib-devel mailing list >> Cpp...@li... >> https://lists.sourceforge.net/lists/listinfo/cpp-netlib-devel >> >> > > > -- > An idealist is one who, on noticing that a rose smells better than a > cabbage, concludes that it will also make better soup. > H. L. Mencken (1880 - 1956) > My official web site: http://people.iitr.ernet.in/shp/061305/ > Webmaster: http://www.drkapoorsclinic.com > Blog: http://divyekapoor.blogspot.com > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's > challenge > Build the coolest Linux based applications with Moblin SDK & win great > prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > Cpp-netlib-devel mailing list > Cpp...@li... > https://lists.sourceforge.net/lists/listinfo/cpp-netlib-devel > > |
|
From: Divye K. <div...@gm...> - 2008-10-31 18:50:11
|
Hello Rodrigo, On Fri, Oct 31, 2008 at 5:01 PM, Rodrigo Madera <rod...@gm...>wrote: > Thank you for your reply. > No need for thanks. Glad to help. :-) > > However, I don't need a whole set of libraries. I already have Boost's > ASIO. > Just like the STL is good to have along with vanilla C++, cpp-netlib is good to have along with Boost ASIO, in fact, we build on Boost ASIO to provide a higher level of functionality. > > I'll do the code using what you proposed and we can go from there. There > will be no other way, unfortunately. > > Here is the link you asked for: > http://article.gmane.org/gmane.comp.lib.boost.user/41209 > Although I haven't used Apache MINA, from what I could gather from your post, I would say that you are looking for a generic way to handle protocols by simply plugging in protocol message handlers. Am I correct? Given that we are looking to support a wide variety of protocols (which might be stateful/stateless or have other interdependencies) such a structure has not been used for the design as it might prove to be too brittle for extension. However, our current design (explained at http://cpp-netlib.wiki.sourceforge.net/ ) does a neat job of giving you maximum flexibility in the implementation of a protocol. The syntax you indicated can be easily implemented in a specialized client class. If you look at how http::client is implemented (boost/network/protocol/http/client.hpp) you will see how a basic_message is interpreted according to the specifications of the http protocol and the response is converted into another basic_message. In your scenario, you could defer the parsing of the protocol "headers" to function objects or the like. > Also, please note that if you ever needed to deal with a binary protocol > using ASIO, you should have needed this kind of implementation code. > Currently, we support binary file transfer over HTTP. Given that we don't touch the payload of the message, we have not come across any issues in the implementation of binary transfer. However, if you are interested, please browse the code and see if you can help us improve; we'll be glad of your help. > > Regards, > Rodrigo > > > Hope that helped, Divye -- An idealist is one who, on noticing that a rose smells better than a cabbage, concludes that it will also make better soup. H. L. Mencken (1880 - 1956) My official web site: http://people.iitr.ernet.in/shp/061305/ Webmaster: http://www.drkapoorsclinic.com Blog: http://divyekapoor.blogspot.com |
|
From: Rodrigo M. <rod...@gm...> - 2008-11-01 22:06:31
|
Thanks again. However the current architecture is not generic enough. It feels like HTTP was the only protocol thought about during it's design. I'll post my progress here. Rodrigo On Fri, Oct 31, 2008 at 4:50 PM, Divye Kapoor <div...@gm...> wrote: > Hello Rodrigo, > > On Fri, Oct 31, 2008 at 5:01 PM, Rodrigo Madera <rod...@gm...>wrote: > >> Thank you for your reply. >> > > No need for thanks. Glad to help. :-) > > >> >> However, I don't need a whole set of libraries. I already have Boost's >> ASIO. >> > > Just like the STL is good to have along with vanilla C++, cpp-netlib is > good to have along with Boost ASIO, in fact, we build on Boost ASIO to > provide a higher level of functionality. > > >> >> I'll do the code using what you proposed and we can go from there. There >> will be no other way, unfortunately. >> >> Here is the link you asked for: >> http://article.gmane.org/gmane.comp.lib.boost.user/41209 >> > > Although I haven't used Apache MINA, from what I could gather from your > post, I would say that you are looking for a generic way to handle protocols > by simply plugging in protocol message handlers. Am I correct? > > Given that we are looking to support a wide variety of protocols (which > might be stateful/stateless or have other interdependencies) such a > structure has not been used for the design as it might prove to be too > brittle for extension. However, our current design (explained at > http://cpp-netlib.wiki.sourceforge.net/ ) does a neat job of giving you > maximum flexibility in the implementation of a protocol. The syntax you > indicated can be easily implemented in a specialized client class. If you > look at how http::client is implemented > (boost/network/protocol/http/client.hpp) you will see how a basic_message is > interpreted according to the specifications of the http protocol and the > response is converted into another basic_message. > > In your scenario, you could defer the parsing of the protocol "headers" to > function objects or the like. > > > >> Also, please note that if you ever needed to deal with a binary protocol >> using ASIO, you should have needed this kind of implementation code. >> > > Currently, we support binary file transfer over HTTP. Given that we don't > touch the payload of the message, we have not come across any issues in the > implementation of binary transfer. However, if you are interested, please > browse the code and see if you can help us improve; we'll be glad of your > help. > > >> >> Regards, >> Rodrigo >> >> >> > Hope that helped, > Divye > > > > -- > An idealist is one who, on noticing that a rose smells better than a > cabbage, concludes that it will also make better soup. > H. L. Mencken (1880 - 1956) > My official web site: http://people.iitr.ernet.in/shp/061305/ > Webmaster: http://www.drkapoorsclinic.com > Blog: http://divyekapoor.blogspot.com > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's > challenge > Build the coolest Linux based applications with Moblin SDK & win great > prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > Cpp-netlib-devel mailing list > Cpp...@li... > https://lists.sourceforge.net/lists/listinfo/cpp-netlib-devel > > |
|
From: Dean M. B. <mik...@gm...> - 2008-11-02 19:36:42
|
Hi Rodrigo, Sorry it took a while for me to respond. Please see in-lined below. (BTW, next time please avoid overquoting and top-posting.) On Sun, Nov 2, 2008 at 6:06 AM, Rodrigo Madera <rod...@gm...> wrote: > Thanks again. > However the current architecture is not generic enough. It feels like HTTP > was the only protocol thought about during it's design. That's odd, the message abstraction is very generic -- it doesn't assume anything about the protocols that will be using it. The structure of a message is really very flexible, and using tags you can even completely revamp the way a basic_message<> will look like (and how the interface would be) for your context/protocol. It would be nice to know why you think the architecture is not generic enough when being as generic as possible is what the whole library was designed to be. > I'll post my progress here. Please do, this will be very interesting to us. Thanks and have a good day! -- Dean Michael C. Berris Software Engineer, Friendster, Inc. |
|
From: Rodrigo M. <rod...@gm...> - 2008-11-03 00:14:14
|
Hey there Dean, Basically I see three std::string elements in the message which already make it strange. Is there a reason for them that I don't realize? Also, what is top-posting and overquoting? Thanks, Rodrigo On Sun, Nov 2, 2008 at 5:36 PM, Dean Michael Berris <mik...@gm...>wrote: > Hi Rodrigo, > > Sorry it took a while for me to respond. Please see in-lined below. > > (BTW, next time please avoid overquoting and top-posting.) > > On Sun, Nov 2, 2008 at 6:06 AM, Rodrigo Madera <rod...@gm...> > wrote: > > Thanks again. > > However the current architecture is not generic enough. It feels like > HTTP > > was the only protocol thought about during it's design. > > That's odd, the message abstraction is very generic -- it doesn't > assume anything about the protocols that will be using it. > > The structure of a message is really very flexible, and using tags you > can even completely revamp the way a basic_message<> will look like > (and how the interface would be) for your context/protocol. > > It would be nice to know why you think the architecture is not generic > enough when being as generic as possible is what the whole library was > designed to be. > > > I'll post my progress here. > > Please do, this will be very interesting to us. > > Thanks and have a good day! > > -- > Dean Michael C. Berris > Software Engineer, Friendster, Inc. > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's > challenge > Build the coolest Linux based applications with Moblin SDK & win great > prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > Cpp-netlib-devel mailing list > Cpp...@li... > https://lists.sourceforge.net/lists/listinfo/cpp-netlib-devel > |
|
From: Dean M. B. <mik...@gm...> - 2008-11-03 05:33:13
|
Hi Rodrigo, On Mon, Nov 3, 2008 at 8:14 AM, Rodrigo Madera <rod...@gm...> wrote: > > Basically I see three std::string elements in the message which already make > it strange. Why is this strange? > Is there a reason for them that I don't realize? std::string is the easiest container to use that allows both string semantics and guarantee of contiguous storage (at least as far as I understand). Any other container would be too unwieldy to use especially for algorithms that deal with strings of characters. BTW, strings can contain binary data just fine (as long as you store the data (7-bit chars) as is). > Also, what is top-posting and overquoting? Top-posting is what you do (putting your reply at the top of the message), and overquoting is not snipping unnecessary contents of email messages accordingly. [snipped unnecessary email contents to avoid overquoting] -- Dean Michael C. Berris Software Engineer, Friendster, Inc. |
|
From: Rodrigo M. <rod...@gm...> - 2008-11-03 19:29:10
|
On Mon, Nov 3, 2008 at 3:32 AM, Dean Michael Berris <mik...@gm...>wrote: > Hi Rodrigo, > > On Mon, Nov 3, 2008 at 8:14 AM, Rodrigo Madera <rod...@gm...> > wrote: > > > > Basically I see three std::string elements in the message which already > make > > it strange. > > Why is this strange? > > > Is there a reason for them that I don't realize? > > std::string is the easiest container to use that allows both string > semantics and guarantee of contiguous storage (at least as far as I > understand). Any other container would be too unwieldy to use > especially for algorithms that deal with strings of characters. > > BTW, strings can contain binary data just fine (as long as you store > the data (7-bit chars) as is). > So are you saying that storing a binary block of data (say an image) into a string is okay? Boost has a buffer class made for this. If you want integration with boost you need to use boost. It makes no sense at all to put strings into every message. Regards, Rodrigo |
|
From: Glyn M. <gly...@gm...> - 2008-11-03 19:47:04
|
Hi Rodrigo,
2008/11/3 Rodrigo Madera <rod...@gm...>
>
> So are you saying that storing a binary block of data (say an image) into a
> string is okay?
> Boost has a buffer class made for this. If you want integration with boost
> you need to use boost.
>
> It makes no sense at all to put strings into every message.
>
It's possible to specialize the message to support different containers.
Something like this:
namespace boost { namespace network {
template <>
struct string<binary_parser_tag> {
typedef boost::array<T, 1024> type;
};
}}
Or whatever structure suits your needs. See
<boost/network/message/traits/string.hpp>
Glyn
|
|
From: Rodrigo M. <rod...@gm...> - 2008-11-03 22:19:15
|
>
> So are you saying that storing a binary block of data (say an image) into a
>> string is okay?
>> Boost has a buffer class made for this. If you want integration with boost
>> you need to use boost.
>>
>> It makes no sense at all to put strings into every message.
>>
>
> It's possible to specialize the message to support different containers.
> Something like this:
>
> namespace boost { namespace network {
> template <>
> struct string<binary_parser_tag> {
> typedef boost::array<T, 1024> type;
> };
> }}
>
> Or whatever structure suits your needs. See
> <boost/network/message/traits/string.hpp>
>
>
So why not make this a buffer (or array) directly?
Why have you choosen this string specialization path?
Thank you,
Rodrigo
|
|
From: Glyn M. <gly...@gm...> - 2008-11-04 08:41:11
|
Rodrigo, 2008/11/3 Rodrigo Madera <rod...@gm...> > > So why not make this a buffer (or array) directly? > Why have you choosen this string specialization path? > Because that's what we felt was the most common usage scenario, so this was accepted as the default implementation. Glyn |
|
From: Dean M. B. <mik...@gm...> - 2008-11-04 09:30:54
|
On Tue, Nov 4, 2008 at 6:19 AM, Rodrigo Madera <rod...@gm...> wrote: > > So why not make this a buffer (or array) directly? Because std::string works fine. > Why have you choosen this string specialization path? Because it's the simplest thing that could possibly work. Besides, if we used anything other than an std::string the implementation would be way more complicated than necessary. -- Dean Michael C. Berris Software Engineer, Friendster, Inc. |
|
From: Dean M. B. <mik...@gm...> - 2008-11-04 09:28:31
|
On Tue, Nov 4, 2008 at 12:03 AM, Rodrigo Madera <rod...@gm...> wrote: > > On Mon, Nov 3, 2008 at 3:32 AM, Dean Michael Berris <mik...@gm...> > wrote: >> >> >> BTW, strings can contain binary data just fine (as long as you store >> the data (7-bit chars) as is). > > So are you saying that storing a binary block of data (say an image) into a > string is okay? Yes, it's okay. > Boost has a buffer class made for this. If you want integration with boost > you need to use boost. > What "buffer" class are you talking about? Are you talking about Boost.Array? Because the last time I checked, Boost.Array is statically-sized and is not as flexible as an std::string. IMO, using something from the STL is "better" because you don't need to rely on Boost being there to use the library. I won't even be surprised if it was possible to package the header-only Boost libs with cpp-netlib as a "standalone" download (with the help perhaps of bcp). Of course if we're using Boost.Asio we rely on Boost.System -- but other than that I'm confident there shouldn't be a lot of other libraries cpp-netlib would rely on (of course aside from the STL). > It makes no sense at all to put strings into every message. > Why doesn't it make sense? A message would consist of bytes -- the easiest way to deal with bytes is to use std::string. If you needed a different representation, you can always specialize the basic_message<> to your tag and use your own storage representation. The documentation shows you how to do that IIRC. -- Dean Michael C. Berris Software Engineer, Friendster, Inc. |
|
From: Rodrigo M. <rod...@gm...> - 2008-11-04 12:12:30
|
> > What "buffer" class are you talking about? > The class boost::asio::buffer was made for this. I don't believe that this code can make it into Boost using strings for binary data. Anyways, it's your way of seeing things. I'll post my code later and we can discus approaches later. Thank you for your kind time, Rodrigo |
|
From: Dean M. B. <mik...@gm...> - 2008-11-04 12:42:53
|
On Tue, Nov 4, 2008 at 8:12 PM, Rodrigo Madera <rod...@gm...> wrote:
>> What "buffer" class are you talking about?
>
> The class boost::asio::buffer was made for this.
>
And we use boost::asio::buffer's internally in the protocol
implementations. The messages merely represent data -- treat them as
data objects. If you check the HTTP client code, you will see that we
use the correct buffer types there provided by Asio. Transporting
whatever was gotten from the wire from the buffer to the user of the
client is what the message is supposed to be -- nothing more, nothing
less.
> I don't believe that this code can make it into Boost using strings for
> binary data.
>
I don't think so, considering that _you can customize the storage in
the message class to whatever you like depending on the protocol
you're implementing_ I believe the current design is definitely more
superior than using unweildy buffers in the message representation.
Think of it this way:
Request Message -> Client -> Client performs actions
Response Message <- Client
The type of the request message is dependent on what client you're
using. You can't use an http::request object to instruct an SNMP
client -- this will fail at compile time *and is the intended design
of the library*.
So if you're implementing your own protocol, you define the type of
the message you're going to use as a request and/or as a response
object instance. This means you -- as the implementor -- define
precisely what types to use within the message object specialization
that you need. If you think you need fixed-size buffers (or sometimes
bit-fields work too) then that's entirely up to you.
For instance, you can even change the signature of your specialization
of the basic_message class to reflect the signatures that you need.
For example, if you're thinking of using tuples of booleans to set
flags, then you can do that yourself:
template <>
struct basic_message<my_protocol_tag> {
void header(tuple<bool, bool, bool> flags);
tuple<bool, bool, bool> header();
};
This makes your specialization of the message class even more specific
to your protocol.
That means, you may choose to use the default (that uses strings) or
you can write your own specialization depending on the tag you use to
distinguish your protocol from the other protocols.
> Anyways, it's your way of seeing things.
Actually, it's not just my way of seeing things. The design decision is:
1. Since it's the most common use case for most higher level protocols
like HTTP, SMTP, XMPP, IRC, then it's the most convenient thing to use
-- and therefore what the default implementation uses.
2. The design of the whole infrastructure is meant to be extensible
through the use of templates so in case any other protocol has
different requirements, as long as they rely on the message
abstraction and the message system, I'm confident it should be easy to
implement whatever specializations are required to get the desired
functionality on top of the existing design.
>
> I'll post my code later and we can discus approaches later.
>
It would be interesting to see your approach in code and see why you
think using std::string's (which are perfectly fine on their own by
the way able to hold binary data) is a bad thing.
> Thank you for your kind time,
No problem. Looking forward to your code to see what exactly the
problem with using std::strings in binary protocols are.
--
Dean Michael C. Berris
Software Engineer, Friendster, Inc.
|
|
From: Rodrigo M. <rod...@gm...> - 2008-11-04 15:09:03
|
> > Are you talking about Boost.Array? Because the last time I checked, > Boost.Array is statically-sized and is not as flexible as an > std::string. IMO, using something from the STL is "better" because you > don't need to rely on Boost being there to use the library. I won't > even be surprised if it was possible to package the header-only Boost > libs with cpp-netlib as a "standalone" download (with the help perhaps > of bcp). Of course if we're using Boost.Asio we rely on Boost.System > -- but other than that I'm confident there shouldn't be a lot of other > libraries cpp-netlib would rely on (of course aside from the STL). > I'm curious, why std::string and not std::vector<char>? Thanks, Rodrigo |
|
From: Dean M. B. <mik...@gm...> - 2008-11-04 15:15:21
|
On Tue, Nov 4, 2008 at 11:08 PM, Rodrigo Madera <rod...@gm...> wrote: >> Are you talking about Boost.Array? Because the last time I checked, >> Boost.Array is statically-sized and is not as flexible as an >> std::string. IMO, using something from the STL is "better" because you >> don't need to rely on Boost being there to use the library. I won't >> even be surprised if it was possible to package the header-only Boost >> libs with cpp-netlib as a "standalone" download (with the help perhaps >> of bcp). Of course if we're using Boost.Asio we rely on Boost.System >> -- but other than that I'm confident there shouldn't be a lot of other >> libraries cpp-netlib would rely on (of course aside from the STL). > > I'm curious, why std::string and not std::vector<char>? > Because std::string and std::vector<char> are effectively the same. Aside from that std::string even has nice substring capabilities and is just very easy to use. Imagine trying to optimize copying of std::vector<char> around -- when std::string has the capability to do copy-on-write. In the next version of the standard though I think will invalidate most of the implementations of std::string with concurrency guarantees, but even then std::string and std::vector<char> will essentially stay mostly equivalent (and std::string makes more semantic sense if you're dealing with a string of characters). -- Dean Michael C. Berris Software Engineer, Friendster, Inc. |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X