Re: [cpp-netlib-devel] Query parameters

[Dean M. B. <mik...@gm...>]

Hi Matt!

On Tue, May 26, 2009 at 12:27 PM, Matt Trentini <mat...@gm...> wrote:
> Hello folks,
>
> How are query parameters meant to be set with cpp-netlib?
>

There used to be a project which should have moved the URI
building/manipulation out of the http::request class implementation
into its own. At the moment, it's all ad-hoc and undefined really.

>
>    http::request
> request("http://hudson_server/hudson/api/xml?xpath=//job[name='mainjob']/color/text()");
>    http::client client;
>    http::response response;
>    response = client.get(request);
>    cout << body(response) << endl;
>
> The "/"'s in the xpath query cause problems.  The parser in
> request.hpp (specifically, line 94) fails to match the "/" and
> discards the query parameters.
>
> [As an aside I appreciate that "/" is a reserved character in an URI
> but haven't dug far enough into the RFC
> (http://www.ietf.org/rfc/rfc3986.txt) to determine whether this usage
> of it makes for a valid URI.  I *think* it is valid - even without
> escaping - based on section 3.4 but I'm not 100%.]
>

Thanks for the link, I looked it up and this is what I found about the
Query string:

3.4.  Query

   The query component contains non-hierarchical data that, along with
   data in the path component (Section 3.3), serves to identify a
   resource within the scope of the URI's scheme and naming authority
   (if any).  The query component is indicated by the first question
   mark ("?") character and terminated by a number sign ("#") character
   or by the end of the URI.

      query       = *( pchar / "/" / "?" )

   The characters slash ("/") and question mark ("?") may represent data
   within the query component.  Beware that some older, erroneous
   implementations may not handle such data correctly when it is used as
   the base URI for relative references (Section 5.1), apparently
   because they fail to distinguish query data from path data when
   looking for hierarchical separators.  However, as query components
   are often used to carry identifying information in the form of
   "key=value" pairs and one frequently used value is a reference to
   another URI, it is sometimes better for usability to avoid percent-
   encoding those characters.

So it looks like the parsing should treat whatever is after the first
'?' character upto the character before the '#' character.

> I've worked around the issue with the following change:
>

[snip]

>    -                    >> (+(alnum_p | '&' | '=' | '%' | '_' ))[
>    +                    >> (+(anychar_p | '&' | '=' | '%' | '_' ))[

[snip]

After reading the RFC, I'd think the parser above should accept
anything except the '#' character. I'm a little rusty with my Spirit
powers, but I'd think it can be achieved easily.

[snip]
>
> But that's quite dirty!
>

I agree. :-)

> I'm wondering if there's a better way to set queries for a request?  I
> couldn't find anything in the code but I may well have missed
> something.
>
> I suppose I was looking for something like:
>
>    http::request request("http://hudson_server/hudson/api/xml");
>    request.addQuery("xpath", "//job[name='mainjob']/color/text()")
>
> That way the parser could add the surrounding stuff (question marks,
> amperstands etc) for me.  As it turns out this is similar to how
> Pion's HTTPRequest class handles queries.
>

I am a fan of DSEL's (if you haven't noticed yet ;-) ), and it would
be feasible (with Boost.Proto maybe or the same way I came up with the
request builder protocol) to enable something like:

  http::request request("http://hudson_server/hudson/api/xml");
  build(request) << query_param("xpath", "//job[name='mainjob']/color/text()")
    << query_param("something_else","Some data that gets automatically
encoded!")
    << fragment("home");

I'm positive the encoding and parsing can benefit from an upgrade to
using Spirit 2.0 (Qi and Karma).

> BTW I did notice there is a query() method to *return* the query
> string.  I also noticed the "make_query_string" method but I wasn't
> sure if I could (or should) try and use it since it was in an impl
> file.  It also doesn't seem to be used internally either though...
>

I remember that query() method in the request object -- although I
forget what the 'make_query_string' method was supposed to do.

> Any comments?  Am I doing something obviously wrong?

Aside from what I've said above, I think you found a legitimate bug. :-)

Thanks for raising the concern! :-D

-- 
Dean Michael Berris | Software Engineer, Friendster, Inc.
blog.cplusplus-soup.com | twitter.com/mikhailberis |
linkedin.com/in/mikhailberis | profiles.friendster.com/mikhailberis |
deanberris.com