Re: Help creating a patch for variable URLs

[Robert I. <cor...@gm...>]

Hi Gary,


Have you seen my previous message?

On Mon, Sep 29, 2008 at 8:24 PM, Gary Fitts <ga...@in...> wrote:

> Regarding a post from 2007-11-26 20:46 "Is it possible to use data
> from the response header to a POST in a subsequent GET?"
>
> Ken Mamitsuka wanted to use a variable in the URL for a PUT. I have a
> similar need -- I'd like to be able to take URLs from a list of user-
> specific URLs. Perhaps something like this:
>
> URL = file(./user_urls1)
> REQUEST_TYPE = PUT
> ...
> URL = file(./user_urls2)
> REQUEST_TYPE = POST
>
> or something like that -- maybe there's a better syntax that you could
> suggest. I just need each client to take its URL from the next line of
> a given file.
>
> I do know C and I'd be glad to write a patch if you can give me some
> guidance about where to look.
>
> Thanks!
> Gary Fitts
>
>
The URL list may be supported as described in my previous reply.

PUT request is supposed to be supported as is. If it goes buggy, please,
report and I will investigate the case, whether there is an issue with
curl-library.

Here is below my previous reply:
============================================================
Sounds great! Several people have requested something like this, and you
have formulated
the requirements.

Let's call it

URL_SET_TEMPLATE, which will have any number of %s symbols in any places of
its url template,
which will  be read from a file
URL_SET_TOKEN_FILE

It presumes, that all other tags for the set URL will be the same.

Files parse_conf.c contains
tag_parser_pair_tp_map
where the two new tag can be easily added with their handling functions.

Now the matters to care about.
URLS_NUM from general section should contain number of total URLs, which can
come
from individual URL tags as well as from URL_SET_TEMPLATE.

parser of URL_SET_TEMPLATE does not allocate a URL by itself, but instead
should
read the template and sets a flag to wait for the next URL_SET_TOKEN_FILE.

parser of URL_SET_TOKEN_FILE will create N URL objects, where N is the
number of
rows in the file. It may keep in the batch object the two numbers: where the
SET starts,
and the number of the URLs in the set (N) . Each next tag of the URL section
(e.g.
REQUEST_TYPE, TIMER_AFTER_URL_COMPLETION, etc) should be applied to all the
URL objects, which to be signaled to the parsers of the tags by the non-zero
N, or by a flag,
as you wish.

The next URL or  URL_SET_TEMPLATE tags should care about advancing a field
in batch
object

Therefore, in url_parser (and in url_set_template_parser) instead the
current

bctx->url_index++;

should appear something like this:

if (!N)
{
   bctx->url_index++;
}
else
{
   url_index += N;
   N = 0;
}

>
>
> 2. Build client-specific URLs from tokens returned in the body of previous
> responses.
> For instance, here's a sample client-server exchange (with most headers
> omitted):
>
> POST /login HTTP/1.1
>
> log...@in...&password=guest
>
> HTTP/1.1 200 OK
>
> <?xml version="1.0" encoding="UTF-8"?>
> <user user_id="38" profile_id="3"/>
>
> PUT /users/38/profiles/3/subscriptions HTTP/1.1
>
> stuff ...
>
>
> In this case I'd like to be able to grab user_id and profile_id from the
> returned
> XML, and use these tokens to build the subsequent URL.
> Maybe the config file could look something like this:
>
> URL=http://a.xyz.com/login
> RESPONSE_TOKEN=user_id
> RESPONSE_TOKEN= profile_id
>
> URL=http://a.xyz.com/users/<user_id>/profiles/<profile_id>/subscriptions
>
> I realize that it would be hard to make this completely general, but I'm
> sure
> I could write response-parsing code specific to my needs.
> Any guidance would be appreciated!
> Gary
>

It could be made in generic fashion.

Function client_tracing_function () filters all bytes from headers as well
as from bodies.
Incoming body bytes are passing switch
CURLINFO_DATA_IN

cctx - (client context) object can have a set of filters and parse the
passing body bytes
to extract them. Take care however about partial token, e.g. by keeping
several last
bytes from the previous packet, or etc method.

A suggestion is for you to look at the existing mechanism delivered
by URL_USE_CURRENT.

More questions/ suggestions are welcomed!
-- 
Truly,
Robert Iakobashvili, Ph.D.
......................................................................
Assistive technology that understands you
......................................................................