cpp-netlib-devel

[cpp-netlib-devel] Multiple requests from single agent

[Thomas K. <Jim...@gm...>]

Hello everybody,

I would like to know whether it is possible to use a single async HTTP
client for more than one request. If so, is it furthermore possible to
issue a new request from the callback which is passed to the get
function?
The reason why I'm asking, is because for some reason I get thread
resource exceptions - as if the async thread were not being destroyed
after the callback has been invoked with an eof status.

Anyone who could help me out here?

Kind regards,
Thomas

Re: [cpp-netlib-devel] Multiple requests from single agent

[Dean M. B. <mik...@gm...>]

On Mon, Sep 19, 2011 at 12:09 PM, Thomas Karolski <Jim...@gm...> wrote:
> Hello everybody,
>
> I would like to know whether it is possible to use a single async HTTP
> client for more than one request. If so, is it furthermore possible to
> issue a new request from the callback which is passed to the get
> function?

Yes you can use a single async HTTP client for more than one request.

I have not tried whether it's possible to start another request from
within the streaming callback handler. I would imagine it should be
unless you're doing something funny.

> The reason why I'm asking, is because for some reason I get thread
> resource exceptions - as if the async thread were not being destroyed
> after the callback has been invoked with an eof status.
>
> Anyone who could help me out here?
>

How does your code look like?

Cheers

-- 
Dean Michael Berris
http://goo.gl/CKCJX

Re: [cpp-netlib-devel] Multiple requests from single agent

[Thomas K. <Jim...@gm...>]

Here you go, trimmed down to the essentials:
header (just for reference): http://snipt.org/xnlol
implementation: http://snipt.org/xnlok
Basically all I do is call the ResourceDataFromURL method - it then either:
a) directly starts a new request on the client or
b) queues a request

Once a callback has received all data it will directly start a new
request on the client (if there are any) and call the user-supplied
callback fn.

However on MacOSX I get some 2000 active threads before the
application dies on me. On Ubuntu I get a sigsegv instead on line 72 -
this is after calling ResourceDataFromURL 8 times in total.
Maybe the problem is that I use boost::shared_ptr to store the
requests - and once the callback quits, the shared_ptr will free the
request, thus any further actions the http client may do on the
request (free resources?) will cause unexpected behavior.
What do you think?

Kind regards,
Thomas

2011/9/19 Dean Michael Berris <mik...@gm...>:
> On Mon, Sep 19, 2011 at 12:09 PM, Thomas Karolski <Jim...@gm...> wrote:
>> Hello everybody,
>>
>> I would like to know whether it is possible to use a single async HTTP
>> client for more than one request. If so, is it furthermore possible to
>> issue a new request from the callback which is passed to the get
>> function?
>
> Yes you can use a single async HTTP client for more than one request.
>
> I have not tried whether it's possible to start another request from
> within the streaming callback handler. I would imagine it should be
> unless you're doing something funny.
>
>> The reason why I'm asking, is because for some reason I get thread
>> resource exceptions - as if the async thread were not being destroyed
>> after the callback has been invoked with an eof status.
>>
>> Anyone who could help me out here?
>>
>
> How does your code look like?
>
> Cheers
>
> --
> Dean Michael Berris
> http://goo.gl/CKCJX
>
> ------------------------------------------------------------------------------
> All the data continuously generated in your IT infrastructure contains a
> definitive record of customers, application performance, security
> threats, fraudulent activity and more. Splunk takes this data and makes
> sense of it. Business sense. IT sense. Common sense.
> http://p.sf.net/sfu/splunk-d2dcopy1
> _______________________________________________
> Cpp-netlib-devel mailing list
> Cpp...@li...
> https://lists.sourceforge.net/lists/listinfo/cpp-netlib-devel
>

Re: [cpp-netlib-devel] Multiple requests from single agent

[Dean M. B. <mik...@gm...>]

Actually, it would be better if you send this (and join the official
mailing list) at https://groups.google.com/group/cpp-netlib.

I hope you don't mind doing that before I respond to you on that list.

Cheers

On Tue, Sep 20, 2011 at 7:25 AM, Thomas Karolski <Jim...@gm...> wrote:
> Here you go, trimmed down to the essentials:
> header (just for reference): http://snipt.org/xnlol
> implementation: http://snipt.org/xnlok
> Basically all I do is call the ResourceDataFromURL method - it then either:
> a) directly starts a new request on the client or
> b) queues a request
>
> Once a callback has received all data it will directly start a new
> request on the client (if there are any) and call the user-supplied
> callback fn.
>
> However on MacOSX I get some 2000 active threads before the
> application dies on me. On Ubuntu I get a sigsegv instead on line 72 -
> this is after calling ResourceDataFromURL 8 times in total.
> Maybe the problem is that I use boost::shared_ptr to store the
> requests - and once the callback quits, the shared_ptr will free the
> request, thus any further actions the http client may do on the
> request (free resources?) will cause unexpected behavior.
> What do you think?
>
> Kind regards,
> Thomas
>
> 2011/9/19 Dean Michael Berris <mik...@gm...>:
>> On Mon, Sep 19, 2011 at 12:09 PM, Thomas Karolski <Jim...@gm...> wrote:
>>> Hello everybody,
>>>
>>> I would like to know whether it is possible to use a single async HTTP
>>> client for more than one request. If so, is it furthermore possible to
>>> issue a new request from the callback which is passed to the get
>>> function?
>>
>> Yes you can use a single async HTTP client for more than one request.
>>
>> I have not tried whether it's possible to start another request from
>> within the streaming callback handler. I would imagine it should be
>> unless you're doing something funny.
>>
>>> The reason why I'm asking, is because for some reason I get thread
>>> resource exceptions - as if the async thread were not being destroyed
>>> after the callback has been invoked with an eof status.
>>>
>>> Anyone who could help me out here?
>>>
>>
>> How does your code look like?
>>
>> Cheers
>>
>> --
>> Dean Michael Berris
>> http://goo.gl/CKCJX
>>
>> ------------------------------------------------------------------------------
>> All the data continuously generated in your IT infrastructure contains a
>> definitive record of customers, application performance, security
>> threats, fraudulent activity and more. Splunk takes this data and makes
>> sense of it. Business sense. IT sense. Common sense.
>> http://p.sf.net/sfu/splunk-d2dcopy1
>> _______________________________________________
>> Cpp-netlib-devel mailing list
>> Cpp...@li...
>> https://lists.sourceforge.net/lists/listinfo/cpp-netlib-devel
>>
>
> ------------------------------------------------------------------------------
> All the data continuously generated in your IT infrastructure contains a
> definitive record of customers, application performance, security
> threats, fraudulent activity and more. Splunk takes this data and makes
> sense of it. Business sense. IT sense. Common sense.
> http://p.sf.net/sfu/splunk-d2dcopy1
> _______________________________________________
> Cpp-netlib-devel mailing list
> Cpp...@li...
> https://lists.sourceforge.net/lists/listinfo/cpp-netlib-devel
>



-- 
Dean Michael Berris
http://goo.gl/CKCJX

Re: [cpp-netlib-devel] Multiple requests from single agent

[Thomas K. <Jim...@gm...>]

Done.

2011/9/21 Dean Michael Berris <mik...@gm...>:
> Actually, it would be better if you send this (and join the official
> mailing list) at https://groups.google.com/group/cpp-netlib.
>
> I hope you don't mind doing that before I respond to you on that list.
>
> Cheers
>
> On Tue, Sep 20, 2011 at 7:25 AM, Thomas Karolski <Jim...@gm...> wrote:
>> Here you go, trimmed down to the essentials:
>> header (just for reference): http://snipt.org/xnlol
>> implementation: http://snipt.org/xnlok
>> Basically all I do is call the ResourceDataFromURL method - it then either:
>> a) directly starts a new request on the client or
>> b) queues a request
>>
>> Once a callback has received all data it will directly start a new
>> request on the client (if there are any) and call the user-supplied
>> callback fn.
>>
>> However on MacOSX I get some 2000 active threads before the
>> application dies on me. On Ubuntu I get a sigsegv instead on line 72 -
>> this is after calling ResourceDataFromURL 8 times in total.
>> Maybe the problem is that I use boost::shared_ptr to store the
>> requests - and once the callback quits, the shared_ptr will free the
>> request, thus any further actions the http client may do on the
>> request (free resources?) will cause unexpected behavior.
>> What do you think?
>>
>> Kind regards,
>> Thomas
>>
>> 2011/9/19 Dean Michael Berris <mik...@gm...>:
>>> On Mon, Sep 19, 2011 at 12:09 PM, Thomas Karolski <Jim...@gm...> wrote:
>>>> Hello everybody,
>>>>
>>>> I would like to know whether it is possible to use a single async HTTP
>>>> client for more than one request. If so, is it furthermore possible to
>>>> issue a new request from the callback which is passed to the get
>>>> function?
>>>
>>> Yes you can use a single async HTTP client for more than one request.
>>>
>>> I have not tried whether it's possible to start another request from
>>> within the streaming callback handler. I would imagine it should be
>>> unless you're doing something funny.
>>>
>>>> The reason why I'm asking, is because for some reason I get thread
>>>> resource exceptions - as if the async thread were not being destroyed
>>>> after the callback has been invoked with an eof status.
>>>>
>>>> Anyone who could help me out here?
>>>>
>>>
>>> How does your code look like?
>>>
>>> Cheers
>>>
>>> --
>>> Dean Michael Berris
>>> http://goo.gl/CKCJX
>>>
>>> ------------------------------------------------------------------------------
>>> All the data continuously generated in your IT infrastructure contains a
>>> definitive record of customers, application performance, security
>>> threats, fraudulent activity and more. Splunk takes this data and makes
>>> sense of it. Business sense. IT sense. Common sense.
>>> http://p.sf.net/sfu/splunk-d2dcopy1
>>> _______________________________________________
>>> Cpp-netlib-devel mailing list
>>> Cpp...@li...
>>> https://lists.sourceforge.net/lists/listinfo/cpp-netlib-devel
>>>
>>
>> ------------------------------------------------------------------------------
>> All the data continuously generated in your IT infrastructure contains a
>> definitive record of customers, application performance, security
>> threats, fraudulent activity and more. Splunk takes this data and makes
>> sense of it. Business sense. IT sense. Common sense.
>> http://p.sf.net/sfu/splunk-d2dcopy1
>> _______________________________________________
>> Cpp-netlib-devel mailing list
>> Cpp...@li...
>> https://lists.sourceforge.net/lists/listinfo/cpp-netlib-devel
>>
>
>
>
> --
> Dean Michael Berris
> http://goo.gl/CKCJX
>
> ------------------------------------------------------------------------------
> All the data continuously generated in your IT infrastructure contains a
> definitive record of customers, application performance, security
> threats, fraudulent activity and more. Splunk takes this data and makes
> sense of it. Business sense. IT sense. Common sense.
> http://p.sf.net/sfu/splunk-d2dcopy1
> _______________________________________________
> Cpp-netlib-devel mailing list
> Cpp...@li...
> https://lists.sourceforge.net/lists/listinfo/cpp-netlib-devel
>