Re: [sqlmap-users] CPU utilization

[Kasper F. <th...@ma...>]

Hi

I am no expert at all in Python. It was just my impression as a 
programmer that there is some kind of busy-waiting.
Without knowing anything, I just threw in "Can this really be true?". Is 
there really no other way to implement fetching pages then using 
busy-waiting.

Actually I just wanted to know if this could be implemented, so my 
hardware was not abused. But from what you tell me, it seems that 
urllib2 is fetching pages using busy-waiting and this is a module that 
cannot be changed.

I checked the file you mentioned but I can't really read the code very well.

/Kasper

On 15-03-2010 14:26, Miroslav Stampar wrote:
> Hi.
>
> That's because, when you do the urllib2 page request, current thread
> goes into blocking state - no events here, just plain raw CPU power.
> If you do it with 30 threads for example, all of them go into blocking
> state. I repeat, python's standard module Urllib2 doesn't support
> asynchronous page retrievals (neither any other).
>
> Could you please be so kind and review
> sqlmap/lib/techniques/blind/inference.py, methods bisection(...) and
> getChar(...), and advise with some concrete solution. We'll be more
> than happy to hear from you.
>
> Kind regards.
>
> On Mon, Mar 15, 2010 at 1:58 PM, Kasper Føns<th...@ma...>  wrote:
>    
>> Hi
>>
>> I still don't get why sqlmap is using that much cpu then?
>>
>> On 15-03-2010 13:49, Miroslav Stampar wrote:
>>      
>>> Hi Kasper.
>>>
>>> Threads are indeed in a blocking state while retrieving page content
>>> (basic socket blocking). Basic python urllib2 doesn't support
>>> asynchronous page retrievals, but we could put this on a feature
>>> request list (http://eventlet.net/doc/,
>>> http://twistedmatrix.com/trac/).
>>>
>>> Kind regards.
>>>
>>> On Mon, Mar 15, 2010 at 1:24 PM, Kasper Føns<th...@ma...>    wrote:
>>>
>>>        
>>>> Dead Miroslav
>>>>
>>>> That was not the point I was trying to make. It would ofcourse be nice to
>>>> utilize the 100% of my computer when a program wants to, but I can see
>>>> that
>>>> python is crippled in the matter. But what I was asking about was:
>>>> I think it is strange that sqlmap uses 50% of my cpu - What operations
>>>> could
>>>> require that much cpu? It seems that if a connection to a target is slow,
>>>> sqlmap still uses 50% of the cpu. Is it besy-waiting for the thread that
>>>> is
>>>> doing the I/O operation to complete?
>>>>
>>>> I'll try again:
>>>> I would see sqlmap's main bottlenet as the internet connection. I can't
>>>> find
>>>> anything to justify to use my CPU as much as it does. Therefore I think
>>>> some
>>>> thread is besy-waiting, which really cripples the system (especially if I
>>>> only had 1 CPU). So my question is, is it busy-waiting?
>>>>
>>>> PS: Thanks for the nice link. Was interesting reading.
>>>>
>>>> /Kasper
>>>>
>>>> On 15-03-2010 13:17, Miroslav Stampar wrote:
>>>>
>>>>          
>>>>> Dear Kasper.
>>>>>
>>>>> Python has an "technical" issue dealing with multithreading programs.
>>>>> In part that it can't run threads on multiple cores, it has a really
>>>>> nasty "GIL" problem which is discussed here:
>>>>> http://www.snaplogic.com/blog/?p=94. Threading of sqlmap really speeds
>>>>> it up, but the side effect is that CPU-throttling you are talking
>>>>> about, especially in a high number of threads used.
>>>>>
>>>>> Kind regards.
>>>>>
>>>>> On Mon, Mar 15, 2010 at 12:55 PM, Kasper Føns<th...@ma...>      wrote:
>>>>>
>>>>>
>>>>>            
>>>>>> Hello sqlmap users.
>>>>>>
>>>>>> I have a question about how sqlmap utilizes the cpu. On my computer the
>>>>>> CPU being utilized is nearly always 50%, and since I have two cores I
>>>>>> suspect some thread to be in deadlock or maybe doing busy-waiting.
>>>>>> If using busy-waiting, I would suggest using an event-based approach
>>>>>> instead.
>>>>>>
>>>>>> /Kasper
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>> Download Intel&#174; Parallel Studio Eval
>>>>>> Try the new software tools for yourself. Speed compiling, find bugs
>>>>>> proactively, and fine-tune applications for parallel performance.
>>>>>> See why Intel Parallel Studio got high marks during beta.
>>>>>> http://p.sf.net/sfu/intel-sw-dev
>>>>>> _______________________________________________
>>>>>> sqlmap-users mailing list
>>>>>> sql...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/sqlmap-users
>>>>>>
>>>>>>
>>>>>>
>>>>>>              
>>>>>
>>>>>
>>>>>            
>>>>
>>>>          
>>>
>>>
>>>        
>>
>>      
>
>
>