Re: [sqlmap-users] CPU utilization

[Miroslav S. <mir...@gm...>]

Hi all.

After the initial Kasper's "complaint", preliminary testing and some
researching we've come to this:
http://ted.onflash.org/2005/06/python-cpu-tuning.php

The problem seems to be really in the "Python internal C FrameLoop" - or in
plain speak, python runs the program as fast as it can without any real rest
stops. Research results show that if we put a small time delay in page
retrieval method, which is a main backbone function of the sqlmap, CPU
intensivity dramatically decreases.

Sample command used: time python sqlmap.py -u "
http://xxx.xxx.xxx.xxx/somedir/get_int.php?id=1" --ignore-proxy --threads=2
--read-file /etc/shells --flush-session

1) With time.sleep(.001) (or in plain speak 1/1000 of a second) at end of
getPage method in lib/request/connect module:
5.368u 1.100s 0:14.01 46.1%     0+0k 0+48io 0pf+0w

2) Without:
9.160u 1.932s 0:10.94 101.3%    0+0k 0+48io 0pf+0w

You can spot the difference quite easy.

So, the real question now is to put it as a default value there or leave the
choice to the user. You can see that there is a slow down of 3 seconds after
all, but you have to have on your mind that this was a test done on a local
VM, while there shouldn't be such a difference in a real life case. Also,
benefits are more then obvious (cca. 100% drops to less than 50%)

Kind regards.

On Mon, Mar 15, 2010 at 2:24 PM, Kasper Føns <th...@ma...> wrote:

> Dead Miroslav
>
> That was not the point I was trying to make. It would ofcourse be nice to
> utilize the 100% of my computer when a program wants to, but I can see that
> python is crippled in the matter. But what I was asking about was:
> I think it is strange that sqlmap uses 50% of my cpu - What operations
> could require that much cpu? It seems that if a connection to a target is
> slow, sqlmap still uses 50% of the cpu. Is it besy-waiting for the thread
> that is doing the I/O operation to complete?
>
> I'll try again:
> I would see sqlmap's main bottlenet as the internet connection. I can't
> find anything to justify to use my CPU as much as it does. Therefore I think
> some thread is besy-waiting, which really cripples the system (especially if
> I only had 1 CPU). So my question is, is it busy-waiting?
>
> PS: Thanks for the nice link. Was interesting reading.
>
> /Kasper
>
>
> On 15-03-2010 13:17, Miroslav Stampar wrote:
>
>> Dear Kasper.
>>
>> Python has an "technical" issue dealing with multithreading programs.
>> In part that it can't run threads on multiple cores, it has a really
>> nasty "GIL" problem which is discussed here:
>> http://www.snaplogic.com/blog/?p=94. Threading of sqlmap really speeds
>> it up, but the side effect is that CPU-throttling you are talking
>> about, especially in a high number of threads used.
>>
>> Kind regards.
>>
>> On Mon, Mar 15, 2010 at 12:55 PM, Kasper Føns<th...@ma...>  wrote:
>>
>>
>>> Hello sqlmap users.
>>>
>>> I have a question about how sqlmap utilizes the cpu. On my computer the
>>> CPU being utilized is nearly always 50%, and since I have two cores I
>>> suspect some thread to be in deadlock or maybe doing busy-waiting.
>>> If using busy-waiting, I would suggest using an event-based approach
>>> instead.
>>>
>>> /Kasper
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Download Intel&#174; Parallel Studio Eval
>>> Try the new software tools for yourself. Speed compiling, find bugs
>>> proactively, and fine-tune applications for parallel performance.
>>> See why Intel Parallel Studio got high marks during beta.
>>> http://p.sf.net/sfu/intel-sw-dev
>>> _______________________________________________
>>> sqlmap-users mailing list
>>> sql...@li...
>>> https://lists.sourceforge.net/lists/listinfo/sqlmap-users
>>>
>>>
>>>
>>
>>
>>
>>
>
>


-- 
Miroslav Stampar

E-mail / Jabber: miroslav.stampar (at) gmail.com
Mobile: +385921010204 (HR 0921010204)
PGP Key ID: 0xB5397B1B