Menu
▾
▴
Re: [Iometer-user] Limit IOpS
|
From: Sascha E. <fe...@do...> - 2013-03-11 15:24:18
|
Hello,
I think I found something, I can not beleave. So please help me to
understand: Part of the Dynamo are the header files IOManager.h and
Manager.h. Both define a class Manager (without namespace), which seem
to be different classes. Is this right? So the only way to distinguish
this classes is to hope that at no point in time both header files are
used? I am very confused about this...
Back to my problem: I want only about 5000 IOs per worker. So, what I do
is something like a busy wait in Grun::Do_IOs. Therefore I added
something like the following:
uint64_t wait_until = timer_value() + waiting_tics;
while (wait_until > timer_value()) {
SwitchToThread();
}
More Information about SwitchToThread:
http://msdn.microsoft.com/en-us/library/windows/desktop/ms686352(v=vs.85).aspx.
It is something like yeald in pthreads.
I want to use the code to wait for about 0,5 ms (adjustable). So I need
to get the timer frequency from within Grunt. I got it in the Manager
(as in IOManager.h) from the perf_data, calculated it by
uint64_t waiting_time = 500 *
perf_data[WHOLE_TEST_PERF].timer_resolution / 1000000;
and passed it to the contructor of grunt. Here I stored it in an
attribute waiting_tics. This helped me.
So, at the moment I am able to wait as short I like. The code now has to
be modified, so it is possible to work also with more than one
outstanding IO (so I have to call CompleteIO in the while loop) and I
have to adjust the timing by using the last time I started an
transaction (so I start an transaction every 0,5 ms, not I start an IO
and wait 0,5ms). I also have to change the code to support the other
operating systems, as it will run only for Windows at the moment (It
uses SwitchToThread). All this can be done by me, if it is interesting
for anybody (at least I think this will not be a big problem).
My problem is, that it also should be integrated in the IOMeter GUI, and
that is a pbroblem for me. I haver never done anything developing a GUI
for Windows, so I need some help here. Is there anybody who would like
to help me?
bests
Sascha
Am 04.03.2013 17:26, schrieb Sascha Effert:
> Huh, that was fun... I wrote s script to create a config to use 1600
> workers on a machine... It just crashed... So I will try to find out how
> much threads are possible, but I think this will not work for high IO
> Rates... :-)
>
> bests
>
> Sascha
>
> Am 04.03.2013 13:46, schrieb Sascha Effert:
>> Hello Robert,
>>
>> is it possible to set the delay in the ICF? I did not find anything in
>> this area.
>>
>> I now added a sleep_milisec(1) in the Do_IOs() of IOGrunt. First I added
>> it at the end of the loop, but then it was part of the response time. I
>> now moved it to the start of the transaction, so I changed around line
>> 1264 of trunc to be following:
>>
>> //transaction->start_IO = (grunt_state == TestRecording) ?
>> timer_value() : 0;
>>
>> // If the transaction start time hasn't been set,
>> // this is the first I/O of this transaction. Set the start time.
>> // ERROR(fe...@do...): This is a quick patch to limit
>> to number of IOPS
>> // generated by the dynamo. This
>> should not make it
>> // into the public sources. It
>> should be possible to
>> // turn this behaviour on/off using
>> the GUI (or at
>> // least the command line of the
>> dynamo).
>> if (!transaction->start_transaction) {
>> sleep_milisec(1);
>> transaction->start_IO = (grunt_state == TestRecording) ?
>> timer_value() : 0;
>> transaction->start_transaction = transaction->start_IO;
>> } else {
>> transaction->start_IO = (grunt_state == TestRecording) ?
>> timer_value() : 0;
>> }
>>
>> With this I get more or less the behaviour you discribed: around 63 IOPS
>> per worker. I will check how to get this up to a few hundred workers, so
>> to get 5000 IOPS I will need about 80 workers, to get 100.000 IOPS it
>> will be about 1600. I wonder if my machine will like this...
>>
>> I think there is no good way to make it waiting a shorter amount of time?
>>
>> I will be at CeBit for the rest of the week, so I think I will not be
>> able to test it before next monday.
>>
>> bests
>>
>> Sascha
>>
>> Am 28.02.2013 21:24, schrieb Robert Perry:
>>> You may want to start by looking at how 'delay' is handled in grunt.cpp. You could do it now, without making any changes, but it is a bit tricky.. Start by create an ICF that has 1 worker doing I/O to one lun with 1 outstanding and set the delay to 1.... this will tell you what the actual granularity of the timing mechanism is on the host... I found that on windows I was seeing something like 14ms...i would get ~70 iops versus 1000, but if you see something close to 1000 iops, that is great. You can then play with multiple workers to get more iops... I haven't done this in quite a while, but I think I used a combination of num outstanding,delay and burst to deal with the 14ms granularity and drive certain loads...
>>>
>>>
>>> On Feb 28, 2013, at 7:18 AM, Sascha Effert <fe...@do...> wrote:
>>>
>>>> Hello,
>>>>
>>>> I would like to measure the average and max response time of a storage
>>>> system (SAN with SSDs) with different load. Therefore I want iometer to
>>>> generate (about) 5000 IOPS (later about 10000, 20000, up to some hundred
>>>> thousands). Is this possible using iometer? At the moment I do not see a
>>>> way to solve this. Does anybody know any other tool solving this issue?
>>>>
>>>> If IOMeter does not allow to do so, I would like to play a little bit
>>>> with the code. Therefore I want to implement a sleep after each IO.
>>>> Searching the code I found the TargetDisk implementing the Read and
>>>> Write and the class Grunt, which is the IO worker. I am very unsure
>>>> where I should place the sleep. I am searching something like a look
>>>> where the single Requests are created. Can someone give an idea where to
>>>> find it?
>>>>
>>>> bests
>>>>
>>>> Sascha
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Everyone hates slow websites. So do we.
>>>> Make your web apps faster with AppDynamics
>>>> Download AppDynamics Lite for free today:
>>>> http://p.sf.net/sfu/appdyn_d2d_feb
>>>> _______________________________________________
>>>> Iometer-user mailing list
>>>> Iom...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/iometer-user
>> ------------------------------------------------------------------------------
>> Everyone hates slow websites. So do we.
>> Make your web apps faster with AppDynamics
>> Download AppDynamics Lite for free today:
>> http://p.sf.net/sfu/appdyn_d2d_feb
>> _______________________________________________
>> Iometer-user mailing list
>> Iom...@li...
>> https://lists.sourceforge.net/lists/listinfo/iometer-user
>
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_feb
> _______________________________________________
> Iometer-user mailing list
> Iom...@li...
> https://lists.sourceforge.net/lists/listinfo/iometer-user
|
