Re: [Collectl-interest] Building a custom 'import' file

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 454-5900

Hi Mark,

Since it will just be executed one time.. then I think getting the right
disk from the storage cells would be the way to go.. randy.pl will output
the flash devices... my colleague and I are stucked with the "*sub
helloGetFlashDisks*" on how we can compare the output value to the     for
(my $i=0; $i<$NumDisks; $i++) {   loop...

*> cat randy.pl*
#!/usr/bin/perl

my @temp = `cellcli -e list celldisk attributes deviceName, diskType where
diskType = FlashDisk`;

foreach $line (@temp) {
  next if $line =~ /^\s*$/;  # skip blank lines
  chomp($line);              # remove newline char
  $line =~ s/^\s+//;
  $line =~ s/\s+/ /ig;
  $line =~ s/\/dev\///g;

  my @field=split(' ', $line);

  $dskDevice = $field[0];
  $dskType   = $field[1];

  push @FlashDiskDevices, $dskDevice;

  #print "dskDevive=$dskDevice diskType=$dskType\n";
}

foreach $flashdev (@FlashDiskDevices) {
  print "$flashdev\n";
}

*> perl randy.pl*
sdr
sdaa
sds
sdt
sdu
sdz
sdab
sdac
sdn
sdo
sdp
sdq
sdv
sdw
sdx
sdy

*sub helloGetFlashDisks*
  my @temp = `cellcli -e list celldisk attributes deviceName, diskType where
diskType = FlashDisk`;

  foreach my $line (@temp) {
    next if $line =~ /^\s*$/;  # skip blank lines
    chomp($line);              # remove newline char
    $line =~ s/^\s+//;
    $line =~ s/\s+/ /ig;
    $line =~ s/\/dev\///g;
    my @field=split(' ', $line);
    $dskDevice = $field[0];
    $dskType   = $field[1];
    push @FlashDiskDevices, $dskDevice
    #print "dskDevice=$dskDevice diskType=$dskType\n";
  }
  # Not sure but I think we need to return a reference to the array.
  return %@FlashDiskDevices
}

Also one thing I'd like to do is to hard code the values itself...  I'd like
to compare the $dskName with just hard coded values... can you help me with
this?

    for (my $i=0; $i<$NumDisks; $i++)
    {
      next of $dskName[$i] = 'sda';
      $line=sprintf("$datetime%-11s %6d %6d %4s %4s  %6d %6d %4s %4s   %5d
%5d  %4d   %4d  %3d\n",
                $dskName[$i],
                $dskReadKB[$i]/$intSecs,  $dskReadMrg[$i]/$intSecs,
 cvt($dskRead[$i]/$intSecs),
                $dskRead[$i] ? cvt($dskReadKB[$i]/$dskRead[$i],4,0,1) : 0,
                $dskWriteKB[$i]/$intSecs, $dskWriteMrg[$i]/$intSecs,
cvt($dskWrite[$i]/$intSecs),
                $dskWrite[$i] ? cvt($dskWriteKB[$i]/$dskWrite[$i],4,0,1) :
0,
                $dskRqst[$i], $dskQueLen[$i], $dskWait[$i], $dskSvcTime[$i],
$dskUtil[$i]);
      printText($line);
    }

[enkcel01:root] /usr/share/collectl
> collectl --import hello.ph,d -sD
"my" variable $i masks earlier declaration in same scope at hello.ph line
204.
syntax error at hello.ph line 153, near "next of $dskName"
Compilation failed in require at /usr/bin/collectl line 769.

[enkcel01:root] /usr/share/collectl
>

[enkcel01:root] /usr/share/collectl
> cat hello.ph  | grep -i "dskname"
our ($NumDisks, @dskName, @dskReadKB, @dskReadMrg, @dskRead, @dskWriteKB,
@dskWriteMrg, @dskWrite, @dskRqst, @dskQueLen, @dskWait, @dskSvcTime,
@dskUtil);
      next of $dskName[$i] = 'sda';
                $dskName[$i],

*-Karl*

On Mon, Aug 22, 2011 at 9:35 AM, Mark Seger <mj...@gm...> wrote:

> yep, just went back to your previous note about that showed the contents of
> /proc/diskstats and I see
>
>     8  208 sdn 38406201 737 1698632478 16665706 2083834 0 69472300 749642
> 0 14998912 17405730
>    8  224 sdo 38862003 770 1716768414 17100687 1324499 0 45068644 587768 0
> 15101601 17677971
>    8  240 sdp 39080467 728 1728624030 17058744 1390700 0 47289108 649285 0
> 15155889 17697797
>
> I guess I thought these flash drives were something special and so the
> vendor used unique maj/min numbers.  guess I was wrong ;)
>
> ok, so now we're back to the init section like you said.  what I think
> you're going to have to do is identify each disk you want to process or
> ignore, depending on how your logic in the print loop is written.  further,
> you need to figure out the most efficient way to do this as you don't want
> any extra processing cycles if you can avoid them.  finally, you need to
> figure out a method to determing how the disks are names IN ALL CASES if you
> want this code to be reliable.  welcome to my world ;)
>
> for example, do the bulk of the disks usually have a major number of 65?
>  do disks with a major number of 65 ever non-flash disks?  also, as I think
> about it, it's probably slightly faster to do numeric compares than string
> compares.  so that said, and this is an uneducated guess based on minimal
> information which can be dangerous...
>
> I'd consider building an array called something like $flash[$maj][$min]
> with the major/minor drive numbers of the flash drives.  Now that one line
> test can become:
>
>          next if !defined($flash[$dskMaj[$i]][$dskMin[$i]]);
>
> now back to init.  you need to do whatever you need to do to populate that
> array.  if you have a script that already loops through and get the
> major/minor device numbers it should be pretty straightforward.  just
> remember that this code runs one time, so if the configuration changes
> dynamically,  the rest of your code won't know it.
>
> -mark
>
> On Mon, Aug 22, 2011 at 10:13 AM, Karl Arao <kar...@gm...> wrote:
>
>>
>> Hi Mark,
>>
>> Not all of them has a 65 major number... see the three missing devices..
>>
>> > perl randy.pl
>> /dev/sdr
>> /dev/sdaa
>> /dev/sds
>> /dev/sdt
>> /dev/sdu
>> /dev/sdz
>> /dev/sdab
>> /dev/sdac
>> /dev/sdn  <<
>> /dev/sdo  <<
>> /dev/sdp  <<
>> /dev/sdq
>> /dev/sdv
>> /dev/sdw
>> /dev/sdx
>> /dev/sdy
>>
>>
>> [enkcel01:root] /root
>> > ls -l /dev/ | grep 65
>> brw-r-----  1 root disk  65,   160 Aug 22 09:09 sdaa
>> brw-r-----  1 root disk  65,   176 Aug 22 09:08 sdab
>> brw-r-----  1 root disk  65,   192 Aug 22 09:10 sdac
>> brw-r-----  1 root disk  65,     0 Aug 22 09:10 sdq
>> brw-r-----  1 root disk  65,    16 Aug 22 09:10 sdr
>> brw-r-----  1 root disk  65,    32 Aug 22 09:10 sds
>> brw-r-----  1 root disk  65,    48 Aug 22 09:09 sdt
>> brw-r-----  1 root disk  65,    64 Aug 22 09:08 sdu
>> brw-r-----  1 root disk  65,    80 Aug 22 09:10 sdv
>> brw-r-----  1 root disk  65,    96 Aug 22 09:09 sdw
>> brw-r-----  1 root disk  65,   112 Aug 22 09:10 sdx
>> brw-r-----  1 root disk  65,   128 Aug 22 09:08 sdy
>> brw-r-----  1 root disk  65,   144 Aug 22 09:09 sdz
>> crw-rw----  1 root root   4,    65 Jun  2 13:00 ttyS1
>>
>> [enkcel01:root] /root
>> > ls -l /dev/ | grep sdn
>> brw-r-----  1 root disk   8,   208 Aug 22 09:11 sdn
>>
>>  [enkcel01:root] /root
>> > ls -l /dev/ | grep sdo
>> brw-r-----  1 root disk   8,   224 Aug 22 09:11 sdo
>>
>> [enkcel01:root] /root
>> > ls -l /dev/ | grep sdp
>> brw-r-----  1 root disk   8,   240 Aug 22 09:11 sdp
>>
>>
>> -Karl
>>
>>
>>
>> On Mon, Aug 22, 2011 at 9:06 AM, Mark Seger <mj...@gm...> wrote:
>>
>>>
>>>
>>> On Mon, Aug 22, 2011 at 9:57 AM, Karl Arao <kar...@gm...> wrote:
>>>
>>>>
>>>> I'm pretty rusty on perl..
>>>>
>>>> let's say I just want to filter these devices ...  sdr, sdaa, sds
>>>>
>>>> can you give an example on how can I define this as an array on the init
>>>> section?
>>>>
>>>
>>> let's ignore the init section for a minute and look at the section where
>>> you generate the output.  remember, this code gets called one time after all
>>> the disk data has been read and processed.  I think I'd do what I suggested
>>> in my previous comment, simply looking at the disk's major device number.
>>>
>>> therefore, remove your 'if' below since $dskName has no value yet.
>>>  further, you need to loop through ALL disks since you don't know which are
>>> the flash drives yet.  In other words, I'd just say:
>>>
>>>     for (my $i=0; $i<$NumDisks; $i++
>>>
>>>     {
>>>
>>>          next of $dskMaj[$i] != 65;
>>>
>>>
>>> then, you'd fall though only for the flask disks, no special init
>>> processing required.  it's that simple...
>>>
>>> -mark
>>>
>>>
>>>> if (not $dskName =~ m/^FD/) {
>>>>
>>>>     for (my $i=0; $i<$NumDisks; $i++
>>>>
>>>>     {
>>>>
>>>>       $line=sprintf("$datetime%-11s %6d %6d %4s %4s  %6d %6d %4s %4s
>>>> %5d %5d  %4d   %4d  %3d\n",
>>>>
>>>>                 $dskName[$i],
>>>>
>>>>                 $dskReadKB[$i]/$intSecs,  $dskReadMrg[$i]/$intSecs,
>>>> cvt($dskRead[$i]/$intSecs),
>>>>
>>>>                 $dskRead[$i] ? cvt($dskReadKB[$i]/$dskRead[$i],4,0,1) :
>>>> 0,
>>>>
>>>>                 $dskWriteKB[$i]/$intSecs, $dskWriteMrg[$i]/$intSecs,
>>>> cvt($dskWrite[$i]/$intSecs),
>>>>
>>>>                 $dskWrite[$i] ? cvt($dskWriteKB[$i]/$dskWrite[$i],4,0,1)
>>>> : 0,
>>>>
>>>>                 $dskRqst[$i], $dskQueLen[$i], $dskWait[$i],
>>>> $dskSvcTime[$i], $dskUtil[$i])
>>>>
>>>>       printText($line);
>>>>
>>>>     }
>>>>
>>>> }
>>>>
>>>> ---------------------------------------------
>>>>
>>>> my colleague and I are also working on dynamically pulling the flash
>>>> devices..
>>>>
>>>> sub helloGetFlashDisks
>>>>   my @temp = `cellcli -e list celldisk attributes deviceName, diskType
>>>> where diskType = FlashDisk`;
>>>>
>>>>   foreach my $line (@temp) {
>>>>     next if $line =~ /^\s*$/;  # skip blank lines
>>>>     chomp($line);              # remove newline char
>>>>     $line =~ s/^\s+//;
>>>>     $line =~ s/\s+/ /ig;
>>>>     my @field=split(' ', $line);
>>>>     $dskDevice = $field[0];
>>>>     $dskType   = $field[1];
>>>>     push @FlashDiskDevices, $dskDevice
>>>>     #print "dskDevice=$dskDevice diskType=$dskType\n";
>>>>   }
>>>>   # Not sure but I think we need to return a reference to the array.
>>>>   return %@FlashDiskDevices
>>>> }
>>>>
>>>> 1;
>>>>
>>>>
>>>> ---------------------------------------------------------------------------------------------
>>>>
>>>>
>>>> if you can give us an example on how to do it with hard coded values ..
>>>> that would be really helpful..
>>>>
>>>>
>>>> -Karl
>>>>
>>>>
>>>>
>>>> On Mon, Aug 22, 2011 at 8:38 AM, Mark Seger <mj...@gm...> wrote:
>>>>
>>>>> When I put these postings together I had to reread some of the messages
>>>>> and something jumped out at me. It looks like all the flash disks have a
>>>>> major device number of 65.  Since collectl stores both the major and minor
>>>>> numbers for each disk in @dskMaj and @dskMin so you could look at those in
>>>>> your processing loops and not even worry about individual disk names.
>>>>> -mark
>>>>>
>>>>>
>>>>>>  so now that you can report disk traffic in a butchered up hello.ph file,
>>>>>> there are 2 separate things to do.  the first is to rename the file to
>>>>>> something other than hello.  what you need to do is pick a name, hopefully a
>>>>>> relatively short one, and rename all the entry points from hello* to that
>>>>>> name* as collectl uses the name of the import file to call the entry points.
>>>>>>
>>>>>> the other thing to do is filter out the disks that aren't flash disks.
>>>>>>  in that loop you stole from formatit.ph that prints all the disk
>>>>>> stats, you need to insert a statement into the loop that does something
>>>>>> like:
>>>>>>
>>>>>> next    if $diskName[$i] isn't a flash disk
>>>>>>
>>>>>> you can either hardcode this test to specific disk names, or if you
>>>>>> like dynamically build that list in the 'init' section and then use those
>>>>>> names as the test.
>>>>>>
>>>>>> once you get that far, we can take the next steps to print the right
>>>>>> values in brief format.  Sound like a plan?
>>>>>>
>>>>>> -mark
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Karl Arao
>>>> karlarao.wordpress.com
>>>> karlarao.tiddlyspot.com
>>>>
>>>
>>>
>>
>>
>> --
>> Karl Arao
>> karlarao.wordpress.com
>> karlarao.tiddlyspot.com
>>
>
>

-- 
Karl Arao
karlarao.wordpress.com
karlarao.tiddlyspot.com

Re: [Collectl-interest] Building a custom 'import' file

This is now also available here github.com/sharkcz/collectl.git

Re: [Collectl-interest] Building a custom 'import' file