HPCS-IO Benchmark

[Mark Dotson]

I've been able to run the benchmarks on a small cluster with the scripts from the "unsupported" folder. I've created PBS scripts for each and they run just fine with my custom parameters. Now, I would like to use the postprocess.py script to process the data, but am uncertain as to the usage. I did the "-h" for help, but am still lost. I tried running the script on one of my result files, to no avail. Can anyone help?

Thanks,

Mark

[Jeff Garlough]

Here's a sample file, based on HPCS_test02.csv:
$ cat sample_HPCS_test02.csv
Rank 0,Phase 0,Write file thread 0 -- seconds,5.0,10.0,15.0,
Rank 0,Phase 0,Write file thread 0 -- ops,1,2,3,
Rank 0,Phase 0,Write file thread 0 -- bytes,500000000,1000000000,1500000000,

And running postprocess.py on it gives:
$ ./postprocess.py sample_HPCS_test02.csv
average overall rate (MiB/sec), phase 0, Write file = 95.367
Total Bytes = 1500000000
Max. Sec = 15.0

Do you get the same results?

[Mark Dotson]

Hi, Jeff.

I received the following output using your sample .csv file:

[dotsonml@dev-scl1 scripts]$ ./postprocess.py sample.csv
Traceback (most recent call last):
File "./postprocess.py", line 245, in ?
op_diff = [y-x for x,y in pairwise(ops)]
File "./postprocess.py", line 45, in pairwise
next(b, None)
NameError: global name 'next' is not defined

Thanks for your help,

Mark

[Jeff Garlough]

Hi Mark,

Thanks for trying out the code and reporting problems. What version of
python are running? It works for versions 2.6 and 2.7, but not python
3.* (yet). I haven't tried earlier versions.

--
Jeff

[Mark Dotson]

Jeff,

Looks like 2.4.3. Should I upgrade to 2.7?

Thanks,

Mark

[Mark Dotson]

Jeff,

I just compiled and installed Python 2.7.3 and now it works!

Thanks a lot!

Mark