Re: [Openflow-dev] [openflow-dev] regression test survey

[Reid P. <rei...@gm...>]

Hi Tatsuya,

Sorry for the last-minute feedback, I neglected to sign up for the dev list.
Some things I've come up against or recalled as I type this follow.

Setup
It is not very straightforward.  At least on a debian (lenny) box, I have to
execute 25 commands (counting 3 seds in place of manual edits) to get
to the point where I can attempt to run a test.  I'm also not the biggest
fan
of the setup/teardown, but I have no helpful suggestions for something
better that maintains the same level of overhead.  I was able to find all
the
documentation, eventually, but it seemed like that could be streamlined as
well, I often have to follow a doc trail over 3+ files or into source to
figure
out what is going on.

Debugging
I didn't find it that hard to only run a few tests, once you know where
to comment out things, but a abort-on-first-failure would certainly be
useful.  Overall it was cumbersome. I found that the non-verbose debugging
was too little, and the verbose was too much, some gradient might be nice,
I usually found myself just running tcpdump or wireshark on all the
interfaces.

Features that would be nice
Like Yiannis/Brandon, I wouldn't mind if the framework was python (over
perl),
even if it is unlikely that perl is the necessary and sufficient cause here.
Structure/organization weren't necessarily intuitive for me, as a casual
debugger, we would get that as a side-effect of a stricter language.

If you are considering a complete rewrite it might be nice if there were
additional options for larger/smaller scale tests, instead of the static
configuration of 4 ports.  This may be my inexperience, but I wasn't
aware of any way of (for example) testing a 2-3 port physical openflow
switch, or wifi.

  -Reid