Re: [proteowizard-developer] end-user test suite for msconvert

[Matthew C. <mat...@va...>]

Test suites for binary distributions are unusual AFAIK, but I agree that 
it could be useful for troubleshooting. If it's a standalone test, then 
it can't be part of the build system and it'll have to be launched with 
a script (batch file). The build system could build the tarball though.

-Matt


Brian Pratt wrote:
> My view into the TeamCity build/test setup is admittedly a bit murky, 
> but ProteoWizard\pwiz\scripts\regression\run_windows_latest appears to 
> be doing regression testing on msconvert with the data coming out of 
> a nicely organized data directory tree.  Just tarring up that data 
> tree would be useful.
>  
> My thinking was that it would be helpful to users in determining if 
> (or why not) a msconvert setup was functional.  Not the same as real 
> world data, no, but still quite useful on the way to handling real 
> world data.
>  
> It sounds like you object to the idea of a data only tarball on the 
> basis that one can get the data from the source tarball.  I was 
> thinking it would be a fairly common scenario for folks to want 
> binaries and test data but not source, and a fairly simple thing to 
> implement alongside the already existing tarball generation steps.
>  
> Brian
>
> On Wed, Sep 23, 2009 at 11:10 AM, Matthew Chambers 
> <mat...@va... 
> <mailto:mat...@va...>> wrote:
>
>     What do you mean by "Looks like they're already nicely gathered
>     together
>     as part of the TeamCity process"? Are you talking about how the
>     example/unit-test data is included in the source tarballs? If so, the
>     unit test data could certainly be reused for msconvert testing,
>     but it's
>     not (supposed to be) real-world data which is probably what most
>     end-users are concerned with. Like I mentioned before though, if the
>     msconvert testing is integrated into the build system, it could easily
>     use the same data as the vendor readers without having to create a
>     different tarball with just the example data.
>
>     -Matt
>
>
>     Brian Pratt wrote:
>     > So I've got msconvert and all freely available vendor DLLs installed
>     > on an Amazon EC2 node (more annoying than it sounds, they really
>     have
>     > Windows security screwed down tight), and I'm realizing that it's a
>     > bit awkward getting test data together to verify msconvert
>     operation.
>     >
>     > Hardly an insurmountable problem, but it might be nice if one of the
>     > artifacts we delivered was a tarball of the available msconvert test
>     > files and blessed output equivalents.  Looks like they're already
>     > nicely gathered together as part of the TeamCity process (which
>     > totally, totally, rocks BTW).
>     >
>     > My thinking is this:
>     > 1) it might allow users to do a bit of installation debugging they'd
>     > otherwise bother us with
>     > 2) I'm lazy and don't want to go through a multi step manual process
>     > to obtain test data every time I set one of these things up, and I
>     > don't want to install a bunch of dev tools like SVN or wget on
>     the EC2
>     > node*
>     >
>     >
>     > * granted the idea of an EC2 AMI is that you set it up just once
>     then
>     > save the machine image, but there are usually several iterations
>     along
>     > the way to completion and future maintenance
>     >
>     > Cheers,
>     >
>     > Brian
>     >
>
>