[Treebase-devel] performance benchmark checking

[Hilmar L. <hl...@ne...>]

Rutger -

in response to your (I assume?) suggestion: "Rather have this  
contingent on the [performance optimization], i.e. produce final  
performance numbers after optimization"

I disagree with this somewhat. There are three main reasons to do the  
benchmarking: 1) is the NESCent-hosted instance so severely slower  
(overall or in specific areas) than the SDSC-hosted one so as to  
amount to a release show-stopper or serious user experience issue; 2)  
do any of the significant performance differences hint at schema bugs  
(such as missing indexes, or failure to collect row and column  
statistics); and 3) indicate scalability issues.

None of these depend on optimization having been completed.

That being said, we will be running another set of performance tests  
after optimization to have an assessment as to how much optimization  
helped with any of the above. So, if the tests you have run so far in  
your opinion suffice to answer the above 3 questions, we can table all  
remaining ones until we rerun the suite after optimization.

Did you already have a plan for where to document the results? Does  
the TB2 wiki sound like a good place?

	-hilmar
-- 
===========================================================
: Hilmar Lapp  -:- Durham, NC -:- informatics.nescent.org :
===========================================================