Re: [Smtcomp-discussion] StarExec competitions '14 questionnaire

[David D. <dav...@lo...>]

David,

Your answer includes all the remarks that I could have made. 

Best regards,

-- David.

On Aug 2, 2014, at 1:34 PM, co...@fr... wrote:

> Tjark, David,
> 
> This is what I sent Aaron - I should have asked for your input, but in my eagerness to get this off my plate, I just sent it without thinking. Additional comments are welcome and I'll amend the response.
> 
> - David
> 
> 
> ----- Forwarded Message -----
> From: "co...@fr..." <co...@fr...>
> To: Aaron Stump <aar...@ui...> 
> Sent: Wednesday, July 30, 2014 3:05 PM
> Subject: Re: StarExec competitions '14 questionnaire
> 
> Feel free to send follow-up questions if you need more detail.
> 
> - David
> 
> 
> 
> 
> From: Aaron Stump <aar...@ui...>
> To: Martina Seidl <mar...@jk...>; Harald Roman Zankl <Har...@ui...>; Dana Fisman <fi...@se...>; Rishabh Singh <ri...@cs...>; Skip (Charles) Jordan <sk...@is...>; David Cok <co...@fr...>; Geoff Sutcliffe <ge...@cs...>; Johannes Waldmann <joh...@ht...>; Tjark Weber <tja...@it...>; David Deharbe <dav...@lo...> 
> Cc: "Tinelli, Cesare" <ces...@ui...> 
> Sent: Monday, July 28, 2014 2:15 PM
> Subject: StarExec competitions '14 questionnaire
> 
> Dear Summer 2014 competition organizers,
> 
> Now that all your events have concluded -- and things have consequently 
> calmed down a bit for the StarExec team! -- we are writing to ask for 
> your feedback on the system and your experience using it this summer for 
> your competitions.  Many of you have already offered feedback in various 
> forms, for which we are grateful.  And the data you supplied Cesare for 
> his talk about StarExec at IJCAR was also very helpful.  Appreciative of 
> your patience with more requests for feedback, we are asking for your 
> responses to the questions below, to try to get even more specific 
> guidance and make plans for the future.  We have to turn in a report at 
> the end of August to the government funders, so we would be very 
> grateful if you could give us your responses within, say, one week (so 
> please return to us by August 5th, if possible).  If vacation time and 
> such prevents that, then please just get us your responses whenever you 
> can, as they will be valuable regardless of whether they arrive in time 
> for our report or not.
> 
> Best wishes, and thanks in advance,
> Aaron & Cesare
> --------------------------------------------------------------------
> Questionnaire for organizers of Summer 2014 competitions running on StarExec
> 
> 1. What is the full official name of your competition, and what are the 
> names and affiliations of all the organizers?
> 
> 2. What are all the StarExec job ids for jobs that were part of your 
> competition?
> 
> 3. Are those competition jobs currently publicly visible on StarExec 
> (i.e., in a public space -- you can make a space public with one of the 
> actions at the bottom of the page on the space explorer)?
> 
> 4. What were the advantages and disadvantages of using StarExec for you 
> as competition organizer, in comparison with whatever alternative you 
> would have utilized instead of StarExec?
> 
> 5. What were the most positive aspects of your experience running your 
> competition on StarExec?
> 
> 6. What were the negative aspects?
> 
> 7. Roughly how many questions would you estimate you fielded about 
> StarExec from participants in your competition?
> 
> 8. Did you use StarExec's pages for viewing job results, or did you 
> create your own web display for showing the data from StarExec?
> 
> 9. (Very) roughly what percentages of your interactions with StarExec 
> for organizing the competition were through the web interface, through 
> StarExecCommand, and through other browser-free interaction with the 
> StarExec web server (e.g., sending POST and GET requests directly to 
> StarExec URLs)?
> 
> 10. Where do you think we should focus our development efforts going 
> forward?  A few of the things we are considering are: additional 
> interfaces for viewing job results (maybe similar to the 2-dimensional 
> grid used in Termination; other suggestions?), solver pipelines (output 
> from one stage becomes benchmark for input to the next stage; other 
> details to be determined), more work to make it possible to install and 
> run StarExec on your own server or computer (abstract out 
> security-critical details, installation documention, and abstract the 
> interface to GridEngine so you can use a different third-party tool to 
> run jobs on a cluster, or even just your own desktop), commitment to and 
> documentation of the URLs used for communicating directly with the server.
> 
> 11. Do you expect you or your colleagues will likely want to run the 
> next edition of your competition on StarExec?
> 
> 12. How adequate was the help you were able to get either directly from 
> the StarExec team or from the forum?
> 
> 13. How satisfied do you perceive the participants in your competition 
> were with their experience using StarExec?
> 
> 14. Were there any recurring issues or complaints from the participants 
> in your competition?
> 
> 15. Any further comments or suggestions?
> 
> 
> 
> 
> <StarExecQuestionnaire-SMTCOMP.rtf>------------------------------------------------------------------------------
> Want fast and easy access to all the code in your enterprise? Index and
> search up to 200,000 lines of code with a free copy of Black Duck
> Code Sight - the same software that powers the world's largest code
> search on Ohloh, the Black Duck Open Hub! Try it now.
> http://p.sf.net/sfu/bds_______________________________________________
> Smtcomp-discussion mailing list
> Smt...@li...
> https://lists.sourceforge.net/lists/listinfo/smtcomp-discussion