Re: [Msi-workgroups-board] minutes of the MSI teleconference Thursday, October 11, 7-9 a.m. Pacific (10-12 a.m. EST)

[Chris T. <chr...@eb...>]

Hi all.

My sincere apologies for missing the call yesterday -- like an 
idiot I messed up the time by an hour and was dutifully on the 
phone at 4pm uk time (instead of 3).

Anyway some comments on the minutes, for what it is worth:

>       In addition, Rima Kaddurah-Daouk will send out a letter to the
>       Metabolomics Society members (which will be drafted by Oliver
>       Fiehn) that calls for community feedback based on the current MSI
>       documents. This period of feedback should extend to 6 months to
>       gather a certain amount of experience.

It is crucial that this be promoted more widely than just the 
society (whether any extra responses are generated from that or 
not) if the guidelines are intended to apply to that wider 
community (think about the general science journals -- for 
example Nature Group would not have taken the 'word' of PSI/HUPO 
that we got things right and even after running their own 
consultation exercise they will not implement immediately -- not 
till there are good tools).

So at the very least we need another letter to as many big 
journals as will take it promoting again the location of the 
documents and the feedback mechanism. Probably there should be 
prior agreement from each journal that takes such a letter that 
they will start to encourage voluntary compliance for a 
probationary period (no-one will enforce straight away); a 
sweetener Could be to have submitters who provide compliant data 
sets benefit from a form of 'fast tracking' to print.

Crucially, we need clear mechanisms in place though (in the form 
of a formal _process_ document) if these standards are to thrive 
-- without a visibly-responsive channel for comment and a 
robust, explicit process for revision I don't think they'll do 
as well as they could (i.e. to succeed in journals outside the 
sphere of influence of senior MSI participants), as they will be 
perceived as a static entity generated by a subset of the 
community they apply to, with no way for the balance of that 
community to 'come on board'.

Incdentally, NBT now expect that the MIAPE authors who are 
experimentalists will provide MIAPE compliant material  :)  I'm 
not sure they all knew that when they signed on but it is now 
the case -- I think that reinforces your commendable idea that 
MSI participants should lead by example, i.e. from the front.

>       /__/
>     * /_(b) efforts to continue group activities towards ‘best practice’
>       documents_/
>       Currently, MSI groups are mostly dormant. Writing initial MSI
>       papers helped fostering group activities but groups have not been
>       active since. The idea was revitalized to unify the ‘biological
>       context’ documents into a single ‘umbrella’ document that would
>       have further details for the different research areas; however, TC
>       participants did not actively agree on that.

A reasonable course of action here is to pursue that goal 
through MIBBI, thereby involving other relevant groups such as 
the genomic sequencing people and others concerned with the 
description of the origin of a sample. This is also an issue 
further along the workflow as it were -- chemical analysis, PSI 
and a couple of other groups need to chat for one; then there's 
the likely leading role in the stats activity within MIBBI to be 
taken by the data analysis group of MSI. Again we desperately 
need clear documented processes so that come the time that 
integrated reporting standards are available from MIBBI, the 
different contributing communities can credibly sign off on them.

>       A disagreement was stated on the usefulness of working towards
>       ‘best practice’ documents, especially for the ‘data processing’
>       working group. One argument raised was that any official MSI ‘best
>       practice’ document would be counterproductive because such an
>       official stamp would be easily violating the idea of a
>       continuation of improving practices and methods. Instead, a group
>       of individuals might publish their view of ‘current best practice’
>       in order to lead by example which could be improved by other
>       authors using the regular way of scientific discussion.

This is a minefield and I would counsel extreme caution. I know 
everyone knows that it's a potentially controversial area, but 
my specific point is that to link the relatively uncontroversial 
(by comparison) activity of guidelines/formats/CV development to 
arguing about best practice threatens to bog down our current 
products in the unending scrap about whose favourite method is 
best (whatever the objective merits of each). These things are 
best kept completely separate. If there is a real will to do it 
then perhaps the approach taken by HUPO, of instituting an 
entirely separate body to do it, would be best.

>       Nevertheless, the point was raised that for ‘data processing’ it
>       would be really hard to distinguish between ‘minimal standards’
>       and ‘best practice’ and that it would be MSI-compliant to report a
>       wrong way to do a certain piece of research (the same would of
>       course be true for all MSI-areas). So, in a way, right now
>       MSI-compliant would give papers or projects a touch of validation,
>       even if the study design or the way of data acquisition and
>       processing would be very bad.

Such quality judgements are (even where most think it clear cut) 
a nightmare for all sorts of reasons and even for the DA group 
(as we discussed on Boston) this is not a good idea. It is 
_always_ possible to provide guidance on how to report _without_ 
straying into value judgements. Really such judgements are best 
made by referees, guided by journal policy (reference Molecular 
and Cellular Proteomics' approach to this wrt mass spectrometry 
-- they are very clear about their view of certain methods). If 
the wider Society wants to have an input to that and try to help 
build consensus amongst journals (itself an almost impossible 
goal anyway tbh) then fine, but I implore you, keep it separate 
from the other standards stuff. We don't need to argue about the 
pipes _and_ what should flow through them at the same time.

>       This discussion was consolidated in the general agreement that
>       ‘lead-by-example’ papers will also serve as best practice
>       documents. One of such instances is a current submission to the
>       journal Metabolomics on use and misuse of PLS in statistics.

This is a good approach, because the leverage comes through the 
journal and the community, rather than an 'authority'; really 
though this is just a recasting of how things happen anyway no?

>     * /_(c) efforts to draft instances of units and data exchange
>       formats originating from the ‘minimal’ standards_//
>       /In practical terms, currently there are a variety of efforts
>       underway to accumulate data and distribute these via websites.
>       However, none of the current efforts really embrace the need of
>       data exchange formats in order to standardize and compile
>       different data sets or data sets of different origins. Some
>       cyberinfrastructure projects (e.g. NSF funded in plant science)
>       will be carried out that might yield standardized repositories,
>       e.g. for biological context data. It is unclear, if other fields
>       of research (e.g. clinical, biomedical) have similar efforts, but
>       it is clear that currently, there are no efforts for (U.S.)
>       national repositories for metabolomics data although in Europe,
>       there are consortia (e.g. Metafor) that might be working into that
>       direction. So, for now this topic is an unsolved problem that
>       would require active research and funding to go forward.

Well, there is some stuff coming; funding is always tricky but 
we're nearly there at EBI for one (probably two years maximum to 
a working three-omics system).

A different approach is to build something that is just 
'FuGE-friendly' (FuGE was in NBT this month btw) like CPAS -- 
that way the domain specificity is sacrificed but nothing is 
lost per se. In the interim such an approach can fairly 
straightforwardly provide the repository needed, while we wait 
for more sophisticated resources; that said I'm not aware of 
many doing that at the moment apart from the Fred Hutch (CPAS) 
and a couple of universities in the UK (not for general use 
though). There is a strong argument that if the job is done 
unimpeachably with the MI documents and they achieve decent 
market penetration then you create a market for databases and 
tools from both the public and private sectors and thus (with a 
little encouragement) tools and DBs will 'appear'. Honest.

>     * /_other points
>       _/Before an email can be sent to the Metabolomics Soc members, it
>       is important to ensure that the *msi-sourceforge* pages are up to
>       date. Nigel will contact Susanna on that behalf to find out. The
>       pages might need to be revisited with respect to easiness of
>       communication and use – if we want to obtain feedback from members
>       (or other researchers), there must be low-entry ways to do so. A
>       wiki attached to the sourceforge pages was discussed or branching
>       out to the Metabolomics Society pages.
>       With respect to contacts to other efforts, especially the *MIBBI*
>       efforts led by Chris Taylor, Nigel will serve as contact point (in
>       addition to Susanna’s continuing efforts). It is anticipated that
>       there potentially might be significant synergies that could be
>       gathered from such unifying biology-focused reporting efforts.
>       Susanna has acquired funding for workshop(s) that will foster
>       efforts in this direction. //

Great to see that -- thanks. I'll of course have my brain 
engaged though and will be able to make future calls myself to 
represent MIBBI. I do think there's more to do for the MI groups 
though than ensure that the email address(es) work okay and such 
-- we need stated (in documents) mechanisms and structures -- we 
have to be a grown up body now in the manner of any of the large 
standards bodies. Confidence will be inspired in others by our 
having a visible process that works and that we cannot lightly 
alter. If there is even the chance that we could arbitrarily 
alter the 'release' version of these things, confidence 
(especially in commercial settings) will evaporate quicker than 
liquid helium at RTP.

Sorry for the length...

Cheers, Chris.

~~~~~~~~~~~~~~~~~~~~~~~~
  chr...@eb...
  http://mibbi.sf.net/
~~~~~~~~~~~~~~~~~~~~~~~~