You can subscribe to this list here.
2001 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(1) |
Oct
(43) |
Nov
(31) |
Dec
(113) |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2002 |
Jan
(63) |
Feb
(82) |
Mar
(71) |
Apr
(92) |
May
(94) |
Jun
(39) |
Jul
(75) |
Aug
(148) |
Sep
(202) |
Oct
(229) |
Nov
(179) |
Dec
(87) |
2003 |
Jan
(34) |
Feb
(29) |
Mar
(71) |
Apr
(37) |
May
(11) |
Jun
(41) |
Jul
(34) |
Aug
(30) |
Sep
(26) |
Oct
(32) |
Nov
(42) |
Dec
(35) |
2004 |
Jan
(42) |
Feb
(46) |
Mar
(14) |
Apr
(7) |
May
(14) |
Jun
(8) |
Jul
(15) |
Aug
(19) |
Sep
(20) |
Oct
(6) |
Nov
(8) |
Dec
(30) |
2005 |
Jan
(9) |
Feb
(13) |
Mar
(17) |
Apr
(4) |
May
(7) |
Jun
(26) |
Jul
(9) |
Aug
(8) |
Sep
(12) |
Oct
|
Nov
(2) |
Dec
(15) |
2006 |
Jan
(17) |
Feb
|
Mar
(4) |
Apr
|
May
(14) |
Jun
(20) |
Jul
(14) |
Aug
(9) |
Sep
(37) |
Oct
(4) |
Nov
(3) |
Dec
(2) |
2007 |
Jan
(23) |
Feb
(8) |
Mar
(35) |
Apr
(4) |
May
(1) |
Jun
(3) |
Jul
|
Aug
(4) |
Sep
|
Oct
|
Nov
(2) |
Dec
|
2008 |
Jan
(4) |
Feb
(3) |
Mar
(2) |
Apr
(2) |
May
(1) |
Jun
|
Jul
|
Aug
(5) |
Sep
|
Oct
|
Nov
(1) |
Dec
|
2009 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
(1) |
Dec
|
From: Chris S. <sto...@pc...> - 2007-03-23 17:57:02
|
Great idea. Chris On Mar 23, 2007, at 1:18 PM, Helen Parkinson wrote: > I suggest that we do some round tripping of MAGETAB. > > Don Maier wrote: >> Hi Junmin, >> >> As I mentioned, it will be great for us to have a chance to review >> your work with processing MIAME-compliant documents of various sorts. >> >> Your concern about MAGE-TAB compliance ties directly into the agenda >> item under my name: If we have one and only one parser (accepter) for >> MAGE-TAB, then every question of compliance is settled once and for >> everyone. >> >> Regards, >> Don Maier >> >> >> On Mar 23, 2007, at 9:16 AM, Junmin Liu wrote: >> >>> Hi, Don and Jamboree attendees, >>> Since last Oct., for one of the requirement of our in-house data >>> loading pipeline, we developed an extendable framework MR_Ti and a >>> set of toolkits to help loading MIAMI-compliant documents in >>> different formats into our GUS schema. Chris has suggested me to >>> present this on the Jamboree to get feedback and to see if it can be >>> useful to the communities. >>> >>> Note MAGE-Tab is part of our pipeline, we have started to generate >>> and load MAGE-Tabs to GUS, we will also like to show public the =20 >>> files >>> we have and make sure they are exchangable and compliant with >>> MAGE-Tab spec. 1.0 release. >>> >>> Thanks, >>> ---junmin >>> >>> >>> >>> On Wed, 21 Mar 2007, Don Maier wrote: >>> >>>> Hi friends, >>>> >>>> We're looking forward to seeing and meeting with you in next =20 >>>> week in >>>> sunny California. Current forecasts are for a cloudy Tuesday, but a >>>> sunny remainder of the week in the upper 60's. >>>> >>>> Here's my current list of attendees and agenda. Please let me know >>>> if you wish to contribute any more bodies or agenda items! >>>> >>>> Regards, >>>> Don Maier >>>> >>>> >>>> MAGE Jamboree: March 27 - 30 >>>> Stanford University: Clark Center, Rm. 362 >>>> >>>> Attendees: >>>> >>>> 1) Cathy Ball (SMD) >>>> 2) Cate Beauheim (Stanford) >>>> 3) Alvis Brazma (EBI) >>>> 4) Steve Chervitz (Affymetrix) >>>> 5) Junmin Liu (U. Penn) >>>> 6) Don Maier (SMD) >>>> 7) Tim Rayner (EBI) >>>> 8) Ugis Sarkans (EBI) >>>> 9) Paul Spellman (Lawrence Berkeley National Laboratory) >>>> 10) Joe White (Dana-Farber/Harvard Cancer Center) >>>> >>>> >>>> Agenda: >>>> >>>> 1) Tim =96 Final version of MAGE-TAB >>>> >>>> 2) Alvis/Ugis =96 MAGE-TAB object model: >>>> >>>> a) Possible move to FUGE-TAB >>>> >>>> b) Discussion of basic concepts (no formal proposal yet). >>>> >>>> 3) Don =96 The =93one-parser=94 requirement and how to satisfy it = for >>>> MAGE-TAB: >>>> >>>> a) There is thought to be a strong requirement to have exactly one >>>> MAGE-TAB parser in one body of code in one language that can be =20 >>>> used >>>> from either a Java or a Perl programming environment. Satisfying >>>> this requirement would ensure that MAGE-TAB never destroys =20 >>>> itself on >>>> the shoals of disagreeing acceptors. >>>> >>>> b) Proposal for how to satisfy this requirement. >>> --------------------------------------------------------------------=20= >>> ----- >>> Take Surveys. Earn Cash. Influence the Future of IT >>> Join SourceForge.net's Techsay panel and you'll get the chance to >>> share your >>> opinions on IT & business topics through brief surveys-and earn cash >>> http://www.techsay.com/default.php?=20 >>> page=3Djoin.php&p=3Dsourceforge&CID=3DDEVDEV__________________________= ____=20 >>> _________________ >>> <http://www.techsay.com/default.php?=20 >>> page=3Djoin.php&p=3Dsourceforge&CID=3DDEVDEV__________________________= ____=20 >>> _________________> >>> Mged-mage mailing list >>> Mge...@li... <mailto:Mged-=20 >>> ma...@li...> >>> https://lists.sourceforge.net/lists/listinfo/mged-mage >> >> Don Maier >> Sr. Software Designer/Research >> Dept. of Biochemistry >> Stanford University School of Medicine >> >> >> ---------------------------------------------------------------------=20= >> --- >> >> ---------------------------------------------------------------------=20= >> ---- >> Take Surveys. Earn Cash. Influence the Future of IT >> Join SourceForge.net's Techsay panel and you'll get the chance to =20 >> share your >> opinions on IT & business topics through brief surveys-and earn cash >> http://www.techsay.com/default.php?=20 >> page=3Djoin.php&p=3Dsourceforge&CID=3DDEVDEV >> ---------------------------------------------------------------------=20= >> --- >> >> _______________________________________________ >> Mged-mage mailing list >> Mge...@li... >> https://lists.sourceforge.net/lists/listinfo/mged-mage >> > > --=20 > Helen Parkinson, PhD > Curation Coordinator > Microarray Informatics Team, > EBI > > EBI 01223 494672 > Skype: helen.parkinson.ebi > > > ----------------------------------------------------------------------=20= > --- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to =20 > share your > opinions on IT & business topics through brief surveys-and earn cash > http://www.techsay.com/default.php?=20 > page=3Djoin.php&p=3Dsourceforge&CID=3DDEVDEV > _______________________________________________ > Mged-mage mailing list > Mge...@li... > https://lists.sourceforge.net/lists/listinfo/mged-mage |
From: Miller, M. D (Rosetta) <Michael_Miller@Rosettabio.com> - 2007-03-23 17:51:49
|
hi don, =20 as this is the real world, one cannot dictate that developers cannot go off on their own to do what ever they want. so my thought is that realistically we can not require something that is unlikely to be satisfied. =20 what we can do is release THE official validating parser which if there is a question as to whether a set of files that claims to be MAGE_TAB compliant is or isn't, it will definitively answer yea or nay. =20 if someone were to find this official parser wasn't efficient enough in the context of their application they would not be doing their job to develop one that did work fast enough for their application. but having this official validating parser would be wonderful to make sure the in-house application stayed compliant. =20 for instance, in-house, we have our own version of the java MAGEstk parser which is much more efficient and faster than the one in the official stk. (actually i'd like to put it in the official stk code but it got caught up in some unnecessary imports that prevent releasing it until those can be removed.) =20 cheers, michael =20 ________________________________ From: Don Maier [mailto:dM...@ge...]=20 Sent: Friday, March 23, 2007 10:31 AM To: Miller, Michael D (Rosetta) Cc: Junmin Liu; mged-mage2; MGED-mage Subject: Re: [Mged-mage] [Mged-mage2] MGED Jamboree next week - March 27-30! =09 =09 Hi Michael,=20 Thank you for your enthusiastic support for the "one-parser" requirement -- at least, in your initial statement about it. Two comments about your comments: i) There likely will be a performance tradeoff. In my view, any modest performance cost is utterly trivial compared to the gain in solving this problem once and for everyone. Of course, there were those who invoked the specter of performance to justify continuing continuing careers in assembly language programming. ii) The architecture of a push parser makes it possible to perform any set of "custom" actions that are conjoined with the parse. This includes performing operations on your favorite database and flagging constructs that an installation prefers not to support. There *is* a problem, a really big problem, with having more than one parser. Two is too many. I hope that I can clearly convey this to the group. Regards, Don M. On Mar 23, 2007, at 9:55 AM, Miller, Michael D (Rosetta) wrote: hi junmin, that's great work, if i could get a copy of the presentation when it's ready, that would be much appreciated. hi don, a) There is thought to be a strong requirement to have=20 exactly one=20 MAGE-TAB parser in one body of code in one language that=20 can be used from=20 either a Java or a Perl programming environment. =20 Satisfying this requirement=20 would ensure that MAGE-TAB never destroys itself on the=20 shoals of disagreeing=20 acceptors. i think this is a great requirement, similar to the OMG compliancy section, but i have one caveat with the wording. yes, we definitely should have as a goal a definitive MAGE-TAB parser that has a mode that verifies that a set of files is MAGE-TAB compliant (including resolving the references between the files). And it should have a mode that developers could certainly use as the basis of their import implementation. exactly one=20 MAGE-TAB parser in one body of code in one language that=20 can be used from=20 either a Java or a Perl programming environment. =20 my caveat is that there is no problem having more than one, often for efficiency reasons for an application there would be a parser tuned to specific database schema and additional application rules for standard, compliant MAGE-TAB files (for instance, the application rules might restrict MAGE-TAB to require certain columns for the application that in general aren't required). i would change the wording to: exactly one official validating MAGE-TAB parser in one body of code in one language that=20 can also be used from=20 either a Java or a Perl programming environment by any import application. note that if it is in one language there will be by necessity, i think, a bit of an impedance using it in a different language. cheers, michael -----Original Message----- From: mge...@li...=20 =09 [mailto:mge...@li...] On Behalf=20 Of Junmin Liu Sent: Friday, March 23, 2007 9:17 AM To: Don Maier Cc: mged-mage2; MGED-mage Subject: Re: [Mged-mage2] MGED Jamboree next week - March 27-30! Hi, Don and Jamboree attendees, Since last Oct., for one of the requirement of our in-house=20 data loading=20 pipeline, we developed an extendable framework MR_Ti and a set of=20 toolkits to help loading MIAMI-compliant documents in=20 different formats=20 into our GUS schema. Chris has suggested me to present this on the=20 Jamboree to get feedback and to see if it can be useful to the=20 communities. Note MAGE-Tab is part of our pipeline, we have started to=20 generate and=20 load MAGE-Tabs to GUS, we will also like to show public the files we=20 have and make sure they are exchangable and compliant with=20 MAGE-Tab spec.=20 1.0 release. Thanks, ---junmin On Wed, 21 Mar 2007, Don Maier wrote: Hi friends, We're looking forward to seeing and meeting with you in=20 next week in sunny=20 California. Current forecasts are for a cloudy Tuesday,=20 but a sunny=20 remainder of the week in the upper 60's. Here's my current list of attendees and agenda. Please=20 let me know if you=20 wish to contribute any more bodies or agenda items! Regards, Don Maier MAGE Jamboree: March 27 - 30 Stanford University: Clark Center, Rm. 362 Attendees: 1) Cathy Ball (SMD) 2) Cate Beauheim (Stanford) 3) Alvis Brazma (EBI) 4) Steve Chervitz (Affymetrix) 5) Junmin Liu (U. Penn) 6) Don Maier (SMD) 7) Tim Rayner (EBI) 8) Ugis Sarkans (EBI) 9) Paul Spellman (Lawrence Berkeley National Laboratory) 10) Joe White (Dana-Farber/Harvard Cancer Center) Agenda: 1) Tim - Final version of MAGE-TAB 2) Alvis/Ugis - MAGE-TAB object model: a) Possible move to FUGE-TAB b) Discussion of basic concepts (no formal proposal yet). 3) Don - The "one-parser" requirement and how to satisfy=20 it for MAGE-TAB: a) There is thought to be a strong requirement to have=20 exactly one=20 MAGE-TAB parser in one body of code in one language that=20 can be used from=20 either a Java or a Perl programming environment. =20 Satisfying this requirement=20 would ensure that MAGE-TAB never destroys itself on the=20 shoals of disagreeing=20 acceptors. b) Proposal for how to satisfy this requirement. =09 ------------------------------------------------------------------------ - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash =09 http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge&CID=3D= DEVDE V _______________________________________________ Mged-mage mailing list Mge...@li... https://lists.sourceforge.net/lists/listinfo/mged-mage =09 =09 |
From: Don M. <dM...@ge...> - 2007-03-23 17:31:04
|
Hi Michael, Thank you for your enthusiastic support for the "one-parser" requirement -- at least, in your initial statement about it. Two comments about your comments: i) There likely will be a performance tradeoff. In my view, any modest performance cost is utterly trivial compared to the gain in solving this problem once and for everyone. Of course, there were those who invoked the specter of performance to justify continuing continuing careers in assembly language programming. ii) The architecture of a push parser makes it possible to perform any set of "custom" actions that are conjoined with the parse. This includes performing operations on your favorite database and flagging constructs that an installation prefers not to support. There *is* a problem, a really big problem, with having more than one parser. Two is too many. I hope that I can clearly convey this to the group. Regards, Don M. On Mar 23, 2007, at 9:55 AM, Miller, Michael D (Rosetta) wrote: > hi junmin, > > that's great work, if i could get a copy of the presentation when it's > ready, that would be much appreciated. > > hi don, > >>> a) There is thought to be a strong requirement to have >> exactly one >>> MAGE-TAB parser in one body of code in one language that >> can be used from >>> either a Java or a Perl programming environment. >> Satisfying this requirement >>> would ensure that MAGE-TAB never destroys itself on the >> shoals of disagreeing >>> acceptors. > > i think this is a great requirement, similar to the OMG compliancy > section, but i have one caveat with the wording. > > yes, we definitely should have as a goal a definitive MAGE-TAB parser > that has a mode that verifies that a set of files is MAGE-TAB > compliant > (including resolving the references between the files). And it should > have a mode that developers could certainly use as the basis of their > import implementation. > >> exactly one >>> MAGE-TAB parser in one body of code in one language that >> can be used from >>> either a Java or a Perl programming environment. > > my caveat is that there is no problem having more than one, often for > efficiency reasons for an application there would be a parser tuned to > specific database schema and additional application rules for > standard, > compliant MAGE-TAB files (for instance, the application rules might > restrict MAGE-TAB to require certain columns for the application > that in > general aren't required). > > i would change the wording to: > >> exactly one official validating MAGE-TAB parser in one body of >> code in > one language that >> can also be used from >>> either a Java or a Perl programming environment by any import > application. > > note that if it is in one language there will be by necessity, i > think, > a bit of an impedance using it in a different language. > > cheers, > michael > > >> -----Original Message----- >> From: mge...@li... >> [mailto:mge...@li...] On Behalf >> Of Junmin Liu >> Sent: Friday, March 23, 2007 9:17 AM >> To: Don Maier >> Cc: mged-mage2; MGED-mage >> Subject: Re: [Mged-mage2] MGED Jamboree next week - March 27-30! >> >> Hi, Don and Jamboree attendees, >> Since last Oct., for one of the requirement of our in-house >> data loading >> pipeline, we developed an extendable framework MR_Ti and a set of >> toolkits to help loading MIAMI-compliant documents in >> different formats >> into our GUS schema. Chris has suggested me to present this on the >> Jamboree to get feedback and to see if it can be useful to the >> communities. >> >> Note MAGE-Tab is part of our pipeline, we have started to >> generate and >> load MAGE-Tabs to GUS, we will also like to show public the files we >> have and make sure they are exchangable and compliant with >> MAGE-Tab spec. >> 1.0 release. >> >> Thanks, >> ---junmin >> >> >> >> On Wed, 21 Mar 2007, Don Maier wrote: >> >>> Hi friends, >>> >>> We're looking forward to seeing and meeting with you in >> next week in sunny >>> California. Current forecasts are for a cloudy Tuesday, >> but a sunny >>> remainder of the week in the upper 60's. >>> >>> Here's my current list of attendees and agenda. Please >> let me know if you >>> wish to contribute any more bodies or agenda items! >>> >>> Regards, >>> Don Maier >>> >>> >>> MAGE Jamboree: March 27 - 30 >>> Stanford University: Clark Center, Rm. 362 >>> >>> Attendees: >>> >>> 1) Cathy Ball (SMD) >>> 2) Cate Beauheim (Stanford) >>> 3) Alvis Brazma (EBI) >>> 4) Steve Chervitz (Affymetrix) >>> 5) Junmin Liu (U. Penn) >>> 6) Don Maier (SMD) >>> 7) Tim Rayner (EBI) >>> 8) Ugis Sarkans (EBI) >>> 9) Paul Spellman (Lawrence Berkeley National Laboratory) >>> 10) Joe White (Dana-Farber/Harvard Cancer Center) >>> >>> >>> Agenda: >>> >>> 1) Tim - Final version of MAGE-TAB >>> >>> 2) Alvis/Ugis - MAGE-TAB object model: >>> >>> a) Possible move to FUGE-TAB >>> >>> b) Discussion of basic concepts (no formal proposal yet). >>> >>> 3) Don - The "one-parser" requirement and how to satisfy >> it for MAGE-TAB: >>> >>> a) There is thought to be a strong requirement to have >> exactly one >>> MAGE-TAB parser in one body of code in one language that >> can be used from >>> either a Java or a Perl programming environment. >> Satisfying this requirement >>> would ensure that MAGE-TAB never destroys itself on the >> shoals of disagreeing >>> acceptors. >>> >>> b) Proposal for how to satisfy this requirement. >> > > > ---------------------------------------------------------------------- > --- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to > share your > opinions on IT & business topics through brief surveys-and earn cash > http://www.techsay.com/default.php? > page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Mged-mage mailing list > Mge...@li... > https://lists.sourceforge.net/lists/listinfo/mged-mage |
From: Helen P. <par...@eb...> - 2007-03-23 17:18:30
|
I suggest that we do some round tripping of MAGETAB. Don Maier wrote: > Hi Junmin, > > As I mentioned, it will be great for us to have a chance to review > your work with processing MIAME-compliant documents of various sorts. > > Your concern about MAGE-TAB compliance ties directly into the agenda > item under my name: If we have one and only one parser (accepter) for > MAGE-TAB, then every question of compliance is settled once and for > everyone. > > Regards, > Don Maier > > > On Mar 23, 2007, at 9:16 AM, Junmin Liu wrote: > >> Hi, Don and Jamboree attendees, >> Since last Oct., for one of the requirement of our in-house data >> loading pipeline, we developed an extendable framework MR_Ti and a >> set of toolkits to help loading MIAMI-compliant documents in >> different formats into our GUS schema. Chris has suggested me to >> present this on the Jamboree to get feedback and to see if it can be >> useful to the communities. >> >> Note MAGE-Tab is part of our pipeline, we have started to generate >> and load MAGE-Tabs to GUS, we will also like to show public the files >> we have and make sure they are exchangable and compliant with >> MAGE-Tab spec. 1.0 release. >> >> Thanks, >> ---junmin >> >> >> >> On Wed, 21 Mar 2007, Don Maier wrote: >> >>> Hi friends, >>> >>> We're looking forward to seeing and meeting with you in next week in >>> sunny California. Current forecasts are for a cloudy Tuesday, but a >>> sunny remainder of the week in the upper 60's. >>> >>> Here's my current list of attendees and agenda. Please let me know >>> if you wish to contribute any more bodies or agenda items! >>> >>> Regards, >>> Don Maier >>> >>> >>> MAGE Jamboree: March 27 - 30 >>> Stanford University: Clark Center, Rm. 362 >>> >>> Attendees: >>> >>> 1) Cathy Ball (SMD) >>> 2) Cate Beauheim (Stanford) >>> 3) Alvis Brazma (EBI) >>> 4) Steve Chervitz (Affymetrix) >>> 5) Junmin Liu (U. Penn) >>> 6) Don Maier (SMD) >>> 7) Tim Rayner (EBI) >>> 8) Ugis Sarkans (EBI) >>> 9) Paul Spellman (Lawrence Berkeley National Laboratory) >>> 10) Joe White (Dana-Farber/Harvard Cancer Center) >>> >>> >>> Agenda: >>> >>> 1) Tim – Final version of MAGE-TAB >>> >>> 2) Alvis/Ugis – MAGE-TAB object model: >>> >>> a) Possible move to FUGE-TAB >>> >>> b) Discussion of basic concepts (no formal proposal yet). >>> >>> 3) Don – The “one-parser” requirement and how to satisfy it for >>> MAGE-TAB: >>> >>> a) There is thought to be a strong requirement to have exactly one >>> MAGE-TAB parser in one body of code in one language that can be used >>> from either a Java or a Perl programming environment. Satisfying >>> this requirement would ensure that MAGE-TAB never destroys itself on >>> the shoals of disagreeing acceptors. >>> >>> b) Proposal for how to satisfy this requirement. >> ------------------------------------------------------------------------- >> Take Surveys. Earn Cash. Influence the Future of IT >> Join SourceForge.net's Techsay panel and you'll get the chance to >> share your >> opinions on IT & business topics through brief surveys-and earn cash >> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV_______________________________________________ >> <http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV_______________________________________________> >> Mged-mage mailing list >> Mge...@li... <mailto:Mge...@li...> >> https://lists.sourceforge.net/lists/listinfo/mged-mage > > Don Maier > Sr. Software Designer/Research > Dept. of Biochemistry > Stanford University School of Medicine > > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys-and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > ------------------------------------------------------------------------ > > _______________________________________________ > Mged-mage mailing list > Mge...@li... > https://lists.sourceforge.net/lists/listinfo/mged-mage > -- Helen Parkinson, PhD Curation Coordinator Microarray Informatics Team, EBI EBI 01223 494672 Skype: helen.parkinson.ebi |
From: Don M. <dM...@ge...> - 2007-03-23 17:05:12
|
Hi Junmin, As I mentioned, it will be great for us to have a chance to review =20 your work with processing MIAME-compliant documents of various sorts. Your concern about MAGE-TAB compliance ties directly into the agenda =20 item under my name: If we have one and only one parser (accepter) =20 for MAGE-TAB, then every question of compliance is settled once and =20 for everyone. Regards, Don Maier On Mar 23, 2007, at 9:16 AM, Junmin Liu wrote: > Hi, Don and Jamboree attendees, > Since last Oct., for one of the requirement of our in-house data =20 > loading pipeline, we developed an extendable framework MR_Ti and a =20 > set of toolkits to help loading MIAMI-compliant documents in =20 > different formats into our GUS schema. Chris has suggested me to =20 > present this on the Jamboree to get feedback and to see if it can =20 > be useful to the communities. > > Note MAGE-Tab is part of our pipeline, we have started to generate =20 > and load MAGE-Tabs to GUS, we will also like to show public the =20 > files we have and make sure they are exchangable and compliant with =20= > MAGE-Tab spec. 1.0 release. > > Thanks, > ---junmin > > > > On Wed, 21 Mar 2007, Don Maier wrote: > >> Hi friends, >> >> We're looking forward to seeing and meeting with you in next week =20 >> in sunny California. Current forecasts are for a cloudy Tuesday, =20= >> but a sunny remainder of the week in the upper 60's. >> >> Here's my current list of attendees and agenda. Please let me =20 >> know if you wish to contribute any more bodies or agenda items! >> >> Regards, >> Don Maier >> >> >> MAGE Jamboree: March 27 - 30 >> Stanford University: Clark Center, Rm. 362 >> >> Attendees: >> >> 1) Cathy Ball (SMD) >> 2) Cate Beauheim (Stanford) >> 3) Alvis Brazma (EBI) >> 4) Steve Chervitz (Affymetrix) >> 5) Junmin Liu (U. Penn) >> 6) Don Maier (SMD) >> 7) Tim Rayner (EBI) >> 8) Ugis Sarkans (EBI) >> 9) Paul Spellman (Lawrence Berkeley National Laboratory) >> 10) Joe White (Dana-Farber/Harvard Cancer Center) >> >> >> Agenda: >> >> 1) Tim =96 Final version of MAGE-TAB >> >> 2) Alvis/Ugis =96 MAGE-TAB object model: >> >> a) Possible move to FUGE-TAB >> >> b) Discussion of basic concepts (no formal proposal yet). >> >> 3) Don =96 The =93one-parser=94 requirement and how to satisfy it = for =20 >> MAGE-TAB: >> >> a) There is thought to be a strong requirement to have exactly =20= >> one MAGE-TAB parser in one body of code in one language that can =20 >> be used from either a Java or a Perl programming environment. =20 >> Satisfying this requirement would ensure that MAGE-TAB never =20 >> destroys itself on the shoals of disagreeing acceptors. >> >> b) Proposal for how to satisfy this requirement. > ----------------------------------------------------------------------=20= > --- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to =20 > share your > opinions on IT & business topics through brief surveys-and earn cash > http://www.techsay.com/default.php?=20 > page=3Djoin.php&p=3Dsourceforge&CID=3DDEVDEV____________________________= ____=20 > _______________ > Mged-mage mailing list > Mge...@li... > https://lists.sourceforge.net/lists/listinfo/mged-mage Don Maier Sr. Software Designer/Research Dept. of Biochemistry Stanford University School of Medicine |
From: Miller, M. D (Rosetta) <Michael_Miller@Rosettabio.com> - 2007-03-23 16:55:49
|
hi junmin, that's great work, if i could get a copy of the presentation when it's ready, that would be much appreciated. hi don, > > a) There is thought to be a strong requirement to have=20 > exactly one=20 > > MAGE-TAB parser in one body of code in one language that=20 > can be used from=20 > > either a Java or a Perl programming environment. =20 > Satisfying this requirement=20 > > would ensure that MAGE-TAB never destroys itself on the=20 > shoals of disagreeing=20 > > acceptors. i think this is a great requirement, similar to the OMG compliancy section, but i have one caveat with the wording. yes, we definitely should have as a goal a definitive MAGE-TAB parser that has a mode that verifies that a set of files is MAGE-TAB compliant (including resolving the references between the files). And it should have a mode that developers could certainly use as the basis of their import implementation. > exactly one=20 > > MAGE-TAB parser in one body of code in one language that=20 > can be used from=20 > > either a Java or a Perl programming environment. =20 my caveat is that there is no problem having more than one, often for efficiency reasons for an application there would be a parser tuned to specific database schema and additional application rules for standard, compliant MAGE-TAB files (for instance, the application rules might restrict MAGE-TAB to require certain columns for the application that in general aren't required). i would change the wording to: > exactly one official validating MAGE-TAB parser in one body of code in one language that=20 > can also be used from=20 > > either a Java or a Perl programming environment by any import application. note that if it is in one language there will be by necessity, i think, a bit of an impedance using it in a different language. cheers, michael =20 > -----Original Message----- > From: mge...@li...=20 > [mailto:mge...@li...] On Behalf=20 > Of Junmin Liu > Sent: Friday, March 23, 2007 9:17 AM > To: Don Maier > Cc: mged-mage2; MGED-mage > Subject: Re: [Mged-mage2] MGED Jamboree next week - March 27-30! >=20 > Hi, Don and Jamboree attendees, > Since last Oct., for one of the requirement of our in-house=20 > data loading=20 > pipeline, we developed an extendable framework MR_Ti and a set of=20 > toolkits to help loading MIAMI-compliant documents in=20 > different formats=20 > into our GUS schema. Chris has suggested me to present this on the=20 > Jamboree to get feedback and to see if it can be useful to the=20 > communities. >=20 > Note MAGE-Tab is part of our pipeline, we have started to=20 > generate and=20 > load MAGE-Tabs to GUS, we will also like to show public the files we=20 > have and make sure they are exchangable and compliant with=20 > MAGE-Tab spec.=20 > 1.0 release. >=20 > Thanks, > ---junmin >=20 >=20 >=20 > On Wed, 21 Mar 2007, Don Maier wrote: >=20 > > Hi friends, > > > > We're looking forward to seeing and meeting with you in=20 > next week in sunny=20 > > California. Current forecasts are for a cloudy Tuesday,=20 > but a sunny=20 > > remainder of the week in the upper 60's. > > > > Here's my current list of attendees and agenda. Please=20 > let me know if you=20 > > wish to contribute any more bodies or agenda items! > > > > Regards, > > Don Maier > > > > > > MAGE Jamboree: March 27 - 30 > > Stanford University: Clark Center, Rm. 362 > > > > Attendees: > > > > 1) Cathy Ball (SMD) > > 2) Cate Beauheim (Stanford) > > 3) Alvis Brazma (EBI) > > 4) Steve Chervitz (Affymetrix) > > 5) Junmin Liu (U. Penn) > > 6) Don Maier (SMD) > > 7) Tim Rayner (EBI) > > 8) Ugis Sarkans (EBI) > > 9) Paul Spellman (Lawrence Berkeley National Laboratory) > > 10) Joe White (Dana-Farber/Harvard Cancer Center) > > > > > > Agenda: > > > > 1) Tim - Final version of MAGE-TAB > > > > 2) Alvis/Ugis - MAGE-TAB object model: > > > > a) Possible move to FUGE-TAB > > > > b) Discussion of basic concepts (no formal proposal yet). > > > > 3) Don - The "one-parser" requirement and how to satisfy=20 > it for MAGE-TAB: > > > > a) There is thought to be a strong requirement to have=20 > exactly one=20 > > MAGE-TAB parser in one body of code in one language that=20 > can be used from=20 > > either a Java or a Perl programming environment. =20 > Satisfying this requirement=20 > > would ensure that MAGE-TAB never destroys itself on the=20 > shoals of disagreeing=20 > > acceptors. > > > > b) Proposal for how to satisfy this requirement. >=20 |
From: Junmin L. <ju...@pc...> - 2007-03-23 16:19:23
|
Hi, Tim, Upon we release MAGE-Tab 1.0 spec in Stanford, I will make sure the mage2tab converter will be upgraded and generate the right MAGE-Tab. Thanks, ---junmin On Fri, 23 Mar 2007, Tim Rayner wrote: > Hi Junmin, > > Thanks for the email; just a quick question - is there an updated version of > the mage2tab code you would like us to put on the SF page before the > jamboree? I think the latest version we have is 0.6. > > Cheers, > > Tim > > On Fri, 23 Mar 2007, Junmin Liu wrote: > >> Hi, Tim and all, >> We, here in CBIL, also developed a suite of toolkit for MAGE-Tab, actually >> potentially for any kind of MIAMI-complianct mage doc including MAGE-ML, >> SOFT and MAGE-Tab. >> >> Included in the toolkit are parsers for mage-ml, mage-tab, soft (soon to >> completed), writers for mage-tab, mage-ml, soft (need to implemented), >> conversion,visualization and validation tool, GUI interface on top of >> those tools. >> >> Note we developed a simple object model for consuming the mage-tab, the >> mage-tab reader will deserialize the mage-tab into the objects. And I >> understand there are similar agenda items on stanford workshop for these.. >> >> The mage2tab we put on tab2mage sourceforge site is part of the toolkit. >> We already heavily use the toolkit as part of data loading pipeline. We >> will figure out how to release the toolkit very soon. >> >> ---junmin >> >> On Fri, 23 Mar 2007, Tim Rayner wrote: >> >>> Hi, >>> >>> I'm pleased to be able to announce the initial release of our MAGE-TAB >>> parser on SourceForge. The parsing scripts, implemented in Perl, are >>> currently rather alpha-grade (or beta if you're charitable). The code is >>> available from the following web page: >>> >>> http://sourceforge.net/project/showfiles.php?group_id=120325&package_id=149795&release_id=495838 >>> >>> The scripts are included in the Tab2MAGE development package, version >>> 1.9.7. Installation and usage instructions are included with the download >>> (see also this link: http://tab2mage.sourceforge.net/#magetab). >>> >>> We have also made available an early release of a script, written by >>> Faisal Rezwan in our group, which can be used to convert Tab2MAGE >>> spreadsheets into MAGE-TAB IDF and SDRF documents: >>> >>> http://sourceforge.net/project/showfiles.php?group_id=120325&package_id=226041&release_id=495679 >>> >>> Cheers, >>> >>> Tim >>> >>> -- >>> Tim Rayner >>> ArrayExpress >>> >>> ------------------------------------------------------------------------- >>> Take Surveys. Earn Cash. Influence the Future of IT >>> Join SourceForge.net's Techsay panel and you'll get the chance to share >>> your >>> opinions on IT & business topics through brief surveys-and earn cash >>> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV >>> _______________________________________________ >>> Mged-mage mailing list >>> Mge...@li... >>> https://lists.sourceforge.net/lists/listinfo/mged-mage >>> >> >> ------------------------------------------------------------------------- >> Take Surveys. Earn Cash. Influence the Future of IT >> Join SourceForge.net's Techsay panel and you'll get the chance to share >> your >> opinions on IT & business topics through brief surveys-and earn cash >> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV >> _______________________________________________ >> Mged-mage mailing list >> Mge...@li... >> https://lists.sourceforge.net/lists/listinfo/mged-mage >> > |
From: Junmin L. <ju...@pc...> - 2007-03-23 16:17:00
|
Hi, Don and Jamboree attendees, Since last Oct., for one of the requirement of our in-house data loading=20 pipeline, we developed an extendable framework MR_Ti and a set of=20 toolkits to help loading MIAMI-compliant documents in different formats=20 into our GUS schema. Chris has suggested me to present this on the=20 Jamboree to get feedback and to see if it can be useful to the=20 communities. Note MAGE-Tab is part of our pipeline, we have started to generate and=20 load MAGE-Tabs to GUS, we will also like to show public the files we=20 have and make sure they are exchangable and compliant with MAGE-Tab spec.= =20 1.0 release. Thanks, ---junmin On Wed, 21 Mar 2007, Don Maier wrote: > Hi friends, > > We're looking forward to seeing and meeting with you in next week in sunn= y=20 > California. Current forecasts are for a cloudy Tuesday, but a sunny=20 > remainder of the week in the upper 60's. > > Here's my current list of attendees and agenda. Please let me know if y= ou=20 > wish to contribute any more bodies or agenda items! > > Regards, > Don Maier > > > MAGE Jamboree: March 27 - 30 > Stanford University: Clark Center, Rm. 362 > > Attendees: > > 1) Cathy Ball (SMD) > 2) Cate Beauheim (Stanford) > 3) Alvis Brazma (EBI) > 4) Steve Chervitz (Affymetrix) > 5) Junmin Liu (U. Penn) > 6) Don Maier (SMD) > 7) Tim Rayner (EBI) > 8) Ugis Sarkans (EBI) > 9) Paul Spellman (Lawrence Berkeley National Laboratory) > 10) Joe White (Dana-Farber/Harvard Cancer Center) > > > Agenda: > > 1) Tim =96 Final version of MAGE-TAB > > 2) Alvis/Ugis =96 MAGE-TAB object model: > > a) Possible move to FUGE-TAB > > b) Discussion of basic concepts (no formal proposal yet). > > 3) Don =96 The =93one-parser=94 requirement and how to satisfy it for = MAGE-TAB: > > a) There is thought to be a strong requirement to have exactly one=20 > MAGE-TAB parser in one body of code in one language that can be used from= =20 > either a Java or a Perl programming environment. Satisfying this require= ment=20 > would ensure that MAGE-TAB never destroys itself on the shoals of disagre= eing=20 > acceptors. > > b) Proposal for how to satisfy this requirement. |
From: Tim R. <ra...@eb...> - 2007-03-23 15:32:41
|
Hi Junmin, Thanks for the email; just a quick question - is there an updated version of the mage2tab code you would like us to put on the SF page before the jamboree? I think the latest version we have is 0.6. Cheers, Tim On Fri, 23 Mar 2007, Junmin Liu wrote: > Hi, Tim and all, > We, here in CBIL, also developed a suite of toolkit for MAGE-Tab, actually > potentially for any kind of MIAMI-complianct mage doc including MAGE-ML, > SOFT and MAGE-Tab. > > Included in the toolkit are parsers for mage-ml, mage-tab, soft (soon to > completed), writers for mage-tab, mage-ml, soft (need to implemented), > conversion,visualization and validation tool, GUI interface on top of > those tools. > > Note we developed a simple object model for consuming the mage-tab, the > mage-tab reader will deserialize the mage-tab into the objects. And I > understand there are similar agenda items on stanford workshop for these.. > > The mage2tab we put on tab2mage sourceforge site is part of the toolkit. > We already heavily use the toolkit as part of data loading pipeline. We > will figure out how to release the toolkit very soon. > > ---junmin > > On Fri, 23 Mar 2007, Tim Rayner wrote: > >> Hi, >> >> I'm pleased to be able to announce the initial release of our MAGE-TAB >> parser on SourceForge. The parsing scripts, implemented in Perl, are >> currently rather alpha-grade (or beta if you're charitable). The code is >> available from the following web page: >> >> http://sourceforge.net/project/showfiles.php?group_id=120325&package_id=149795&release_id=495838 >> >> The scripts are included in the Tab2MAGE development package, version >> 1.9.7. Installation and usage instructions are included with the download >> (see also this link: http://tab2mage.sourceforge.net/#magetab). >> >> We have also made available an early release of a script, written by >> Faisal Rezwan in our group, which can be used to convert Tab2MAGE >> spreadsheets into MAGE-TAB IDF and SDRF documents: >> >> http://sourceforge.net/project/showfiles.php?group_id=120325&package_id=226041&release_id=495679 >> >> Cheers, >> >> Tim >> >> -- >> Tim Rayner >> ArrayExpress >> >> ------------------------------------------------------------------------- >> Take Surveys. Earn Cash. Influence the Future of IT >> Join SourceForge.net's Techsay panel and you'll get the chance to share your >> opinions on IT & business topics through brief surveys-and earn cash >> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV >> _______________________________________________ >> Mged-mage mailing list >> Mge...@li... >> https://lists.sourceforge.net/lists/listinfo/mged-mage >> > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys-and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Mged-mage mailing list > Mge...@li... > https://lists.sourceforge.net/lists/listinfo/mged-mage > |
From: Junmin L. <ju...@pc...> - 2007-03-23 15:18:51
|
Hi, Tim and all, We, here in CBIL, also developed a suite of toolkit for MAGE-Tab, actually potentially for any kind of MIAMI-complianct mage doc including MAGE-ML, SOFT and MAGE-Tab. Included in the toolkit are parsers for mage-ml, mage-tab, soft (soon to completed), writers for mage-tab, mage-ml, soft (need to implemented), conversion,visualization and validation tool, GUI interface on top of those tools. Note we developed a simple object model for consuming the mage-tab, the mage-tab reader will deserialize the mage-tab into the objects. And I understand there are similar agenda items on stanford workshop for these.. The mage2tab we put on tab2mage sourceforge site is part of the toolkit. We already heavily use the toolkit as part of data loading pipeline. We will figure out how to release the toolkit very soon. ---junmin On Fri, 23 Mar 2007, Tim Rayner wrote: > Hi, > > I'm pleased to be able to announce the initial release of our MAGE-TAB > parser on SourceForge. The parsing scripts, implemented in Perl, are > currently rather alpha-grade (or beta if you're charitable). The code is > available from the following web page: > > http://sourceforge.net/project/showfiles.php?group_id=120325&package_id=149795&release_id=495838 > > The scripts are included in the Tab2MAGE development package, version > 1.9.7. Installation and usage instructions are included with the download > (see also this link: http://tab2mage.sourceforge.net/#magetab). > > We have also made available an early release of a script, written by > Faisal Rezwan in our group, which can be used to convert Tab2MAGE > spreadsheets into MAGE-TAB IDF and SDRF documents: > > http://sourceforge.net/project/showfiles.php?group_id=120325&package_id=226041&release_id=495679 > > Cheers, > > Tim > > -- > Tim Rayner > ArrayExpress > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys-and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Mged-mage mailing list > Mge...@li... > https://lists.sourceforge.net/lists/listinfo/mged-mage > |
From: Tim R. <ra...@eb...> - 2007-03-23 11:56:16
|
Hi, I'm pleased to be able to announce the initial release of our MAGE-TAB parser on SourceForge. The parsing scripts, implemented in Perl, are currently rather alpha-grade (or beta if you're charitable). The code is available from the following web page: http://sourceforge.net/project/showfiles.php?group_id=120325&package_id=149795&release_id=495838 The scripts are included in the Tab2MAGE development package, version 1.9.7. Installation and usage instructions are included with the download (see also this link: http://tab2mage.sourceforge.net/#magetab). We have also made available an early release of a script, written by Faisal Rezwan in our group, which can be used to convert Tab2MAGE spreadsheets into MAGE-TAB IDF and SDRF documents: http://sourceforge.net/project/showfiles.php?group_id=120325&package_id=226041&release_id=495679 Cheers, Tim -- Tim Rayner ArrayExpress |
From: Don M. <dM...@ge...> - 2007-03-21 22:13:05
|
Hi friends, We're looking forward to seeing and meeting with you in next week in =20 sunny California. Current forecasts are for a cloudy Tuesday, but a =20= sunny remainder of the week in the upper 60's. Here's my current list of attendees and agenda. Please let me know =20 if you wish to contribute any more bodies or agenda items! Regards, Don Maier MAGE Jamboree: March 27 - 30 Stanford University: Clark Center, Rm. 362 Attendees: 1) Cathy Ball (SMD) 2) Cate Beauheim (Stanford) 3) Alvis Brazma (EBI) 4) Steve Chervitz (Affymetrix) 5) Junmin Liu (U. Penn) 6) Don Maier (SMD) 7) Tim Rayner (EBI) 8) Ugis Sarkans (EBI) 9) Paul Spellman (Lawrence Berkeley National Laboratory) 10) Joe White (Dana-Farber/Harvard Cancer Center) Agenda: 1) Tim =96 Final version of MAGE-TAB 2) Alvis/Ugis =96 MAGE-TAB object model: a) Possible move to FUGE-TAB b) Discussion of basic concepts (no formal proposal yet). 3) Don =96 The =93one-parser=94 requirement and how to satisfy it for = =20 MAGE-TAB: a) There is thought to be a strong requirement to have exactly =20 one MAGE-TAB parser in one body of code in one language that can be =20 used from either a Java or a Perl programming environment. =20 Satisfying this requirement would ensure that MAGE-TAB never destroys =20= itself on the shoals of disagreeing acceptors. b) Proposal for how to satisfy this requirement.= |
From: Tim R. <ra...@eb...> - 2007-03-19 09:30:11
|
Hi Gavin, I agree with Michael - this is the way I'd code what you've described in MAGE-TAB. The only addition I'd suggest is to add a Label column with the label names. A single Labeled Extract can have many Labels; this differs from tab2mage, where a separate LE is created for each label. You should end up with two rows per hybridization. I've checked, and the current parser implementation will support this arrangement (although I don't know if the accompanying data file parser copes with idat files as yet). As mentioned in my previous email, the feature extraction protocol currently has to be combined with the scanning protocol; we may want to consider how to fix that at the jamboree. Cheers, Tim -- On Sun, 18 Mar 2007, Miller, Michael D (Rosetta) wrote: > hi gavin, > > i'm not that familiar with the genotyping protocols, so this may be off > base, but i would think one could use the 'Labeled Extract Name' column > in the order where the labeling occurred. > > the tiffs i would imagine would be in the 'Image' column with perhaps a > Comment column to specify which image it was. > > if i understand right, the idat files are the Feature Extracted data > that's been collapsed per bead type from the raw data, so there could be > a 'Scan Name' column with two 'Array Data File' columns for the two idat > files, again with a Comment column for each to differentiate which was > which. > > tim discussed some issues around the feature extraction protocol in his > 3/16 e-mail which might be relevant. > > but i'm not sure what meaning the tiff files would have since the idat > files are per bead type, not per bead? > > cheers, > michael > >> -----Original Message----- >> From: mge...@li... >> [mailto:mge...@li...] On Behalf >> Of Gavin Sherlock >> Sent: Saturday, March 17, 2007 2:37 PM >> To: Tim Rayner >> Cc: mged-mage2; MGED-mage >> Subject: [Mged-mage2] MAGE-TAB Illumina question >> >> Hi Tim, >> >> Do you have any example MAGE-TAB files for Illumina Genotyping? >> Because there is no sample labeling per se, but instead 2 labels are >> incorporated during an allele specific extension step, after which >> you get 2 tiffs (one for each channel) and 2 idat files, I was a >> little unsure how exactly to deal with this in the sdrf file. Any >> pointers would be appreciated, >> >> Cheers, >> Gavin >> >> -------------------------------------------------------------- >> ----------- >> Take Surveys. Earn Cash. Influence the Future of IT >> Join SourceForge.net's Techsay panel and you'll get the >> chance to share your >> opinions on IT & business topics through brief surveys-and earn cash >> http://www.techsay.com/default.php?page=join.php&p=sourceforge >> &CID=DEVDEV >> _______________________________________________ >> Mged-MAGE2 mailing list >> Mge...@li... >> https://lists.sourceforge.net/lists/listinfo/mged-mage2 >> >> > > |
From: Miller, M. D (Rosetta) <Michael_Miller@Rosettabio.com> - 2007-03-18 18:32:23
|
hi gavin, i'm not that familiar with the genotyping protocols, so this may be off base, but i would think one could use the 'Labeled Extract Name' column in the order where the labeling occurred. the tiffs i would imagine would be in the 'Image' column with perhaps a Comment column to specify which image it was. if i understand right, the idat files are the Feature Extracted data that's been collapsed per bead type from the raw data, so there could be a 'Scan Name' column with two 'Array Data File' columns for the two idat files, again with a Comment column for each to differentiate which was which. tim discussed some issues around the feature extraction protocol in his 3/16 e-mail which might be relevant. but i'm not sure what meaning the tiff files would have since the idat files are per bead type, not per bead? cheers, michael=20 > -----Original Message----- > From: mge...@li...=20 > [mailto:mge...@li...] On Behalf=20 > Of Gavin Sherlock > Sent: Saturday, March 17, 2007 2:37 PM > To: Tim Rayner > Cc: mged-mage2; MGED-mage > Subject: [Mged-mage2] MAGE-TAB Illumina question >=20 > Hi Tim, >=20 > Do you have any example MAGE-TAB files for Illumina Genotyping? =20 > Because there is no sample labeling per se, but instead 2 labels are =20 > incorporated during an allele specific extension step, after which =20 > you get 2 tiffs (one for each channel) and 2 idat files, I was a =20 > little unsure how exactly to deal with this in the sdrf file. Any =20 > pointers would be appreciated, >=20 > Cheers, > Gavin >=20 > -------------------------------------------------------------- > ----------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the=20 > chance to share your > opinions on IT & business topics through brief surveys-and earn cash > http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge > &CID=3DDEVDEV > _______________________________________________ > Mged-MAGE2 mailing list > Mge...@li... > https://lists.sourceforge.net/lists/listinfo/mged-mage2 >=20 >=20 |
From: Gavin S. <she...@ge...> - 2007-03-17 21:37:05
|
Hi Tim, Do you have any example MAGE-TAB files for Illumina Genotyping? Because there is no sample labeling per se, but instead 2 labels are incorporated during an allele specific extension step, after which you get 2 tiffs (one for each channel) and 2 idat files, I was a little unsure how exactly to deal with this in the sdrf file. Any pointers would be appreciated, Cheers, Gavin |
From: Trish W. <wh...@pc...> - 2007-03-14 17:29:04
|
hi michael, apols for the delay. for background, what is the need to represent annotation properties in MAGE? > i've been thinking about how ontology annotation properties can be > handled in MAGE and wanted to run some ideas past you all. > > in looking at the w3c owl documentation i found > (http://www.w3.org/TR/owl-ref/#Annotations): > > "Five annotation pes are predefined by OWL, namely: > * owl:versionInfo > * rdfs:label > * rdfs:comment > * rdfs:seeAlso > * rdfs:isDefinedBy" > plus how addition annotation properties can be defined per ontology. so > for any arbitrary ontology, it seems there can be a set of annotation > properties distinct to that ontology and they can be on classes, and > object and datatype properties. so the question becomes how that can be > encoded in MAGE. yes, this is true, owl/rdf/rdfs specifies the above set (and more) as well as protege itself specifies a few (mainly administrative types), the Dublin Core Metadata Initiative (mainly adminsitrative) and also SKOS and a newly developed resource to add to the mix, RA/RU. oh, i forgot that the conversion of OBO to OWL also uses a self-contained, but not necessarily shared set of annotation properties. as for annotation properties as either object or datatype annotation properties this is true of OWL 1.0 (the format that MO exists in and what OBI is being developed in). For OWL 1.1, this distinction does not exist for annotation property 'types'. I'm not sure when OWL 1.1 will be released (I think that it is at least a few months away), but wanted to mention this since presumably it will have at least a small impact on coding. > to keep with the current best practice does this seem appropriate? haven't been following mage so closely lately, but will take a stab at this. > <OntologyEntry category = "classname" value = "classname"> for clarification, i would say class identifier. for OBI and those ontologies converted from OBO to OWL the class identifier will be something like OBI_1234, for MO it is the classname itself since that is the value of the rdf:ID field, therefore the class identifier. > <OntologyReference_assn> > <DatabaseEntry accession="@classname"> > <Database_assnref> > <Database_ref > identifier="www.ontology.com:Database:ontology name" name="ontology > name"/> > </Database_assnref> > </DatabaseEntry> > </OntologyReference_assn> > <Associations_assnlist> > <!-- annotation properties for a class --> > <OntologyEntry category = "annotation_properties" > value = "annotation_properties" > > <Associations_assnlist> > <OntologyEntry category = "label" value = "Class > Name" /> > <OntologyEntry category = "arbitrary property" > value = "arbitrary value" /> > </Associations_assnlist> > </OntologyEntry> > <OntologyEntry category = "datatypepropertyname" value > = "datatypepropertyname" > > <Associations_assnlist> > <OntologyEntry category = "datatypeproperty" value > = "value" /> > <!-- annotation properties for a datatype property > --> > <OntologyEntry category = "annotation_properties" > value = "annotation_properties" > > <Associations_assnlist> > <OntologyEntry category = "label" value = > "Datatype Property Name" /> > </Associations_assnlist> > </OntologyEntry> > </Associations_assnlist> > </OntologyEntry> > </Associations_assnlist> > </OntologyEntry> Are you separating out the owl annotation properties from object or datatype annotation properties, not sure I follow the comments? As for other questions, it prob would be easier to discuss via phone if that is ok. Trish |
From: Miller, M. D (Rosetta) <Michael_Miller@Rosettabio.com> - 2007-03-12 14:11:40
|
hi trish, chris and helen, i've been thinking about how ontology annotation properties can be handled in MAGE and wanted to run some ideas past you all. in looking at the w3c owl documentation i found (http://www.w3.org/TR/owl-ref/#Annotations): "Five annotation properties are predefined by OWL, namely: * owl:versionInfo=20 * rdfs:label=20 * rdfs:comment=20 * rdfs:seeAlso=20 * rdfs:isDefinedBy"=20 plus how addition annotation properties can be defined per ontology. so for any arbitrary ontology, it seems there can be a set of annotation properties distinct to that ontology and they can be on classes, and object and datatype properties. so the question becomes how that can be encoded in MAGE. to keep with the current best practice does this seem appropriate? <OntologyEntry category =3D "classname" value =3D = "classname"> <OntologyReference_assn> <DatabaseEntry accession=3D"@classname"> <Database_assnref> <Database_ref identifier=3D"www.ontology.com:Database:ontology name" name=3D"ontology name"/> </Database_assnref> </DatabaseEntry> </OntologyReference_assn> <Associations_assnlist> <!-- annotation properties for a class --> <OntologyEntry category =3D "annotation_properties" value =3D "annotation_properties" > <Associations_assnlist> <OntologyEntry category =3D "label" value =3D = "Class Name" /> <OntologyEntry category =3D "arbitrary property" value =3D "arbitrary value" /> </Associations_assnlist> </OntologyEntry> <OntologyEntry category =3D "datatypepropertyname" = value =3D "datatypepropertyname" > <Associations_assnlist> <OntologyEntry category =3D "datatypeproperty" = value =3D "value" /> <!-- annotation properties for a datatype property --> <OntologyEntry category =3D = "annotation_properties" value =3D "annotation_properties" > <Associations_assnlist> <OntologyEntry category =3D "label" value =3D "Datatype Property Name" /> </Associations_assnlist> </OntologyEntry> </Associations_assnlist> </OntologyEntry> </Associations_assnlist> </OntologyEntry> cheers, michael Michael Miller Lead Software Developer Rosetta Biosoftware Business Unit www.rosettabio.com |
From: Catherine B. <ba...@ge...> - 2007-03-07 15:35:06
|
Hi all, In preparation for the upcoming MAGE jamboree, please suggest agenda items (Don Maier will keep track of them for us). Also let him know if you are planning to attend. My current nose count is probably not correct, but it is: Chervitz, Spellman, Li, White, Maier, Beauheim, Miller, Rayner (maybe), Parkinson (maybe), Ball. Details: > Where: Clark Center, Room S362, Stanford University, > Stanford, CA, USA > When: 9am-5pm, March 27-30, 2007 > Who: Anyone willing to contribute their time to MGED/MAGE > projects. We need to keep attendance down to about 25. > How much: No registration required, there is some funding for > travel . If you would like funding for travel, contact Cathy Ball > (ba...@ge...) > Hotels: Can be expensive, so do your research. Stanford has a > comprehensive list of hotels (http://www.stanford.edu/dept/ > visitorinfo/resources/Lodging.html) and a guest house (http:// > www.stanford.edu/dept/hds/guesthouse//) > Transportation: Can be a pain in the neck, but on-campus parking is > expensive and scarce. See Stanford's suggestions (http:// > transportation.stanford.edu/) > |
From: Trish W. <wh...@pc...> - 2007-02-09 00:37:11
|
As most you you know, members of the MGED Ontology working group are working with other communities on a new project called the Ontology for Biomedical Investigations (obi.sf.net). This ontology is being developed to contain both common and domain-specific terms and is being developed following OBO Foundry best practices (obofoundry.org). One artifact of this design is that not all term names that a given community would prefer to see in an annotation application are the term names in the ontology. As one solution to this issue, the group has proposed an annotation property to store a user community term name. We are reviewing the implementation of this annotation property and it is currently slated to store synonyms, acronyms, abbreviations etc. and some of these may be further qualified by community/user group. I'm writing to get feedback from developers as to implementation requests of this information. Should the various term types be split out into their own annotation property, should some of these term types not be qualified by a user community,...? A listing of these annotation properties can be found at: https://www.cbil.upenn.edu/obiwiki/index.php/WorkshopMetadata Trish |
From: Jason S. <jas...@gm...> - 2007-02-06 19:29:07
|
Hey All, Here is an interesting project in light of MAGEstk - it's also nice that it uses Xerces... Cheers, jas. ---------- Forwarded message ---------- From: Boris Kolpackov <bo...@co...> Date: Jan 24, 2007 7:44 PM Subject: [ANN] CodeSynthesis XSD - open-source XML Schema to C++ compiler To: c-...@xe... Hi, I am pleased to announce the availability of CodeSynthesis XSD 2.3.1. CodeSynthesis XSD is an open-source, cross-platform W3C XML Schema to C++ data binding compiler. Provided with a schema, it generates C++ classes that represent the given vocabulary as well as parsing and serialization code. You can then access the data stored in XML using types and functions that semantically correspond to your application domain rather than dealing with elements, attributes, and text in a direct representation of XML such as DOM or SAX. XSD supports both in-memory and stream-oriented processing models by implementing two C++ mappings: C++/Tree and C++/Parser. The C++/Tree mapping represents the information stored in XML instance documents as a tree-like, in-memory data structure. The C++/Parser mapping generates parser templates for data types defined in XML Schema. Using these parser templates you can build your own in-memory representations or perform immediate processing of XML instance documents. XSD is built using Xerces-C++ and both of its C++ mappings use Xerces-C++ as the underlying XML parser (C++/Parser can also use Expat). XSD is available on AIX, GNU/Linux, HP-UX, Mac OS X, Solaris, and Windows. Supported C++ compilers include: GNU g++, HP aCC, IBM XL C++, Intel C++, Sun C++, and MS Visual C++. More information as well as precompiled binaries for all supported platforms are available from http://www.codesynthesis.com/products/xsd/ Best Regards, Boris Kolpackov |
From: Catherine B. <ba...@ge...> - 2007-02-05 21:08:49
|
Hi Tim and others, I'm getting progressively more worried. Our goal is not to make MAGE- TAB simple "if possible." It is to make it simple, period. While it might be nice to use MAGE-TAB in an automated fashion to schlep data between software applications, the absolutely most important user is a bench biologist, end of story. Please, please, please avoid the temptation to start tacking more and more complex data concepts and structures on what absolutely needs to be a very simple set of spreadsheets that an undergrad or tech can use to annotate a reasonably complex set of microarray data in less than an afternoon. Cathy On Feb 5, 2007, at 12:44 PM, Junmin Liu wrote: > Hi, Tim > I am particularly interested in your following comments. > >> I was under the impression that automated pipeline systems would >> be using >> MAGEv2-ML; I'm very concerned that we're mixing up our use cases >> here. >> MAGE-TAB is not supposed to replace MAGEv2-ML, but complement it. >> It's a >> simple tabular format that bench scientists can use (in theory), >> *not* a >> wholesale replacement for the object modelling approach. I worry >> greatly >> that this has been forgotten in the rush to get this specification >> finished. > > I agree with you that MAGE-Tab is not to replace MAGE-ML, but a > simple way. I think we should stride to make it "SIMPLE" if possible. > > In terms of data submission, actually in the past my lab was using > webforms to annotate the studies and then export MAGE-MLs from DB. > Now we > have MAGE-Tab, my plan is to make the MAGE-Tab as both data loading > tool > and data exporting tool. So the same MAGE-Tab and its associated > data can > be loaded into RAD and also loaded into AE, I think this is feasible. > > Also like "MVC", I think MAGE-Tab is like presentation view on the > underlining model, even we have MAGE-ML v2, or v3, v4, the mapping > from > tab to model are changed, but the MAGE-Tab may not be changed. I > could be > wrong on this. > > Thanks, > ---junmin > > > > > ---------------------------------------------------------------------- > --- > Using Tomcat but need to do more? Need to support web services, > security? > Get stuff done quickly with pre-integrated technology to make your > job easier. > Download IBM WebSphere Application Server v.1.0.1 based on Apache > Geronimo > http://sel.as-us.falkag.net/sel? > cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Mged-mage mailing list > Mge...@li... > https://lists.sourceforge.net/lists/listinfo/mged-mage |
From: Junmin L. <ju...@pc...> - 2007-02-05 20:45:11
|
Hi, Tim I am particularly interested in your following comments. > I was under the impression that automated pipeline systems would be using > MAGEv2-ML; I'm very concerned that we're mixing up our use cases here. > MAGE-TAB is not supposed to replace MAGEv2-ML, but complement it. It's a > simple tabular format that bench scientists can use (in theory), *not* a > wholesale replacement for the object modelling approach. I worry greatly > that this has been forgotten in the rush to get this specification > finished. I agree with you that MAGE-Tab is not to replace MAGE-ML, but a simple way. I think we should stride to make it "SIMPLE" if possible. In terms of data submission, actually in the past my lab was using webforms to annotate the studies and then export MAGE-MLs from DB. Now we have MAGE-Tab, my plan is to make the MAGE-Tab as both data loading tool and data exporting tool. So the same MAGE-Tab and its associated data can be loaded into RAD and also loaded into AE, I think this is feasible. Also like "MVC", I think MAGE-Tab is like presentation view on the underlining model, even we have MAGE-ML v2, or v3, v4, the mapping from tab to model are changed, but the MAGE-Tab may not be changed. I could be wrong on this. Thanks, ---junmin |
From: Miller, M. D (Rosetta) <Michael_Miller@Rosettabio.com> - 2007-02-05 16:18:47
|
hi tim, you've addressed my major concerns, and outside of any feedback on the " issue from others, i'm satisfied > I should perhaps also comment that I fully expect there to be=20 > a whole slew=20 > of things we've missed in 1.0 which will turn up during=20 > various peoples'=20 > implementations,=20 my main concern was about where the sources came from, so the Provider column makes me more than happy to wait until a 1.1 to expand on this and other issues. cheers, michael=20 > -----Original Message----- > From: Tim Rayner [mailto:ra...@eb...]=20 > Sent: Monday, February 05, 2007 6:18 AM > To: Miller, Michael D (Rosetta) > Cc: MGED-mage; mged-mage2 > Subject: RE: [Mged-mage2] [Mged-mage] Collated comments on=20 > MAGETAB spec >=20 >=20 > Hi Michael, >=20 > Thank you once again for your comments; answers inline below... >=20 > On Sat, 3 Feb 2007, Miller, Michael D (Rosetta) wrote: >=20 > > hi tim, > > > > thanks for your time. my initial comments came two weeks=20 > back and my > > main sticking point, where the Sources were obtained from,=20 > i thought was > > covered in the specification itself but when i was working=20 > out examples > > didn't find what i had seen before. i've taken another=20 > look and isn't > > this the "PROVIDER" column (section 3.3.6)? in that case=20 > i'm happy but > > would like to see the example SDRF file updated to include=20 > this column. > > to be MIAME compliant, it looks this column must be used=20 > (from the MIAME > > checklist, caps added by me): >=20 > Yes, that is indeed the intention. It doesn't provide quite the=20 > fine-grained control of identifiers and namespaces that might be=20 > desirable, of course. Note that I'm not adverse to adding=20 > (optional!) IDF=20 > tags "Investigation Namespace" and "Investigation Authority"=20 > or somesuch=20 > in 1.1, but I do think a wider discussion is needed. I've added a new=20 > Provider column to the example SDRF as requested. >=20 > I should perhaps also comment that I fully expect there to be=20 > a whole slew=20 > of things we've missed in 1.0 which will turn up during=20 > various peoples'=20 > implementations, and I've been envisioning a 1.1 specification sooner=20 > rather than later (maybe at the Stanford jamboree, or even=20 > sooner?). Now=20 > is definitely the time to be setting the agenda for such an update. >=20 > > "Samples used, extract preparation and labeling: > > > > The origin of each biological sample (e.g., name of the=20 > organism, the > > PROVIDER of the sample) and its characteristics (e.g., gender, age, > > developmental stage, strain, or disease state)." > > > >> As you wish, although I'm quite sure we can't enforce this > >> when dealing > >> with MAGE-TAB documents generated by end users. > > > > much appreciated. if the examples and the tools that=20 > generate templates > > use this extension, that will help the situation. in=20 > general, people > > are used to this, both Affymetrix and GenePix have standard=20 > extensions > > for their files. >=20 > Good point - I've also fixed that in the example files. >=20 > >> I was under the impression that automated pipeline systems > >> would be using > >> MAGEv2-ML; > > > > you don't give yourself and ArrayExpress enough credit=20 > here. if it is > > possible to get a paper published by submitting the data=20 > and annotation > > using MAGETAB, people are going to expect their GeneExpression > > applications to take these very same file formats. > > > >> I'd rather not have to reinvent the > >> definition of tab-delimited format; as you've seen, things > >> get complicated > >> quite quickly, to the point at which the name of the format > >> *deceptively* > >> implies simplicity. > > > > my point is that allowing quotes means that MAGETAB is no longer a > > simple tab-delimited file, it requires a state machine to parse it > > because " becomes a called out character along with the tab. >=20 > Yes, to an extent it depends on how "tab-delimited" is=20 > defined, and that=20 > seems to be at the crux of this discussion. If a spreadsheet app user=20 > exports "tab-delimited" text, for better or worse the default is to=20 > sprinkle it liberally with quotes around every text field. >=20 > > are tabs > > allowed between the quotes? i would think a user would=20 > expect that they > > are. >=20 > That would be the intent, yes. >=20 > > the fact that excel puts quotes around its fields is an artifice of > > spread-sheet, not tab-delimited format. > > > > but as i said i am perfectly willing to go with the consensus here. >=20 > I whole-heartedly agree (to both points). >=20 > >> Even so, the MAQC experiment (E-TABM-132 in AE) was quite > >> successfully > >> coded on a single Tab2MAGE spreadsheet. I'm not clear why > >> this is not an > >> option for other such data sets. > > > > the curious thing about this is that BioMaterial 2 and 26 for the > > primary human samples do have BioSourceProvider attributes provided. > > also, in principle the experiment isn't MIAME compliant=20 > because the rat > > samples do not have their PROVIDER listed. >=20 > I'll look into that, thanks. Most likely the information was=20 > simply not=20 > provided by the submitter. In the meantime the latest (and I=20 > truly hope,=20 > final this time) v1.0 spec is available from the same link as before. >=20 > Cheers, >=20 > Tim >=20 >=20 > > cheers, > > michael > > > >> -----Original Message----- > >> From: Tim Rayner [mailto:ra...@eb...] > >> Sent: Thursday, February 01, 2007 3:21 AM > >> To: Miller, Michael D (Rosetta) > >> Cc: MGED-mage; mged-mage2 > >> Subject: RE: [Mged-mage2] [Mged-mage] Collated comments on > >> MAGETAB spec > >> > >> > >> Hi Michael, > >> > >> Thanks for your comments. I have attempted to address them below: > >> > >>>> 2. Regarding adding a mandatory .idf extension for the IDF > >>>> file, I think > >>>> this is not necessary, since it's pretty easy to design a > >> submission > >>>> system which would track the IDF without an extension > >>> > >>> this is not true of automated pipeline systems, which is a > >> huge use case > >>> (as much or more gene expression data and annotation are=20 > loaded via > >>> these pipelines). in windows, anyway, it is very easy to > >> set this up to > >>> open automatically in excel or whatever spreadsheet program > >> is desired. > >> > >> I was under the impression that automated pipeline systems > >> would be using > >> MAGEv2-ML; I'm very concerned that we're mixing up our use > >> cases here. > >> MAGE-TAB is not supposed to replace MAGEv2-ML, but complement > >> it. It's a > >> simple tabular format that bench scientists can use (in > >> theory), *not* a > >> wholesale replacement for the object modelling approach. I > >> worry greatly > >> that this has been forgotten in the rush to get this specification > >> finished. > >> > >>> i would be happy with a recommendation that the file > >> extension be 'idf'. > >>> if this could be added to the 1st paragraph of section > >> 3.1.1, that would > >>> be great. > >> > >> As you wish, although I'm quite sure we can't enforce this > >> when dealing > >> with MAGE-TAB documents generated by end users. > >> > >>>> 3. Use of quotes in IDF and SDRF; while maybe they could be > >>>> dispensed with > >>>> in SDRF, they will be needed in IDF to allow e.g. newlines in > >>>> protocol > >>>> text (and believe me, users will want this!). > >>> > >>> as the addition of information on addition annotation files > >> in the idf > >>> was considered a late addition without time for comment, this is a > >>> rather late addition, it was never part of the=20 > specification proper. > >>> they can not be made optional, they either must be=20 > mandatory or not. > >> > >> I know of at least two well-developed parser modules for perl > >> which allow > >> quotes to be optional (Text::CSV_XS and Text::xSV). I have > >> been assuming > >> that similar parsers exist for Java and other languages. > >> Tab2MAGE uses > >> Text::CSV_XS and has had no problems with optional quoting. > >> > >>> i would like to see a handful of fields (like protocol=20 > descriptions) > >>> where they are required and the rest where they must not be > >> used but i > >>> can live with them being mandatory in the IDF and not used > >> in the SDRF. > >> > >> I think we need to use the same treatment for all the files; > >> not only will > >> that allow a uniform coding for parsers, but (critically)=20 > it will not > >> confuse the users as much. > >> > >>> there also must be a provision for escaping quotes within=20 > the quoted > >>> field, believe me, they will definitely occur (the escape=20 > can be as > >>> simple as '\"' and, less likely, but possible, it will be=20 > the first > >>> actual character of the field, which is why they must not > >> be optional > >>> for fields) > >> > >> I agree regarding the \" escape, and I've now added a note on that. > >> However, a decent parser should be able to distinguish the=20 > following: > >> > >> <tab>"your text"<tab> (" used to quote the field) > >> <tab>"your" text<tab> (" used within the content of the field) > >> > >> If the second quote in a pair is followed by a tab then that > >> quote pair is > >> quoting the field and is not part of the content. This is > >> pretty much a > >> solved problem on the perl side of things, although I'd definitely > >> appreciate feedback if there aren't suitable Java parsers > >> (this would, > >> however, surprise me greatly). I'd rather not have to reinvent the > >> definition of tab-delimited format; as you've seen, things > >> get complicated > >> quite quickly, to the point at which the name of the format > >> *deceptively* > >> implies simplicity. However, we're never going to be able to get > >> bench-based end users to comply with a mandatory requirement for or > >> against quoting here. > >> > >> Note that if it turns out there's a consensus against all of > >> this then I > >> can remove all the quotes from all the example files and the > >> discussion in > >> section 3.1.6. Any code I write will, however, handle quotes > >> this way, > >> because I know it works; the alternative does not. I added=20 > it to the > >> specification because I believe that we can't avoid this=20 > in practice. > >> > >>>> 4. We agreed that the authority:namespace component of the > >>>> resulting MAGE > >>>> identifiers would be left to the implementation of any > >>>> parser; while this > >>>> precludes sharing e.g. Sources defined in MAGE-TAB documents, > >>>> this is a > >>>> relatively minor use case. Again, this could be=20 > revisited for a 1.1 > >>>> specification. > >>> > >>> this is not a relatively minor use case for many who would > >> want to use > >>> MAGETAB--it may be for ArrayExpress right now but it is a=20 > common use > >>> case for investigation on how to organize a microarray experiment > >>> (MAQC!!!). the sharing of sources also occurs for a > >> variety of other > >>> reasons. > >> > >> I think I should have said "a relatively minor use case for > >> MAGE-TAB". > >> Again, what do bench biologists know of authorities and namespaces? > >> MAGE-TAB is not a replacement for MAGEv2-ML, which I'd have > >> thought would > >> be a far more natural tool for large coordinated studies such > >> as these. > >> Even so, the MAQC experiment (E-TABM-132 in AE) was quite > >> successfully > >> coded on a single Tab2MAGE spreadsheet. I'm not clear why > >> this is not an > >> option for other such data sets. > >> > >>> this, i thought would be a minor addition to the IDF file > >> to define the > >>> default naming authority. this is not a parser issue--a parser is > >>> developed in accordance to the specification. this would > >> be very bad to > >>> leave until 1.1, it will cause valuable biological=20 > information to be > >>> lost. > >> > >> We decided very early on that the scope of the identifiers in > >> the MAGE-TAB > >> document is limited to within that document. However, if there's a > >> consensus from everyone that this now needs to be changed, > >> then of course > >> we can change it. Given that the deadline for comments was 13 > >> days ago, > >> though, with all these suggestions made after that deadline, > >> I'm somewhat > >> disappointed that we're still having this discussion. > >> > >> Best regards, > >> > >> Tim > >> > >> > >> > >>> > >>> > >>>> -----Original Message----- > >>>> From: mge...@li... > >>>> [mailto:mge...@li...] On Behalf > >>>> Of Tim Rayner > >>>> Sent: Wednesday, January 31, 2007 5:46 AM > >>>> To: 'MGED-mage'; mged-mage2 > >>>> Subject: Re: [Mged-mage2] [Mged-mage] Collated comments on > >>>> MAGETAB spec > >>>> > >>>> > >>>> Hi, > >>>> > >>>> We've discussed the various comments on the previous=20 > version of the > >>>> MAGE-TAB spec (Jan 8), and it appears that the consensus from > >>>> ArrayExpress > >>>> is as follows: > >>>> > >>>> 1. Since we need to get a version 1.0 specification > >> finalised so that > >>>> implementation deadlines are met, we feel that full support > >>>> for external > >>>> annotation files should be deferred to version 1.1. This will > >>>> allow for > >>>> more complete discussion of the requirements, in particular > >>>> in light of > >>>> any considerations from the FuGE crowd. In the meantime a > >> minor note > >>>> has been added to section 3.1.1 regarding suggested use of a > >>>> Comment[] tag > >>>> to support these in the meantime. > >>>> > >>>> 2. Regarding adding a mandatory .idf extension for the IDF > >>>> file, I think > >>>> this is not necessary, since it's pretty easy to design a > >> submission > >>>> system which would track the IDF without an extension (e.g., > >>>> the current > >>>> Tab2MAGE submissions system already does this, in effect). > >>>> Additionally, a > >>>> new file extension would have to be mapped to Excel or > >>>> OpenOffice by the > >>>> end user for it to be any use to them (I believe this is > >> true of both > >>>> Windows and Mac). This may not be a huge deal, but it's > >>>> another barrier to > >>>> the casual user. Such mapping does not always guarantee that > >>>> a document > >>>> opens in the desired application either (OpenOffice, I'm=20 > looking at > >>>> you...). > >>>> > >>>> 3. Use of quotes in IDF and SDRF; while maybe they could be > >>>> dispensed with > >>>> in SDRF, they will be needed in IDF to allow e.g. newlines in > >>>> protocol > >>>> text (and believe me, users will want this!). The=20 > original Tab2MAGE > >>>> implementation didn't allow fields to be quoted like this > >> to preserve > >>>> special characters (i.e., newlines and tabs), and it was=20 > awful. An > >>>> additional advantage to using quotes is that "text" date > >>>> fields such as > >>>> "2007-01-31" will be preserved by spreadsheet software, while > >>>> 2007-01-31 > >>>> is often corrupted by such "helpful" applications. I have > >> added a new > >>>> section (3.1.6) which briefly discusses use of quotes to > >> escape data > >>>> fields. On the bright side, this format tends to be the default > >>>> for "tab-delimited" export from Excel and OpenOffice in any case. > >>>> > >>>> 4. We agreed that the authority:namespace component of the > >>>> resulting MAGE > >>>> identifiers would be left to the implementation of any > >>>> parser; while this > >>>> precludes sharing e.g. Sources defined in MAGE-TAB documents, > >>>> this is a > >>>> relatively minor use case. Again, this could be=20 > revisited for a 1.1 > >>>> specification. > >>>> > >>>> 5. Regarding Junmin's comment about array design accessions, > >>>> it is true > >>>> that submissions to us will be using ArrayExpress accessions > >>>> exclusively > >>>> (at least for the forseeable future) but this is not > >>>> necessarily true of > >>>> other users, e.g. those downloading and distributing MAGE-TAB > >>>> documents > >>>> from us. > >>>> > >>>> 6. As discussed, the specification does indeed allow SDRFs to > >>>> be split (on > >>>> any "Name" column) into as many sub-SDRF documents as necessary. > >>>> > >>>> I've made the other modifications suggested to the list and > >>>> put up a new > >>>> specification document (Jan 31) here: > >>>> > >>>> > >>=20 > http://www.ebi.ac.uk/systems-srv/mp/file-exchange/MAGE-TABv1.0.tar.gz > >>>> > >>>> Unless there are serious arguments to the contrary, I > >>>> personally will be > >>>> treating this as a finalised version 1.0 specification. > >>>> Thanks very much > >>>> for all your comments, > >>>> > >>>> Tim > >>>> > >>>> > >>>> -- > >>>> Tim Rayner, Ph.D. > >>>> Scientific Database Curator > >>>> Microarray Informatics Team > >>>> European Bioinformatics Institute > >>>> > >>>> > >>>> -------------------------------------------------------------- > >>>> ----------- > >>>> Take Surveys. Earn Cash. Influence the Future of IT > >>>> Join SourceForge.net's Techsay panel and you'll get the > >>>> chance to share your > >>>> opinions on IT & business topics through brief surveys - > >> and earn cash > >>>> = http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge > >>> &CID=3DDEVDEV > >>> _______________________________________________ > >>> Mged-MAGE2 mailing list > >>> Mge...@li... > >>> https://lists.sourceforge.net/lists/listinfo/mged-mage2 > >>> > >>> > >>> > >> > >> > > > > >=20 >=20 |
From: Tim R. <ra...@eb...> - 2007-02-05 14:18:57
|
Hi Michael, Thank you once again for your comments; answers inline below... On Sat, 3 Feb 2007, Miller, Michael D (Rosetta) wrote: > hi tim, > > thanks for your time. my initial comments came two weeks back and my > main sticking point, where the Sources were obtained from, i thought was > covered in the specification itself but when i was working out examples > didn't find what i had seen before. i've taken another look and isn't > this the "PROVIDER" column (section 3.3.6)? in that case i'm happy but > would like to see the example SDRF file updated to include this column. > to be MIAME compliant, it looks this column must be used (from the MIAME > checklist, caps added by me): Yes, that is indeed the intention. It doesn't provide quite the fine-grained control of identifiers and namespaces that might be desirable, of course. Note that I'm not adverse to adding (optional!) IDF tags "Investigation Namespace" and "Investigation Authority" or somesuch in 1.1, but I do think a wider discussion is needed. I've added a new Provider column to the example SDRF as requested. I should perhaps also comment that I fully expect there to be a whole slew of things we've missed in 1.0 which will turn up during various peoples' implementations, and I've been envisioning a 1.1 specification sooner rather than later (maybe at the Stanford jamboree, or even sooner?). Now is definitely the time to be setting the agenda for such an update. > "Samples used, extract preparation and labeling: > > The origin of each biological sample (e.g., name of the organism, the > PROVIDER of the sample) and its characteristics (e.g., gender, age, > developmental stage, strain, or disease state)." > >> As you wish, although I'm quite sure we can't enforce this >> when dealing >> with MAGE-TAB documents generated by end users. > > much appreciated. if the examples and the tools that generate templates > use this extension, that will help the situation. in general, people > are used to this, both Affymetrix and GenePix have standard extensions > for their files. Good point - I've also fixed that in the example files. >> I was under the impression that automated pipeline systems >> would be using >> MAGEv2-ML; > > you don't give yourself and ArrayExpress enough credit here. if it is > possible to get a paper published by submitting the data and annotation > using MAGETAB, people are going to expect their GeneExpression > applications to take these very same file formats. > >> I'd rather not have to reinvent the >> definition of tab-delimited format; as you've seen, things >> get complicated >> quite quickly, to the point at which the name of the format >> *deceptively* >> implies simplicity. > > my point is that allowing quotes means that MAGETAB is no longer a > simple tab-delimited file, it requires a state machine to parse it > because " becomes a called out character along with the tab. Yes, to an extent it depends on how "tab-delimited" is defined, and that seems to be at the crux of this discussion. If a spreadsheet app user exports "tab-delimited" text, for better or worse the default is to sprinkle it liberally with quotes around every text field. > are tabs > allowed between the quotes? i would think a user would expect that they > are. That would be the intent, yes. > the fact that excel puts quotes around its fields is an artifice of > spread-sheet, not tab-delimited format. > > but as i said i am perfectly willing to go with the consensus here. I whole-heartedly agree (to both points). >> Even so, the MAQC experiment (E-TABM-132 in AE) was quite >> successfully >> coded on a single Tab2MAGE spreadsheet. I'm not clear why >> this is not an >> option for other such data sets. > > the curious thing about this is that BioMaterial 2 and 26 for the > primary human samples do have BioSourceProvider attributes provided. > also, in principle the experiment isn't MIAME compliant because the rat > samples do not have their PROVIDER listed. I'll look into that, thanks. Most likely the information was simply not provided by the submitter. In the meantime the latest (and I truly hope, final this time) v1.0 spec is available from the same link as before. Cheers, Tim > cheers, > michael > >> -----Original Message----- >> From: Tim Rayner [mailto:ra...@eb...] >> Sent: Thursday, February 01, 2007 3:21 AM >> To: Miller, Michael D (Rosetta) >> Cc: MGED-mage; mged-mage2 >> Subject: RE: [Mged-mage2] [Mged-mage] Collated comments on >> MAGETAB spec >> >> >> Hi Michael, >> >> Thanks for your comments. I have attempted to address them below: >> >>>> 2. Regarding adding a mandatory .idf extension for the IDF >>>> file, I think >>>> this is not necessary, since it's pretty easy to design a >> submission >>>> system which would track the IDF without an extension >>> >>> this is not true of automated pipeline systems, which is a >> huge use case >>> (as much or more gene expression data and annotation are loaded via >>> these pipelines). in windows, anyway, it is very easy to >> set this up to >>> open automatically in excel or whatever spreadsheet program >> is desired. >> >> I was under the impression that automated pipeline systems >> would be using >> MAGEv2-ML; I'm very concerned that we're mixing up our use >> cases here. >> MAGE-TAB is not supposed to replace MAGEv2-ML, but complement >> it. It's a >> simple tabular format that bench scientists can use (in >> theory), *not* a >> wholesale replacement for the object modelling approach. I >> worry greatly >> that this has been forgotten in the rush to get this specification >> finished. >> >>> i would be happy with a recommendation that the file >> extension be 'idf'. >>> if this could be added to the 1st paragraph of section >> 3.1.1, that would >>> be great. >> >> As you wish, although I'm quite sure we can't enforce this >> when dealing >> with MAGE-TAB documents generated by end users. >> >>>> 3. Use of quotes in IDF and SDRF; while maybe they could be >>>> dispensed with >>>> in SDRF, they will be needed in IDF to allow e.g. newlines in >>>> protocol >>>> text (and believe me, users will want this!). >>> >>> as the addition of information on addition annotation files >> in the idf >>> was considered a late addition without time for comment, this is a >>> rather late addition, it was never part of the specification proper. >>> they can not be made optional, they either must be mandatory or not. >> >> I know of at least two well-developed parser modules for perl >> which allow >> quotes to be optional (Text::CSV_XS and Text::xSV). I have >> been assuming >> that similar parsers exist for Java and other languages. >> Tab2MAGE uses >> Text::CSV_XS and has had no problems with optional quoting. >> >>> i would like to see a handful of fields (like protocol descriptions) >>> where they are required and the rest where they must not be >> used but i >>> can live with them being mandatory in the IDF and not used >> in the SDRF. >> >> I think we need to use the same treatment for all the files; >> not only will >> that allow a uniform coding for parsers, but (critically) it will not >> confuse the users as much. >> >>> there also must be a provision for escaping quotes within the quoted >>> field, believe me, they will definitely occur (the escape can be as >>> simple as '\"' and, less likely, but possible, it will be the first >>> actual character of the field, which is why they must not >> be optional >>> for fields) >> >> I agree regarding the \" escape, and I've now added a note on that. >> However, a decent parser should be able to distinguish the following: >> >> <tab>"your text"<tab> (" used to quote the field) >> <tab>"your" text<tab> (" used within the content of the field) >> >> If the second quote in a pair is followed by a tab then that >> quote pair is >> quoting the field and is not part of the content. This is >> pretty much a >> solved problem on the perl side of things, although I'd definitely >> appreciate feedback if there aren't suitable Java parsers >> (this would, >> however, surprise me greatly). I'd rather not have to reinvent the >> definition of tab-delimited format; as you've seen, things >> get complicated >> quite quickly, to the point at which the name of the format >> *deceptively* >> implies simplicity. However, we're never going to be able to get >> bench-based end users to comply with a mandatory requirement for or >> against quoting here. >> >> Note that if it turns out there's a consensus against all of >> this then I >> can remove all the quotes from all the example files and the >> discussion in >> section 3.1.6. Any code I write will, however, handle quotes >> this way, >> because I know it works; the alternative does not. I added it to the >> specification because I believe that we can't avoid this in practice. >> >>>> 4. We agreed that the authority:namespace component of the >>>> resulting MAGE >>>> identifiers would be left to the implementation of any >>>> parser; while this >>>> precludes sharing e.g. Sources defined in MAGE-TAB documents, >>>> this is a >>>> relatively minor use case. Again, this could be revisited for a 1.1 >>>> specification. >>> >>> this is not a relatively minor use case for many who would >> want to use >>> MAGETAB--it may be for ArrayExpress right now but it is a common use >>> case for investigation on how to organize a microarray experiment >>> (MAQC!!!). the sharing of sources also occurs for a >> variety of other >>> reasons. >> >> I think I should have said "a relatively minor use case for >> MAGE-TAB". >> Again, what do bench biologists know of authorities and namespaces? >> MAGE-TAB is not a replacement for MAGEv2-ML, which I'd have >> thought would >> be a far more natural tool for large coordinated studies such >> as these. >> Even so, the MAQC experiment (E-TABM-132 in AE) was quite >> successfully >> coded on a single Tab2MAGE spreadsheet. I'm not clear why >> this is not an >> option for other such data sets. >> >>> this, i thought would be a minor addition to the IDF file >> to define the >>> default naming authority. this is not a parser issue--a parser is >>> developed in accordance to the specification. this would >> be very bad to >>> leave until 1.1, it will cause valuable biological information to be >>> lost. >> >> We decided very early on that the scope of the identifiers in >> the MAGE-TAB >> document is limited to within that document. However, if there's a >> consensus from everyone that this now needs to be changed, >> then of course >> we can change it. Given that the deadline for comments was 13 >> days ago, >> though, with all these suggestions made after that deadline, >> I'm somewhat >> disappointed that we're still having this discussion. >> >> Best regards, >> >> Tim >> >> >> >>> >>> >>>> -----Original Message----- >>>> From: mge...@li... >>>> [mailto:mge...@li...] On Behalf >>>> Of Tim Rayner >>>> Sent: Wednesday, January 31, 2007 5:46 AM >>>> To: 'MGED-mage'; mged-mage2 >>>> Subject: Re: [Mged-mage2] [Mged-mage] Collated comments on >>>> MAGETAB spec >>>> >>>> >>>> Hi, >>>> >>>> We've discussed the various comments on the previous version of the >>>> MAGE-TAB spec (Jan 8), and it appears that the consensus from >>>> ArrayExpress >>>> is as follows: >>>> >>>> 1. Since we need to get a version 1.0 specification >> finalised so that >>>> implementation deadlines are met, we feel that full support >>>> for external >>>> annotation files should be deferred to version 1.1. This will >>>> allow for >>>> more complete discussion of the requirements, in particular >>>> in light of >>>> any considerations from the FuGE crowd. In the meantime a >> minor note >>>> has been added to section 3.1.1 regarding suggested use of a >>>> Comment[] tag >>>> to support these in the meantime. >>>> >>>> 2. Regarding adding a mandatory .idf extension for the IDF >>>> file, I think >>>> this is not necessary, since it's pretty easy to design a >> submission >>>> system which would track the IDF without an extension (e.g., >>>> the current >>>> Tab2MAGE submissions system already does this, in effect). >>>> Additionally, a >>>> new file extension would have to be mapped to Excel or >>>> OpenOffice by the >>>> end user for it to be any use to them (I believe this is >> true of both >>>> Windows and Mac). This may not be a huge deal, but it's >>>> another barrier to >>>> the casual user. Such mapping does not always guarantee that >>>> a document >>>> opens in the desired application either (OpenOffice, I'm looking at >>>> you...). >>>> >>>> 3. Use of quotes in IDF and SDRF; while maybe they could be >>>> dispensed with >>>> in SDRF, they will be needed in IDF to allow e.g. newlines in >>>> protocol >>>> text (and believe me, users will want this!). The original Tab2MAGE >>>> implementation didn't allow fields to be quoted like this >> to preserve >>>> special characters (i.e., newlines and tabs), and it was awful. An >>>> additional advantage to using quotes is that "text" date >>>> fields such as >>>> "2007-01-31" will be preserved by spreadsheet software, while >>>> 2007-01-31 >>>> is often corrupted by such "helpful" applications. I have >> added a new >>>> section (3.1.6) which briefly discusses use of quotes to >> escape data >>>> fields. On the bright side, this format tends to be the default >>>> for "tab-delimited" export from Excel and OpenOffice in any case. >>>> >>>> 4. We agreed that the authority:namespace component of the >>>> resulting MAGE >>>> identifiers would be left to the implementation of any >>>> parser; while this >>>> precludes sharing e.g. Sources defined in MAGE-TAB documents, >>>> this is a >>>> relatively minor use case. Again, this could be revisited for a 1.1 >>>> specification. >>>> >>>> 5. Regarding Junmin's comment about array design accessions, >>>> it is true >>>> that submissions to us will be using ArrayExpress accessions >>>> exclusively >>>> (at least for the forseeable future) but this is not >>>> necessarily true of >>>> other users, e.g. those downloading and distributing MAGE-TAB >>>> documents >>>> from us. >>>> >>>> 6. As discussed, the specification does indeed allow SDRFs to >>>> be split (on >>>> any "Name" column) into as many sub-SDRF documents as necessary. >>>> >>>> I've made the other modifications suggested to the list and >>>> put up a new >>>> specification document (Jan 31) here: >>>> >>>> >> http://www.ebi.ac.uk/systems-srv/mp/file-exchange/MAGE-TABv1.0.tar.gz >>>> >>>> Unless there are serious arguments to the contrary, I >>>> personally will be >>>> treating this as a finalised version 1.0 specification. >>>> Thanks very much >>>> for all your comments, >>>> >>>> Tim >>>> >>>> >>>> -- >>>> Tim Rayner, Ph.D. >>>> Scientific Database Curator >>>> Microarray Informatics Team >>>> European Bioinformatics Institute >>>> >>>> >>>> -------------------------------------------------------------- >>>> ----------- >>>> Take Surveys. Earn Cash. Influence the Future of IT >>>> Join SourceForge.net's Techsay panel and you'll get the >>>> chance to share your >>>> opinions on IT & business topics through brief surveys - >> and earn cash >>>> http://www.techsay.com/default.php?page=join.php&p=sourceforge >>> &CID=DEVDEV >>> _______________________________________________ >>> Mged-MAGE2 mailing list >>> Mge...@li... >>> https://lists.sourceforge.net/lists/listinfo/mged-mage2 >>> >>> >>> >> >> > > |
From: Miller, M. D (Rosetta) <Michael_Miller@Rosettabio.com> - 2007-02-04 00:21:15
|
hi tim, thanks for your time. my initial comments came two weeks back and my main sticking point, where the Sources were obtained from, i thought was covered in the specification itself but when i was working out examples didn't find what i had seen before. i've taken another look and isn't this the "PROVIDER" column (section 3.3.6)? in that case i'm happy but would like to see the example SDRF file updated to include this column. to be MIAME compliant, it looks this column must be used (from the MIAME checklist, caps added by me): "Samples used, extract preparation and labeling:=20 The origin of each biological sample (e.g., name of the organism, the PROVIDER of the sample) and its characteristics (e.g., gender, age, developmental stage, strain, or disease state)." > As you wish, although I'm quite sure we can't enforce this=20 > when dealing=20 > with MAGE-TAB documents generated by end users. much appreciated. if the examples and the tools that generate templates use this extension, that will help the situation. in general, people are used to this, both Affymetrix and GenePix have standard extensions for their files. > I was under the impression that automated pipeline systems=20 > would be using=20 > MAGEv2-ML;=20 you don't give yourself and ArrayExpress enough credit here. if it is possible to get a paper published by submitting the data and annotation using MAGETAB, people are going to expect their GeneExpression applications to take these very same file formats. > I'd rather not have to reinvent the=20 > definition of tab-delimited format; as you've seen, things=20 > get complicated=20 > quite quickly, to the point at which the name of the format=20 > *deceptively*=20 > implies simplicity. =20 my point is that allowing quotes means that MAGETAB is no longer a simple tab-delimited file, it requires a state machine to parse it because " becomes a called out character along with the tab. are tabs allowed between the quotes? i would think a user would expect that they are. the fact that excel puts quotes around its fields is an artifice of spread-sheet, not tab-delimited format. but as i said i am perfectly willing to go with the consensus here. > Even so, the MAQC experiment (E-TABM-132 in AE) was quite=20 > successfully=20 > coded on a single Tab2MAGE spreadsheet. I'm not clear why=20 > this is not an=20 > option for other such data sets. the curious thing about this is that BioMaterial 2 and 26 for the primary human samples do have BioSourceProvider attributes provided. also, in principle the experiment isn't MIAME compliant because the rat samples do not have their PROVIDER listed. cheers, michael > -----Original Message----- > From: Tim Rayner [mailto:ra...@eb...]=20 > Sent: Thursday, February 01, 2007 3:21 AM > To: Miller, Michael D (Rosetta) > Cc: MGED-mage; mged-mage2 > Subject: RE: [Mged-mage2] [Mged-mage] Collated comments on=20 > MAGETAB spec >=20 >=20 > Hi Michael, >=20 > Thanks for your comments. I have attempted to address them below: >=20 > >> 2. Regarding adding a mandatory .idf extension for the IDF > >> file, I think > >> this is not necessary, since it's pretty easy to design a=20 > submission > >> system which would track the IDF without an extension > > > > this is not true of automated pipeline systems, which is a=20 > huge use case > > (as much or more gene expression data and annotation are loaded via > > these pipelines). in windows, anyway, it is very easy to=20 > set this up to > > open automatically in excel or whatever spreadsheet program=20 > is desired. >=20 > I was under the impression that automated pipeline systems=20 > would be using=20 > MAGEv2-ML; I'm very concerned that we're mixing up our use=20 > cases here.=20 > MAGE-TAB is not supposed to replace MAGEv2-ML, but complement=20 > it. It's a=20 > simple tabular format that bench scientists can use (in=20 > theory), *not* a=20 > wholesale replacement for the object modelling approach. I=20 > worry greatly=20 > that this has been forgotten in the rush to get this specification=20 > finished. >=20 > > i would be happy with a recommendation that the file=20 > extension be 'idf'. > > if this could be added to the 1st paragraph of section=20 > 3.1.1, that would > > be great. >=20 > As you wish, although I'm quite sure we can't enforce this=20 > when dealing=20 > with MAGE-TAB documents generated by end users. >=20 > >> 3. Use of quotes in IDF and SDRF; while maybe they could be > >> dispensed with > >> in SDRF, they will be needed in IDF to allow e.g. newlines in > >> protocol > >> text (and believe me, users will want this!). > > > > as the addition of information on addition annotation files=20 > in the idf > > was considered a late addition without time for comment, this is a > > rather late addition, it was never part of the specification proper. > > they can not be made optional, they either must be mandatory or not. >=20 > I know of at least two well-developed parser modules for perl=20 > which allow=20 > quotes to be optional (Text::CSV_XS and Text::xSV). I have=20 > been assuming=20 > that similar parsers exist for Java and other languages.=20 > Tab2MAGE uses=20 > Text::CSV_XS and has had no problems with optional quoting. >=20 > > i would like to see a handful of fields (like protocol descriptions) > > where they are required and the rest where they must not be=20 > used but i > > can live with them being mandatory in the IDF and not used=20 > in the SDRF. >=20 > I think we need to use the same treatment for all the files;=20 > not only will=20 > that allow a uniform coding for parsers, but (critically) it will not=20 > confuse the users as much. >=20 > > there also must be a provision for escaping quotes within the quoted > > field, believe me, they will definitely occur (the escape can be as > > simple as '\"' and, less likely, but possible, it will be the first > > actual character of the field, which is why they must not=20 > be optional > > for fields) >=20 > I agree regarding the \" escape, and I've now added a note on that.=20 > However, a decent parser should be able to distinguish the following: >=20 > <tab>"your text"<tab> (" used to quote the field) > <tab>"your" text<tab> (" used within the content of the field) >=20 > If the second quote in a pair is followed by a tab then that=20 > quote pair is=20 > quoting the field and is not part of the content. This is=20 > pretty much a=20 > solved problem on the perl side of things, although I'd definitely=20 > appreciate feedback if there aren't suitable Java parsers=20 > (this would,=20 > however, surprise me greatly). I'd rather not have to reinvent the=20 > definition of tab-delimited format; as you've seen, things=20 > get complicated=20 > quite quickly, to the point at which the name of the format=20 > *deceptively*=20 > implies simplicity. However, we're never going to be able to get=20 > bench-based end users to comply with a mandatory requirement for or=20 > against quoting here. >=20 > Note that if it turns out there's a consensus against all of=20 > this then I=20 > can remove all the quotes from all the example files and the=20 > discussion in=20 > section 3.1.6. Any code I write will, however, handle quotes=20 > this way,=20 > because I know it works; the alternative does not. I added it to the=20 > specification because I believe that we can't avoid this in practice. >=20 > >> 4. We agreed that the authority:namespace component of the > >> resulting MAGE > >> identifiers would be left to the implementation of any > >> parser; while this > >> precludes sharing e.g. Sources defined in MAGE-TAB documents, > >> this is a > >> relatively minor use case. Again, this could be revisited for a 1.1 > >> specification. > > > > this is not a relatively minor use case for many who would=20 > want to use > > MAGETAB--it may be for ArrayExpress right now but it is a common use > > case for investigation on how to organize a microarray experiment > > (MAQC!!!). the sharing of sources also occurs for a=20 > variety of other > > reasons. >=20 > I think I should have said "a relatively minor use case for=20 > MAGE-TAB".=20 > Again, what do bench biologists know of authorities and namespaces?=20 > MAGE-TAB is not a replacement for MAGEv2-ML, which I'd have=20 > thought would=20 > be a far more natural tool for large coordinated studies such=20 > as these.=20 > Even so, the MAQC experiment (E-TABM-132 in AE) was quite=20 > successfully=20 > coded on a single Tab2MAGE spreadsheet. I'm not clear why=20 > this is not an=20 > option for other such data sets. >=20 > > this, i thought would be a minor addition to the IDF file=20 > to define the > > default naming authority. this is not a parser issue--a parser is > > developed in accordance to the specification. this would=20 > be very bad to > > leave until 1.1, it will cause valuable biological information to be > > lost. >=20 > We decided very early on that the scope of the identifiers in=20 > the MAGE-TAB=20 > document is limited to within that document. However, if there's a=20 > consensus from everyone that this now needs to be changed,=20 > then of course=20 > we can change it. Given that the deadline for comments was 13=20 > days ago,=20 > though, with all these suggestions made after that deadline,=20 > I'm somewhat=20 > disappointed that we're still having this discussion. >=20 > Best regards, >=20 > Tim >=20 >=20 >=20 > > > > > >> -----Original Message----- > >> From: mge...@li... > >> [mailto:mge...@li...] On Behalf > >> Of Tim Rayner > >> Sent: Wednesday, January 31, 2007 5:46 AM > >> To: 'MGED-mage'; mged-mage2 > >> Subject: Re: [Mged-mage2] [Mged-mage] Collated comments on > >> MAGETAB spec > >> > >> > >> Hi, > >> > >> We've discussed the various comments on the previous version of the > >> MAGE-TAB spec (Jan 8), and it appears that the consensus from > >> ArrayExpress > >> is as follows: > >> > >> 1. Since we need to get a version 1.0 specification=20 > finalised so that > >> implementation deadlines are met, we feel that full support > >> for external > >> annotation files should be deferred to version 1.1. This will > >> allow for > >> more complete discussion of the requirements, in particular > >> in light of > >> any considerations from the FuGE crowd. In the meantime a=20 > minor note > >> has been added to section 3.1.1 regarding suggested use of a > >> Comment[] tag > >> to support these in the meantime. > >> > >> 2. Regarding adding a mandatory .idf extension for the IDF > >> file, I think > >> this is not necessary, since it's pretty easy to design a=20 > submission > >> system which would track the IDF without an extension (e.g., > >> the current > >> Tab2MAGE submissions system already does this, in effect). > >> Additionally, a > >> new file extension would have to be mapped to Excel or > >> OpenOffice by the > >> end user for it to be any use to them (I believe this is=20 > true of both > >> Windows and Mac). This may not be a huge deal, but it's > >> another barrier to > >> the casual user. Such mapping does not always guarantee that > >> a document > >> opens in the desired application either (OpenOffice, I'm looking at > >> you...). > >> > >> 3. Use of quotes in IDF and SDRF; while maybe they could be > >> dispensed with > >> in SDRF, they will be needed in IDF to allow e.g. newlines in > >> protocol > >> text (and believe me, users will want this!). The original Tab2MAGE > >> implementation didn't allow fields to be quoted like this=20 > to preserve > >> special characters (i.e., newlines and tabs), and it was awful. An > >> additional advantage to using quotes is that "text" date > >> fields such as > >> "2007-01-31" will be preserved by spreadsheet software, while > >> 2007-01-31 > >> is often corrupted by such "helpful" applications. I have=20 > added a new > >> section (3.1.6) which briefly discusses use of quotes to=20 > escape data > >> fields. On the bright side, this format tends to be the default > >> for "tab-delimited" export from Excel and OpenOffice in any case. > >> > >> 4. We agreed that the authority:namespace component of the > >> resulting MAGE > >> identifiers would be left to the implementation of any > >> parser; while this > >> precludes sharing e.g. Sources defined in MAGE-TAB documents, > >> this is a > >> relatively minor use case. Again, this could be revisited for a 1.1 > >> specification. > >> > >> 5. Regarding Junmin's comment about array design accessions, > >> it is true > >> that submissions to us will be using ArrayExpress accessions > >> exclusively > >> (at least for the forseeable future) but this is not > >> necessarily true of > >> other users, e.g. those downloading and distributing MAGE-TAB > >> documents > >> from us. > >> > >> 6. As discussed, the specification does indeed allow SDRFs to > >> be split (on > >> any "Name" column) into as many sub-SDRF documents as necessary. > >> > >> I've made the other modifications suggested to the list and > >> put up a new > >> specification document (Jan 31) here: > >> > >>=20 > http://www.ebi.ac.uk/systems-srv/mp/file-exchange/MAGE-TABv1.0.tar.gz > >> > >> Unless there are serious arguments to the contrary, I > >> personally will be > >> treating this as a finalised version 1.0 specification. > >> Thanks very much > >> for all your comments, > >> > >> Tim > >> > >> > >> -- > >> Tim Rayner, Ph.D. > >> Scientific Database Curator > >> Microarray Informatics Team > >> European Bioinformatics Institute > >> > >> > >> -------------------------------------------------------------- > >> ----------- > >> Take Surveys. Earn Cash. Influence the Future of IT > >> Join SourceForge.net's Techsay panel and you'll get the > >> chance to share your > >> opinions on IT & business topics through brief surveys -=20 > and earn cash > >> http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge > > &CID=3DDEVDEV > > _______________________________________________ > > Mged-MAGE2 mailing list > > Mge...@li... > > https://lists.sourceforge.net/lists/listinfo/mged-mage2 > > > > > > >=20 >=20 |