xmlpipedb-developer Mailing List for XMLPipeDB (Page 7)
Brought to you by:
kdahlquist,
zugzugglug
Menu
▾
▴
xmlpipedb-developer — Mailing list for XMLPipeDB software development questions and discussions.
You can subscribe to this list here.
| 2009 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
(16) |
Sep
|
Oct
(9) |
Nov
(3) |
Dec
(6) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2010 |
Jan
(2) |
Feb
(8) |
Mar
|
Apr
(22) |
May
(1) |
Jun
|
Jul
|
Aug
(3) |
Sep
(32) |
Oct
(2) |
Nov
|
Dec
|
| 2011 |
Jan
|
Feb
(60) |
Mar
(42) |
Apr
(35) |
May
(17) |
Jun
(2) |
Jul
(23) |
Aug
(72) |
Sep
(15) |
Oct
(10) |
Nov
(14) |
Dec
(4) |
| 2012 |
Jan
(6) |
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2013 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
(11) |
Dec
|
| 2014 |
Jan
(1) |
Feb
(12) |
Mar
(14) |
Apr
(8) |
May
|
Jun
(14) |
Jul
(2) |
Aug
|
Sep
(5) |
Oct
(6) |
Nov
|
Dec
|
| 2015 |
Jan
|
Feb
(5) |
Mar
(2) |
Apr
|
May
|
Jun
(2) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Richard B. <rbr...@gm...> - 2011-08-20 15:58:45
|
sound advice... I'll keep it in mind going forward. added logging to getSystemTableManager() query richard On Fri, Aug 19, 2011 at 10:10 PM, John David N. Dionisio <do...@lm...>wrote: > Greetings, > > Ah yes, I know this type of bug well. :) This is how I avoid/deal with > it: before running the application, I formulate the query separately (some > combination of copy/paste and manual typing) then issue it directly in the > relational database first. This way, I can assure myself that the query is > correct on its own, and can fix problems with that query in that context > before embedding it in the application code. > > That way, when the application runs, you can attribute problems to the code > that processes the query results, or, in the case of dynamically generated > queries like this one, to the code that forms the query string. You can be > fairly confident that the query itself is right, because you have run it > independently beforehand and verified that it returns the results that you > want. > > It is also helpful to log the final query string prior to executing it. > That way, you can see the exact query that was issued, and can copy/paste > it directly into pgAdmin in case it requires some troubleshooting or > debugging. > > Hope this helps (in case query problems persist)... > > John David N. Dionisio, PhD > Associate Professor, Computer Science > Loyola Marymount University > > > On Aug 19, 2011, at 8:47 PM, Richard Brous wrote: > > > Fatal error during export which was related to "as" which I suspect I > left out a right parens when building the ps string. > > > > Rerunning export now but must head to bed. > > > > Will commit the changes I have and look for email comments in the morn. > > > > Richard > > > > On Fri, Aug 19, 2011 at 8:04 PM, Richard Brous <rbr...@gm...> > wrote: > > OK, made changes to getSystemTableManager() based on your comments below. > > > > The ps string is programmatically created by adding "id = ?" for each > species profile within selectedSpeciesProfiles > > > > also added for loops below to cycle through the ps.set... > > > > and a for loop to grab each species customization. > > > > Running an export now which will be done in an hour or so to review prior > to submitting changes to sourceforge. > > > > Richard > > > > On Thu, Aug 18, 2011 at 11:19 PM, John David N. Dionisio <do...@lm...> > wrote: > > Greetings, > > > > OK, the transitional prep code (i.e., using the get(0) trick) looks good. > Looks like we can safely hunker down now, starting with > getSystemTableManager. > > > > I agree that the clause: > > > > WHERE dbreferencetype.type = 'NCBI Taxonomy' and id = ? > > > > ...where "?" becomes the single taxon ID to be exported should now > become: > > > > WHERE dbreferencetype.type = 'NCBI Taxonomy' and (id = ? or id = ? > or id = ? ......) > > > > ...where the "?"s each get a taxon ID. This requires that this single > large string must be built programmatically before being passed into > prepareStatement. Plus, the ps.set.... calls now change from a single > statement into a loop. > > > > On a side note, you can change ps.setString to ps.setLong, thus avoiding > the concatenation with the empty string. > > > > I traced where that speciesCustomizeTextField value leads and found that > it ultimately goes into the getSpeciesName property of the species profile, > which is in turn what is used in the Species column of the ID tables. It > appears that the purpose of speciesCustomizeTextField was to allow the user > to modify the species name; seeing as we hardly (if ever) made use of this > customization, I propose that we just auto-set this property in the species > profiles to the actual species name stored in the UniProt records. > > > > John David N. Dionisio, PhD > > Associate Professor, Computer Science > > Loyola Marymount University > > > > > > > > On Aug 18, 2011, at 8:17 PM, Richard Brous wrote: > > > > > Posted new working copy to the db wiki > > > Committed changes to SourceForge > > > > > > On Thu, Aug 18, 2011 at 7:41 PM, Richard Brous <rbr...@gm...> > wrote: > > > getRelationsTableManager() > > > Interim changes complete using selectedSpeciesProfiles.get(0) > > > we skipped this one as its pretty complex. Need to discuss how I need > to tackle this. > > > > > > getSystemTableManager() > > > Interim changes complete using selectedSpeciesProfiles.get(0) > > > After reviewing the code I think the way to update this method is to > loop through each selected species profile prior to if conditional at line > 525? > > > Seem sound? > > > > > > getRelationshipTableManager() > > > Interim changes complete using selectedSpeciesProfiles.get(0) > > > Not quite sure how to make method aware of multispecies. Use looping or > a single command vs actually changing the sql table commands? > > > > > > getSecondPassTableManager() > > > Interim change complete using selectedSpeciesProfiles.get(0) > > > I think we need to loop through each species profile here, to add the > customizations for each species > > > > > > > ------------------------------------------------------------------------------------------------------------------------------------- > > > > > > Attempted another Ms export and this time figured out what was going > on. > > > > > > As Dr. D noted, no IDs existed in the previous GDB attempt and this is > related to the speciesCustomizeTextField which we had commented out a few > weeks ago. I changed it to > speciesCustomizeTextField.setText(selectedSpecies.get(0).getName()); to > ensure the field was populated with something and that did the trick. Now > exports are nearly identical for Ms for pre-multispecies build and the > current working build. > > > > > > I'm going to place my current working multispecies build (which only > exports a single species still) up on the db class wiki for your review. > > > > > > Look for SourceForge changes soonish... > > > > > > Richard > > > > > > > > > > > > On Thu, Aug 18, 2011 at 11:40 AM, Kam Dahlquist <kda...@lm...> > wrote: > > > Hi, > > > > > > That's good news. Let me know when there's a build that I can run. > > > > > > Also, would you guys mind changing the subject line to more closely > > > reflect the task being described in the message? > > > > > > Thanks, > > > Kam > > > > > > At 10:29 AM 8/18/2011, you wrote: > > > >Greetings, > > > > > > > >FYI I've committed some changes to the GO import handling. The most > > > >recent "breakage" in GO importing was caused by GO's addition of an > > > ><ontology> element to the <header>. Interestingly, the text in this > > > >element was simply "go" --- an interesting change, implying that > > > >other ontologies may be accommodated. > > > > > > > >In any case, I've updated the GO database libraries and source code, > > > >and I did a successful test import on a daily OBO file downloaded > > > >yesterday. So, it looks like we can use the latest OBO files again, > > > >at least until another element gets added to the schema. > > > > > > > >John David N. Dionisio, PhD > > > >Associate Professor, Computer Science > > > >Loyola Marymount University > > > > > > > > > > ------------------------------------------------------------------------------ > > > Get a FREE DOWNLOAD! and learn more about uberSVN rich system, > > > user administration capabilities and model configuration. Take > > > the hassle out of deploying and managing Subversion and the > > > tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 > > > _______________________________________________ > > > xmlpipedb-developer mailing list > > > xml...@li... > > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > > > > > > > > > <ATT00001..txt><ATT00002..txt> > > > > > > > ------------------------------------------------------------------------------ > > Get a FREE DOWNLOAD! and learn more about uberSVN rich system, > > user administration capabilities and model configuration. Take > > the hassle out of deploying and managing Subversion and the > > tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 > > _______________________________________________ > > xmlpipedb-developer mailing list > > xml...@li... > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > > > > > > <ATT00001..txt><ATT00002..txt> > > > > ------------------------------------------------------------------------------ > Get a FREE DOWNLOAD! and learn more about uberSVN rich system, > user administration capabilities and model configuration. Take > the hassle out of deploying and managing Subversion and the > tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 > _______________________________________________ > xmlpipedb-developer mailing list > xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > |
|
From: John D. N. D. <do...@lm...> - 2011-08-20 07:02:48
|
Hi Rich, Having now taken a closer look at the code, I definitely recommended that you do what was suggested in the previous reply, which is to form and issue the SQL query directly before coding it up. More details have been committed in comments, so go ahead and take a look. You are also now getting further exposure to the JDBC (Java Database Connectivity) API. There is a section in the textbook that provides a brief introduction to JDBC, and there are extensive tutorials and references available on the web. If you haven't already done so, it might be worthwhile to glance at these. Looking at the PreparedStatement class in particular may help solidify your understanding of the code in question. Happy querying :) John David N. Dionisio, PhD Associate Professor, Computer Science Loyola Marymount University On Aug 19, 2011, at 8:47 PM, Richard Brous wrote: > Fatal error during export which was related to "as" which I suspect I left out a right parens when building the ps string. > > Rerunning export now but must head to bed. > > Will commit the changes I have and look for email comments in the morn. > > Richard > > On Fri, Aug 19, 2011 at 8:04 PM, Richard Brous <rbr...@gm...> wrote: > OK, made changes to getSystemTableManager() based on your comments below. > > The ps string is programmatically created by adding "id = ?" for each species profile within selectedSpeciesProfiles > > also added for loops below to cycle through the ps.set... > > and a for loop to grab each species customization. > > Running an export now which will be done in an hour or so to review prior to submitting changes to sourceforge. > > Richard > > On Thu, Aug 18, 2011 at 11:19 PM, John David N. Dionisio <do...@lm...> wrote: > Greetings, > > OK, the transitional prep code (i.e., using the get(0) trick) looks good. Looks like we can safely hunker down now, starting with getSystemTableManager. > > I agree that the clause: > > WHERE dbreferencetype.type = 'NCBI Taxonomy' and id = ? > > ...where "?" becomes the single taxon ID to be exported should now become: > > WHERE dbreferencetype.type = 'NCBI Taxonomy' and (id = ? or id = ? or id = ? ......) > > ...where the "?"s each get a taxon ID. This requires that this single large string must be built programmatically before being passed into prepareStatement. Plus, the ps.set.... calls now change from a single statement into a loop. > > On a side note, you can change ps.setString to ps.setLong, thus avoiding the concatenation with the empty string. > > I traced where that speciesCustomizeTextField value leads and found that it ultimately goes into the getSpeciesName property of the species profile, which is in turn what is used in the Species column of the ID tables. It appears that the purpose of speciesCustomizeTextField was to allow the user to modify the species name; seeing as we hardly (if ever) made use of this customization, I propose that we just auto-set this property in the species profiles to the actual species name stored in the UniProt records. > > John David N. Dionisio, PhD > Associate Professor, Computer Science > Loyola Marymount University > > > > On Aug 18, 2011, at 8:17 PM, Richard Brous wrote: > > > Posted new working copy to the db wiki > > Committed changes to SourceForge > > > > On Thu, Aug 18, 2011 at 7:41 PM, Richard Brous <rbr...@gm...> wrote: > > getRelationsTableManager() > > Interim changes complete using selectedSpeciesProfiles.get(0) > > we skipped this one as its pretty complex. Need to discuss how I need to tackle this. > > > > getSystemTableManager() > > Interim changes complete using selectedSpeciesProfiles.get(0) > > After reviewing the code I think the way to update this method is to loop through each selected species profile prior to if conditional at line 525? > > Seem sound? > > > > getRelationshipTableManager() > > Interim changes complete using selectedSpeciesProfiles.get(0) > > Not quite sure how to make method aware of multispecies. Use looping or a single command vs actually changing the sql table commands? > > > > getSecondPassTableManager() > > Interim change complete using selectedSpeciesProfiles.get(0) > > I think we need to loop through each species profile here, to add the customizations for each species > > > > ------------------------------------------------------------------------------------------------------------------------------------- > > > > Attempted another Ms export and this time figured out what was going on. > > > > As Dr. D noted, no IDs existed in the previous GDB attempt and this is related to the speciesCustomizeTextField which we had commented out a few weeks ago. I changed it to speciesCustomizeTextField.setText(selectedSpecies.get(0).getName()); to ensure the field was populated with something and that did the trick. Now exports are nearly identical for Ms for pre-multispecies build and the current working build. > > > > I'm going to place my current working multispecies build (which only exports a single species still) up on the db class wiki for your review. > > > > Look for SourceForge changes soonish... > > > > Richard > > > > > > > > On Thu, Aug 18, 2011 at 11:40 AM, Kam Dahlquist <kda...@lm...> wrote: > > Hi, > > > > That's good news. Let me know when there's a build that I can run. > > > > Also, would you guys mind changing the subject line to more closely > > reflect the task being described in the message? > > > > Thanks, > > Kam > > > > At 10:29 AM 8/18/2011, you wrote: > > >Greetings, > > > > > >FYI I've committed some changes to the GO import handling. The most > > >recent "breakage" in GO importing was caused by GO's addition of an > > ><ontology> element to the <header>. Interestingly, the text in this > > >element was simply "go" --- an interesting change, implying that > > >other ontologies may be accommodated. > > > > > >In any case, I've updated the GO database libraries and source code, > > >and I did a successful test import on a daily OBO file downloaded > > >yesterday. So, it looks like we can use the latest OBO files again, > > >at least until another element gets added to the schema. > > > > > >John David N. Dionisio, PhD > > >Associate Professor, Computer Science > > >Loyola Marymount University > > > > > > ------------------------------------------------------------------------------ > > Get a FREE DOWNLOAD! and learn more about uberSVN rich system, > > user administration capabilities and model configuration. Take > > the hassle out of deploying and managing Subversion and the > > tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 > > _______________________________________________ > > xmlpipedb-developer mailing list > > xml...@li... > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > > > > > > <ATT00001..txt><ATT00002..txt> > > > ------------------------------------------------------------------------------ > Get a FREE DOWNLOAD! and learn more about uberSVN rich system, > user administration capabilities and model configuration. Take > the hassle out of deploying and managing Subversion and the > tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 > _______________________________________________ > xmlpipedb-developer mailing list > xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > > > <ATT00001..txt><ATT00002..txt> |
|
From: John D. N. D. <do...@lm...> - 2011-08-20 05:10:24
|
Greetings, Ah yes, I know this type of bug well. :) This is how I avoid/deal with it: before running the application, I formulate the query separately (some combination of copy/paste and manual typing) then issue it directly in the relational database first. This way, I can assure myself that the query is correct on its own, and can fix problems with that query in that context before embedding it in the application code. That way, when the application runs, you can attribute problems to the code that processes the query results, or, in the case of dynamically generated queries like this one, to the code that forms the query string. You can be fairly confident that the query itself is right, because you have run it independently beforehand and verified that it returns the results that you want. It is also helpful to log the final query string prior to executing it. That way, you can see the exact query that was issued, and can copy/paste it directly into pgAdmin in case it requires some troubleshooting or debugging. Hope this helps (in case query problems persist)... John David N. Dionisio, PhD Associate Professor, Computer Science Loyola Marymount University On Aug 19, 2011, at 8:47 PM, Richard Brous wrote: > Fatal error during export which was related to "as" which I suspect I left out a right parens when building the ps string. > > Rerunning export now but must head to bed. > > Will commit the changes I have and look for email comments in the morn. > > Richard > > On Fri, Aug 19, 2011 at 8:04 PM, Richard Brous <rbr...@gm...> wrote: > OK, made changes to getSystemTableManager() based on your comments below. > > The ps string is programmatically created by adding "id = ?" for each species profile within selectedSpeciesProfiles > > also added for loops below to cycle through the ps.set... > > and a for loop to grab each species customization. > > Running an export now which will be done in an hour or so to review prior to submitting changes to sourceforge. > > Richard > > On Thu, Aug 18, 2011 at 11:19 PM, John David N. Dionisio <do...@lm...> wrote: > Greetings, > > OK, the transitional prep code (i.e., using the get(0) trick) looks good. Looks like we can safely hunker down now, starting with getSystemTableManager. > > I agree that the clause: > > WHERE dbreferencetype.type = 'NCBI Taxonomy' and id = ? > > ...where "?" becomes the single taxon ID to be exported should now become: > > WHERE dbreferencetype.type = 'NCBI Taxonomy' and (id = ? or id = ? or id = ? ......) > > ...where the "?"s each get a taxon ID. This requires that this single large string must be built programmatically before being passed into prepareStatement. Plus, the ps.set.... calls now change from a single statement into a loop. > > On a side note, you can change ps.setString to ps.setLong, thus avoiding the concatenation with the empty string. > > I traced where that speciesCustomizeTextField value leads and found that it ultimately goes into the getSpeciesName property of the species profile, which is in turn what is used in the Species column of the ID tables. It appears that the purpose of speciesCustomizeTextField was to allow the user to modify the species name; seeing as we hardly (if ever) made use of this customization, I propose that we just auto-set this property in the species profiles to the actual species name stored in the UniProt records. > > John David N. Dionisio, PhD > Associate Professor, Computer Science > Loyola Marymount University > > > > On Aug 18, 2011, at 8:17 PM, Richard Brous wrote: > > > Posted new working copy to the db wiki > > Committed changes to SourceForge > > > > On Thu, Aug 18, 2011 at 7:41 PM, Richard Brous <rbr...@gm...> wrote: > > getRelationsTableManager() > > Interim changes complete using selectedSpeciesProfiles.get(0) > > we skipped this one as its pretty complex. Need to discuss how I need to tackle this. > > > > getSystemTableManager() > > Interim changes complete using selectedSpeciesProfiles.get(0) > > After reviewing the code I think the way to update this method is to loop through each selected species profile prior to if conditional at line 525? > > Seem sound? > > > > getRelationshipTableManager() > > Interim changes complete using selectedSpeciesProfiles.get(0) > > Not quite sure how to make method aware of multispecies. Use looping or a single command vs actually changing the sql table commands? > > > > getSecondPassTableManager() > > Interim change complete using selectedSpeciesProfiles.get(0) > > I think we need to loop through each species profile here, to add the customizations for each species > > > > ------------------------------------------------------------------------------------------------------------------------------------- > > > > Attempted another Ms export and this time figured out what was going on. > > > > As Dr. D noted, no IDs existed in the previous GDB attempt and this is related to the speciesCustomizeTextField which we had commented out a few weeks ago. I changed it to speciesCustomizeTextField.setText(selectedSpecies.get(0).getName()); to ensure the field was populated with something and that did the trick. Now exports are nearly identical for Ms for pre-multispecies build and the current working build. > > > > I'm going to place my current working multispecies build (which only exports a single species still) up on the db class wiki for your review. > > > > Look for SourceForge changes soonish... > > > > Richard > > > > > > > > On Thu, Aug 18, 2011 at 11:40 AM, Kam Dahlquist <kda...@lm...> wrote: > > Hi, > > > > That's good news. Let me know when there's a build that I can run. > > > > Also, would you guys mind changing the subject line to more closely > > reflect the task being described in the message? > > > > Thanks, > > Kam > > > > At 10:29 AM 8/18/2011, you wrote: > > >Greetings, > > > > > >FYI I've committed some changes to the GO import handling. The most > > >recent "breakage" in GO importing was caused by GO's addition of an > > ><ontology> element to the <header>. Interestingly, the text in this > > >element was simply "go" --- an interesting change, implying that > > >other ontologies may be accommodated. > > > > > >In any case, I've updated the GO database libraries and source code, > > >and I did a successful test import on a daily OBO file downloaded > > >yesterday. So, it looks like we can use the latest OBO files again, > > >at least until another element gets added to the schema. > > > > > >John David N. Dionisio, PhD > > >Associate Professor, Computer Science > > >Loyola Marymount University > > > > > > ------------------------------------------------------------------------------ > > Get a FREE DOWNLOAD! and learn more about uberSVN rich system, > > user administration capabilities and model configuration. Take > > the hassle out of deploying and managing Subversion and the > > tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 > > _______________________________________________ > > xmlpipedb-developer mailing list > > xml...@li... > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > > > > > > <ATT00001..txt><ATT00002..txt> > > > ------------------------------------------------------------------------------ > Get a FREE DOWNLOAD! and learn more about uberSVN rich system, > user administration capabilities and model configuration. Take > the hassle out of deploying and managing Subversion and the > tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 > _______________________________________________ > xmlpipedb-developer mailing list > xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > > > <ATT00001..txt><ATT00002..txt> |
|
From: Richard B. <rbr...@gm...> - 2011-08-20 03:47:57
|
Fatal error during export which was related to "as" which I suspect I left out a right parens when building the ps string. Rerunning export now but must head to bed. Will commit the changes I have and look for email comments in the morn. Richard On Fri, Aug 19, 2011 at 8:04 PM, Richard Brous <rbr...@gm...> wrote: > OK, made changes to getSystemTableManager() based on your comments below. > > The ps string is programmatically created by adding "id = ?" for each > species profile within selectedSpeciesProfiles > > also added for loops below to cycle through the ps.set... > > and a for loop to grab each species customization. > > Running an export now which will be done in an hour or so to review prior > to submitting changes to sourceforge. > > Richard > > On Thu, Aug 18, 2011 at 11:19 PM, John David N. Dionisio <do...@lm...>wrote: > >> Greetings, >> >> OK, the transitional prep code (i.e., using the get(0) trick) looks good. >> Looks like we can safely hunker down now, starting with >> getSystemTableManager. >> >> I agree that the clause: >> >> WHERE dbreferencetype.type = 'NCBI Taxonomy' and id = ? >> >> ...where "?" becomes the single taxon ID to be exported should now become: >> >> WHERE dbreferencetype.type = 'NCBI Taxonomy' and (id = ? or id = ? >> or id = ? ......) >> >> ...where the "?"s each get a taxon ID. This requires that this single >> large string must be built programmatically before being passed into >> prepareStatement. Plus, the ps.set.... calls now change from a single >> statement into a loop. >> >> On a side note, you can change ps.setString to ps.setLong, thus avoiding >> the concatenation with the empty string. >> >> I traced where that speciesCustomizeTextField value leads and found that >> it ultimately goes into the getSpeciesName property of the species profile, >> which is in turn what is used in the Species column of the ID tables. It >> appears that the purpose of speciesCustomizeTextField was to allow the user >> to modify the species name; seeing as we hardly (if ever) made use of this >> customization, I propose that we just auto-set this property in the species >> profiles to the actual species name stored in the UniProt records. >> >> John David N. Dionisio, PhD >> Associate Professor, Computer Science >> Loyola Marymount University >> >> >> >> On Aug 18, 2011, at 8:17 PM, Richard Brous wrote: >> >> > Posted new working copy to the db wiki >> > Committed changes to SourceForge >> > >> > On Thu, Aug 18, 2011 at 7:41 PM, Richard Brous <rbr...@gm...> >> wrote: >> > getRelationsTableManager() >> > Interim changes complete using selectedSpeciesProfiles.get(0) >> > we skipped this one as its pretty complex. Need to discuss how I need to >> tackle this. >> > >> > getSystemTableManager() >> > Interim changes complete using selectedSpeciesProfiles.get(0) >> > After reviewing the code I think the way to update this method is to >> loop through each selected species profile prior to if conditional at line >> 525? >> > Seem sound? >> > >> > getRelationshipTableManager() >> > Interim changes complete using selectedSpeciesProfiles.get(0) >> > Not quite sure how to make method aware of multispecies. Use looping or >> a single command vs actually changing the sql table commands? >> > >> > getSecondPassTableManager() >> > Interim change complete using selectedSpeciesProfiles.get(0) >> > I think we need to loop through each species profile here, to add the >> customizations for each species >> > >> > >> ------------------------------------------------------------------------------------------------------------------------------------- >> > >> > Attempted another Ms export and this time figured out what was going on. >> > >> > As Dr. D noted, no IDs existed in the previous GDB attempt and this is >> related to the speciesCustomizeTextField which we had commented out a few >> weeks ago. I changed it to >> speciesCustomizeTextField.setText(selectedSpecies.get(0).getName()); to >> ensure the field was populated with something and that did the trick. Now >> exports are nearly identical for Ms for pre-multispecies build and the >> current working build. >> > >> > I'm going to place my current working multispecies build (which only >> exports a single species still) up on the db class wiki for your review. >> > >> > Look for SourceForge changes soonish... >> > >> > Richard >> > >> > >> > >> > On Thu, Aug 18, 2011 at 11:40 AM, Kam Dahlquist <kda...@lm...> >> wrote: >> > Hi, >> > >> > That's good news. Let me know when there's a build that I can run. >> > >> > Also, would you guys mind changing the subject line to more closely >> > reflect the task being described in the message? >> > >> > Thanks, >> > Kam >> > >> > At 10:29 AM 8/18/2011, you wrote: >> > >Greetings, >> > > >> > >FYI I've committed some changes to the GO import handling. The most >> > >recent "breakage" in GO importing was caused by GO's addition of an >> > ><ontology> element to the <header>. Interestingly, the text in this >> > >element was simply "go" --- an interesting change, implying that >> > >other ontologies may be accommodated. >> > > >> > >In any case, I've updated the GO database libraries and source code, >> > >and I did a successful test import on a daily OBO file downloaded >> > >yesterday. So, it looks like we can use the latest OBO files again, >> > >at least until another element gets added to the schema. >> > > >> > >John David N. Dionisio, PhD >> > >Associate Professor, Computer Science >> > >Loyola Marymount University >> > >> > >> > >> ------------------------------------------------------------------------------ >> > Get a FREE DOWNLOAD! and learn more about uberSVN rich system, >> > user administration capabilities and model configuration. Take >> > the hassle out of deploying and managing Subversion and the >> > tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 >> > _______________________________________________ >> > xmlpipedb-developer mailing list >> > xml...@li... >> > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer >> > >> > >> > <ATT00001..txt><ATT00002..txt> >> >> >> >> ------------------------------------------------------------------------------ >> Get a FREE DOWNLOAD! and learn more about uberSVN rich system, >> user administration capabilities and model configuration. Take >> the hassle out of deploying and managing Subversion and the >> tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 >> _______________________________________________ >> xmlpipedb-developer mailing list >> xml...@li... >> https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer >> > > |
|
From: Richard B. <rbr...@gm...> - 2011-08-20 03:05:04
|
OK, made changes to getSystemTableManager() based on your comments below. The ps string is programmatically created by adding "id = ?" for each species profile within selectedSpeciesProfiles also added for loops below to cycle through the ps.set... and a for loop to grab each species customization. Running an export now which will be done in an hour or so to review prior to submitting changes to sourceforge. Richard On Thu, Aug 18, 2011 at 11:19 PM, John David N. Dionisio <do...@lm...>wrote: > Greetings, > > OK, the transitional prep code (i.e., using the get(0) trick) looks good. > Looks like we can safely hunker down now, starting with > getSystemTableManager. > > I agree that the clause: > > WHERE dbreferencetype.type = 'NCBI Taxonomy' and id = ? > > ...where "?" becomes the single taxon ID to be exported should now become: > > WHERE dbreferencetype.type = 'NCBI Taxonomy' and (id = ? or id = ? > or id = ? ......) > > ...where the "?"s each get a taxon ID. This requires that this single > large string must be built programmatically before being passed into > prepareStatement. Plus, the ps.set.... calls now change from a single > statement into a loop. > > On a side note, you can change ps.setString to ps.setLong, thus avoiding > the concatenation with the empty string. > > I traced where that speciesCustomizeTextField value leads and found that it > ultimately goes into the getSpeciesName property of the species profile, > which is in turn what is used in the Species column of the ID tables. It > appears that the purpose of speciesCustomizeTextField was to allow the user > to modify the species name; seeing as we hardly (if ever) made use of this > customization, I propose that we just auto-set this property in the species > profiles to the actual species name stored in the UniProt records. > > John David N. Dionisio, PhD > Associate Professor, Computer Science > Loyola Marymount University > > > > On Aug 18, 2011, at 8:17 PM, Richard Brous wrote: > > > Posted new working copy to the db wiki > > Committed changes to SourceForge > > > > On Thu, Aug 18, 2011 at 7:41 PM, Richard Brous <rbr...@gm...> > wrote: > > getRelationsTableManager() > > Interim changes complete using selectedSpeciesProfiles.get(0) > > we skipped this one as its pretty complex. Need to discuss how I need to > tackle this. > > > > getSystemTableManager() > > Interim changes complete using selectedSpeciesProfiles.get(0) > > After reviewing the code I think the way to update this method is to loop > through each selected species profile prior to if conditional at line 525? > > Seem sound? > > > > getRelationshipTableManager() > > Interim changes complete using selectedSpeciesProfiles.get(0) > > Not quite sure how to make method aware of multispecies. Use looping or a > single command vs actually changing the sql table commands? > > > > getSecondPassTableManager() > > Interim change complete using selectedSpeciesProfiles.get(0) > > I think we need to loop through each species profile here, to add the > customizations for each species > > > > > ------------------------------------------------------------------------------------------------------------------------------------- > > > > Attempted another Ms export and this time figured out what was going on. > > > > As Dr. D noted, no IDs existed in the previous GDB attempt and this is > related to the speciesCustomizeTextField which we had commented out a few > weeks ago. I changed it to > speciesCustomizeTextField.setText(selectedSpecies.get(0).getName()); to > ensure the field was populated with something and that did the trick. Now > exports are nearly identical for Ms for pre-multispecies build and the > current working build. > > > > I'm going to place my current working multispecies build (which only > exports a single species still) up on the db class wiki for your review. > > > > Look for SourceForge changes soonish... > > > > Richard > > > > > > > > On Thu, Aug 18, 2011 at 11:40 AM, Kam Dahlquist <kda...@lm...> > wrote: > > Hi, > > > > That's good news. Let me know when there's a build that I can run. > > > > Also, would you guys mind changing the subject line to more closely > > reflect the task being described in the message? > > > > Thanks, > > Kam > > > > At 10:29 AM 8/18/2011, you wrote: > > >Greetings, > > > > > >FYI I've committed some changes to the GO import handling. The most > > >recent "breakage" in GO importing was caused by GO's addition of an > > ><ontology> element to the <header>. Interestingly, the text in this > > >element was simply "go" --- an interesting change, implying that > > >other ontologies may be accommodated. > > > > > >In any case, I've updated the GO database libraries and source code, > > >and I did a successful test import on a daily OBO file downloaded > > >yesterday. So, it looks like we can use the latest OBO files again, > > >at least until another element gets added to the schema. > > > > > >John David N. Dionisio, PhD > > >Associate Professor, Computer Science > > >Loyola Marymount University > > > > > > > ------------------------------------------------------------------------------ > > Get a FREE DOWNLOAD! and learn more about uberSVN rich system, > > user administration capabilities and model configuration. Take > > the hassle out of deploying and managing Subversion and the > > tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 > > _______________________________________________ > > xmlpipedb-developer mailing list > > xml...@li... > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > > > > > > <ATT00001..txt><ATT00002..txt> > > > > ------------------------------------------------------------------------------ > Get a FREE DOWNLOAD! and learn more about uberSVN rich system, > user administration capabilities and model configuration. Take > the hassle out of deploying and managing Subversion and the > tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 > _______________________________________________ > xmlpipedb-developer mailing list > xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > |
|
From: John D. N. D. <do...@lm...> - 2011-08-19 06:19:46
|
Greetings, OK, the transitional prep code (i.e., using the get(0) trick) looks good. Looks like we can safely hunker down now, starting with getSystemTableManager. I agree that the clause: WHERE dbreferencetype.type = 'NCBI Taxonomy' and id = ? ...where "?" becomes the single taxon ID to be exported should now become: WHERE dbreferencetype.type = 'NCBI Taxonomy' and (id = ? or id = ? or id = ? ......) ...where the "?"s each get a taxon ID. This requires that this single large string must be built programmatically before being passed into prepareStatement. Plus, the ps.set.... calls now change from a single statement into a loop. On a side note, you can change ps.setString to ps.setLong, thus avoiding the concatenation with the empty string. I traced where that speciesCustomizeTextField value leads and found that it ultimately goes into the getSpeciesName property of the species profile, which is in turn what is used in the Species column of the ID tables. It appears that the purpose of speciesCustomizeTextField was to allow the user to modify the species name; seeing as we hardly (if ever) made use of this customization, I propose that we just auto-set this property in the species profiles to the actual species name stored in the UniProt records. John David N. Dionisio, PhD Associate Professor, Computer Science Loyola Marymount University On Aug 18, 2011, at 8:17 PM, Richard Brous wrote: > Posted new working copy to the db wiki > Committed changes to SourceForge > > On Thu, Aug 18, 2011 at 7:41 PM, Richard Brous <rbr...@gm...> wrote: > getRelationsTableManager() > Interim changes complete using selectedSpeciesProfiles.get(0) > we skipped this one as its pretty complex. Need to discuss how I need to tackle this. > > getSystemTableManager() > Interim changes complete using selectedSpeciesProfiles.get(0) > After reviewing the code I think the way to update this method is to loop through each selected species profile prior to if conditional at line 525? > Seem sound? > > getRelationshipTableManager() > Interim changes complete using selectedSpeciesProfiles.get(0) > Not quite sure how to make method aware of multispecies. Use looping or a single command vs actually changing the sql table commands? > > getSecondPassTableManager() > Interim change complete using selectedSpeciesProfiles.get(0) > I think we need to loop through each species profile here, to add the customizations for each species > > ------------------------------------------------------------------------------------------------------------------------------------- > > Attempted another Ms export and this time figured out what was going on. > > As Dr. D noted, no IDs existed in the previous GDB attempt and this is related to the speciesCustomizeTextField which we had commented out a few weeks ago. I changed it to speciesCustomizeTextField.setText(selectedSpecies.get(0).getName()); to ensure the field was populated with something and that did the trick. Now exports are nearly identical for Ms for pre-multispecies build and the current working build. > > I'm going to place my current working multispecies build (which only exports a single species still) up on the db class wiki for your review. > > Look for SourceForge changes soonish... > > Richard > > > > On Thu, Aug 18, 2011 at 11:40 AM, Kam Dahlquist <kda...@lm...> wrote: > Hi, > > That's good news. Let me know when there's a build that I can run. > > Also, would you guys mind changing the subject line to more closely > reflect the task being described in the message? > > Thanks, > Kam > > At 10:29 AM 8/18/2011, you wrote: > >Greetings, > > > >FYI I've committed some changes to the GO import handling. The most > >recent "breakage" in GO importing was caused by GO's addition of an > ><ontology> element to the <header>. Interestingly, the text in this > >element was simply "go" --- an interesting change, implying that > >other ontologies may be accommodated. > > > >In any case, I've updated the GO database libraries and source code, > >and I did a successful test import on a daily OBO file downloaded > >yesterday. So, it looks like we can use the latest OBO files again, > >at least until another element gets added to the schema. > > > >John David N. Dionisio, PhD > >Associate Professor, Computer Science > >Loyola Marymount University > > > ------------------------------------------------------------------------------ > Get a FREE DOWNLOAD! and learn more about uberSVN rich system, > user administration capabilities and model configuration. Take > the hassle out of deploying and managing Subversion and the > tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 > _______________________________________________ > xmlpipedb-developer mailing list > xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > > > <ATT00001..txt><ATT00002..txt> |
|
From: Richard B. <rbr...@gm...> - 2011-08-19 03:17:46
|
Posted new working copy to the db wiki Committed changes to SourceForge On Thu, Aug 18, 2011 at 7:41 PM, Richard Brous <rbr...@gm...> wrote: > *getRelationsTableManager()* > Interim changes complete using selectedSpeciesProfiles.get(0) > we skipped this one as its pretty complex. Need to discuss how I need to > tackle this. > ** > *getSystemTableManager()* > Interim changes complete using selectedSpeciesProfiles.get(0) > After reviewing the code I think the way to update this method is to loop > through each selected species profile prior to if conditional at line 525? > Seem sound? > > *getRelationshipTableManager()* > Interim changes complete using selectedSpeciesProfiles.get(0) > Not quite sure how to make method aware of multispecies. Use looping or a > single command vs actually changing the sql table commands? > > *getSecondPassTableManager()* > Interim change complete using selectedSpeciesProfiles.get(0) > I think we need to loop through each species profile here, to add the > customizations for each species > > > ------------------------------------------------------------------------------------------------------------------------------------- > > Attempted another Ms export and this time figured out what was going on. > > As Dr. D noted, no IDs existed in the previous GDB attempt and this is > related to the speciesCustomizeTextField which we had commented out a few > weeks ago. I changed it to speciesCustomizeTextField.setText(selectedSpecies.get(0).getName()); > to ensure the field was populated with something and that did the trick. > Now exports are nearly identical for Ms for pre-multispecies build and the > current working build. > > I'm going to place my current working multispecies build (which only > exports a single species still) up on the db class wiki for your review. > > Look for SourceForge changes soonish... > > Richard > > > > On Thu, Aug 18, 2011 at 11:40 AM, Kam Dahlquist <kda...@lm...>wrote: > >> Hi, >> >> That's good news. Let me know when there's a build that I can run. >> >> Also, would you guys mind changing the subject line to more closely >> reflect the task being described in the message? >> >> Thanks, >> Kam >> >> At 10:29 AM 8/18/2011, you wrote: >> >Greetings, >> > >> >FYI I've committed some changes to the GO import handling. The most >> >recent "breakage" in GO importing was caused by GO's addition of an >> ><ontology> element to the <header>. Interestingly, the text in this >> >element was simply "go" --- an interesting change, implying that >> >other ontologies may be accommodated. >> > >> >In any case, I've updated the GO database libraries and source code, >> >and I did a successful test import on a daily OBO file downloaded >> >yesterday. So, it looks like we can use the latest OBO files again, >> >at least until another element gets added to the schema. >> > >> >John David N. Dionisio, PhD >> >Associate Professor, Computer Science >> >Loyola Marymount University >> >> >> >> ------------------------------------------------------------------------------ >> Get a FREE DOWNLOAD! and learn more about uberSVN rich system, >> user administration capabilities and model configuration. Take >> the hassle out of deploying and managing Subversion and the >> tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 >> _______________________________________________ >> xmlpipedb-developer mailing list >> xml...@li... >> https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer >> > > |
|
From: Richard B. <rbr...@gm...> - 2011-08-19 02:41:44
|
*getRelationsTableManager()* Interim changes complete using selectedSpeciesProfiles.get(0) we skipped this one as its pretty complex. Need to discuss how I need to tackle this. ** *getSystemTableManager()* Interim changes complete using selectedSpeciesProfiles.get(0) After reviewing the code I think the way to update this method is to loop through each selected species profile prior to if conditional at line 525? Seem sound? *getRelationshipTableManager()* Interim changes complete using selectedSpeciesProfiles.get(0) Not quite sure how to make method aware of multispecies. Use looping or a single command vs actually changing the sql table commands? *getSecondPassTableManager()* Interim change complete using selectedSpeciesProfiles.get(0) I think we need to loop through each species profile here, to add the customizations for each species ------------------------------------------------------------------------------------------------------------------------------------- Attempted another Ms export and this time figured out what was going on. As Dr. D noted, no IDs existed in the previous GDB attempt and this is related to the speciesCustomizeTextField which we had commented out a few weeks ago. I changed it to speciesCustomizeTextField.setText(selectedSpecies.get(0).getName()); to ensure the field was populated with something and that did the trick. Now exports are nearly identical for Ms for pre-multispecies build and the current working build. I'm going to place my current working multispecies build (which only exports a single species still) up on the db class wiki for your review. Look for SourceForge changes soonish... Richard On Thu, Aug 18, 2011 at 11:40 AM, Kam Dahlquist <kda...@lm...> wrote: > Hi, > > That's good news. Let me know when there's a build that I can run. > > Also, would you guys mind changing the subject line to more closely > reflect the task being described in the message? > > Thanks, > Kam > > At 10:29 AM 8/18/2011, you wrote: > >Greetings, > > > >FYI I've committed some changes to the GO import handling. The most > >recent "breakage" in GO importing was caused by GO's addition of an > ><ontology> element to the <header>. Interestingly, the text in this > >element was simply "go" --- an interesting change, implying that > >other ontologies may be accommodated. > > > >In any case, I've updated the GO database libraries and source code, > >and I did a successful test import on a daily OBO file downloaded > >yesterday. So, it looks like we can use the latest OBO files again, > >at least until another element gets added to the schema. > > > >John David N. Dionisio, PhD > >Associate Professor, Computer Science > >Loyola Marymount University > > > > ------------------------------------------------------------------------------ > Get a FREE DOWNLOAD! and learn more about uberSVN rich system, > user administration capabilities and model configuration. Take > the hassle out of deploying and managing Subversion and the > tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 > _______________________________________________ > xmlpipedb-developer mailing list > xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > |
|
From: Kam D. <kda...@lm...> - 2011-08-18 18:40:22
|
Hi, That's good news. Let me know when there's a build that I can run. Also, would you guys mind changing the subject line to more closely reflect the task being described in the message? Thanks, Kam At 10:29 AM 8/18/2011, you wrote: >Greetings, > >FYI I've committed some changes to the GO import handling. The most >recent "breakage" in GO importing was caused by GO's addition of an ><ontology> element to the <header>. Interestingly, the text in this >element was simply "go" --- an interesting change, implying that >other ontologies may be accommodated. > >In any case, I've updated the GO database libraries and source code, >and I did a successful test import on a daily OBO file downloaded >yesterday. So, it looks like we can use the latest OBO files again, >at least until another element gets added to the schema. > >John David N. Dionisio, PhD >Associate Professor, Computer Science >Loyola Marymount University |
|
From: John D. N. D. <do...@lm...> - 2011-08-18 17:29:32
|
Greetings,
FYI I've committed some changes to the GO import handling. The most recent "breakage" in GO importing was caused by GO's addition of an <ontology> element to the <header>. Interestingly, the text in this element was simply "go" --- an interesting change, implying that other ontologies may be accommodated.
In any case, I've updated the GO database libraries and source code, and I did a successful test import on a daily OBO file downloaded yesterday. So, it looks like we can use the latest OBO files again, at least until another element gets added to the schema.
John David N. Dionisio, PhD
Associate Professor, Computer Science
Loyola Marymount University
On Aug 16, 2011, at 9:12 PM, Richard Brous wrote:
> Submitted more changes to SourceForge: changes to ExportToGenMAPP, DatabaseProfile and UniProtDatabaseProfile
>
> interim change to getRelationsManager() to see first species in selectedSpeciesProfiles<SpeciesProfile>
> changed getSystemsTableManager() to iterate through each SpeciesProfile in selectedSpeciesProfiles<SpeciesProfile>
> interim change to getPrimarySystemTableManager to see first species in selectedSpeciesProfiles<SpeciesProfile>
>
> Compiles and runs. Will do a test export of Msmeg now and let it run while I sleep.
>
> Goodnight from the east coast!
>
> Richard
>
>
> On Tue, Aug 16, 2011 at 3:41 PM, John David N. Dionisio <do...@lm...> wrote:
> Yes, sounds right. Each species profile would then, in turn, call "submit" on the given table manager for each ID that should be exported. Presumably there is a taxon ID included with each record, so that when all is said and done you'll have the IDs of each chosen species, and you can tell which ID belongs to which species.
>
> If this goes well, your export should now have the Info table with the concatenated species names, with the system tables (UniProt, GeneID, OrderedLocus, etc.) having the IDs of the multiple species. The relationship tables should have just the IDs of the first species in the collection, per our gradual refactoring strategy. Presumably you can guess how the relationship tables should look once you have tackled that part of the code :)
>
> The QA check here, I would say, is to then match the ID counts *per species* against the equivalent counts in PostgreSQL (total count will not work now, since the user need not necessarily have chosen all of the species loaded in the PostgreSQL database). TallyEngine tweaks may be necessary after this; but we will not worry about that now. In the meantime, manual counts (i.e., issuing the queries manually) should do.
>
> Formulating those SQL queries will require an understanding of grouping and aggregate functions --- a fine exercise that is relevant to the database content of the course, I would say :)
>
> John David N. Dionisio, PhD
> Associate Professor, Computer Science
> Loyola Marymount University
>
>
>
> On Aug 16, 2011, at 3:32 PM, Richard Brous wrote:
>
> > OK, so its pretty much the last line of getSystemsTableManager() which needs to be modified to be aware of the collection of SpeciesProfile:
> >
> > tableManager =
> >
> > speciesProfile.getSystemsTableManagerCustomizations(tableManager, this);
> >
> > return tableManager;
> >
> > So I believe I need to iterate through each species of the collection before returning the final tableManager.
> >
> > I would enclose it in a for loop ( SpeciesProfile speciesProfile : speciesProfiles ) {
> >
> > }
> >
> > This would seem to be the correct implementation to ensure that each species customization would be applied.
> >
> >
> > Sound good??
> >
> > Richard
> >
> >
> >
> > On Tue, Aug 16, 2011 at 12:38 PM, Richard Brous <rbr...@gm...> wrote:
> > gotcha, heading to system tables.
> >
> > gotcha.
> >
> > gotcha.
> >
> > Thanks for the speedy reply =D
> >
> > Richard
> >
> > On Tue, Aug 16, 2011 at 12:21 PM, John David N. Dionisio <do...@lm...> wrote:
> > Hi Rich,
> >
> > Yeah, relations is pretty complicated. I suggest you skip that and go to just the system tables for now (i.e., the tables that hold the IDs themselves). The relationship tables hold pairs of IDs (i.e., which one from one system corresponds to the ID from another) and so is an additional level of complexity, I think.
> >
> > Regarding your prior question about the Info table --- I'm not or'ing the species names. I'm merely concatenating the names with pipes ("|") in between, as Kam specified. While the final value may look like an "or," it is ultimately just a string. The "|" could just have easily been a comma, semicolon, or other separator.
> >
> > Your prior guess as to how that would look, with the multiple { "Species", speciesName } pairs, would actually be equivalent to multiple values for the single Species column, which does not fit the relational model. The Species column can have only one value, and in this case it is a single string that is the concatenation of all selected species names, separated by "|" characters.
> >
> > Hope that clears things up...
> >
> > John David N. Dionisio, PhD
> > Associate Professor, Computer Science
> > Loyola Marymount University
> >
> >
> >
> > On Aug 16, 2011, at 12:11 PM, Richard Brous wrote:
> >
> > > OK, have moved onto getRelationsTableManager() ... this one seems pretty complicated...
> > >
> > > Have reviewed the method and submethods called and have the following questions:
> > >
> > > 1. What are the RelationshipTables that are stored in relationshipTables?
> > > aren't they species dependent? Can't seem to find where they were created?
> > > If not, then:
> > >
> > > 2. How should the if/else conditional be handled?
> > >
> > > if
> > >
> > > (speciesProfile.getSpeciesSpecificSystemTables().containsKey(stp.systemTable1) |
> > >
> > > speciesProfile.getSpeciesSpecificSystemTables().containsKey(stp.systemTable2)) {
> > > tableManager =
> > >
> > > speciesProfile.getRelationsTableManagerCustomizations(stp.systemTable1,
> > > stp.
> > >
> > > systemTable2, templateDefinedSystemToSystemCode, tableManager);
> > > This obviously needs to be made aware of Lists of SpeciesProfiles... to build the correct tableManager, do I need this conditional to run through every species or should we enclose all the code (if and else) within a loop through each species?
> > >
> > > Thanks.
> > >
> > > Richard
> > >
> > >
> > > On Tue, Aug 16, 2011 at 11:02 AM, Richard Brous <rbr...@gm...> wrote:
> > > Sorry I typed and sent that out too quickly before getting my thoughts completely together...
> > >
> > > First off i thought that the array[][] portion of the submit needed to be in the following format: { "Species", speciesName1 }, { "Species", speciesName2 }, { "Species", speciesName3 },...
> > > but I see you specified it as: { "Species", speciesName1 | speciesName2 | speciesName3 }, ... OR'ing each of the species names?
> > >
> > > I see what you did regarding the date object... yes no reason to recreate a second object...
> > >
> > > As mentioned previously I will continue through the next methods, submitting (well thought out) changes and emailing out updates...
> > >
> > > Richard
> > > On Tue, Aug 16, 2011 at 10:36 AM, Richard Brous <rbr...@gm...> wrote:
> > > Took a look at your code and realized what I had done wrong. I should have just broken out the loop as the second argument of the 3 in the submit method. That was a rookie move on my part.
> > >
> > > I am moving on to the next method and will keep my mind on the syntactic solution that I'm not thinking through prior to moving forward with an implementation.
> > >
> > > Richard
> > >
> > > On Mon, Aug 15, 2011 at 1:30 AM, John David N. Dionisio <do...@lm...> wrote:
> > > OK, everything is committed. I got up to the tweaks on the second panel in the wizard (Save As/GO Aspects), as well as the query change for exporting any combination of C, F, or P.
> > >
> > > I still have to do the GO OBO format check, plus the UI work on the remaining two wizard panels. But meanwhile, hope these latest changes work out well.
> > >
> > > John David N. Dionisio, PhD
> > > Associate Professor, Computer Science
> > > Loyola Marymount University
> > >
> > >
> > >
> > > On Aug 14, 2011, at 7:31 PM, Richard Brous wrote:
> > >
> > > > OK, using option 2 I have made changes to DatabaseProfile.java to allow for all species names to be included in the submit argument.
> > > > But I'm stuck on how to change my StringBuilder object to a type that submit wants. Help please!!
> > > >
> > > > Submitted the above and some comment changes to SourceForge this evening.
> > > >
> > > > Richard
> > > >
> > > > On Fri, Aug 12, 2011 at 12:08 PM, Kam Dahlquist <kda...@lm...> wrote:
> > > > Hi,
> > > >
> > > > I think we had better leave the info table with only one record
> > > > (option 2). The species names can be separated by pipes " | " as
> > > > they are in the Systems table where there are multiple species. To
> > > > my knowledge, the only time GenMAPP needs to access the info table is
> > > > for the "DisplayOrder" field, I don't know what would happen if there
> > > > were multiple records there. I know that the spec for the table says
> > > > that it should only be one record, but I don't know if it would crash
> > > > if there were multiple records. To be on the safe side, I think we
> > > > should just keep it to the one record.
> > > >
> > > > Best,
> > > > Kam
> > > >
> > > > At 09:45 PM 8/10/2011, John David N. Dionisio wrote:
> > > > >Greetings,
> > > > >
> > > > >I think we have to turn to Dr. Dahlquist's GenMAPP knowledge here to
> > > > >get the definitive answer. I see two choices:
> > > > >
> > > > >- The Info table should have one record for each species that the
> > > > >.gdb holds, in which case the change you need is to wrap that single
> > > > >submit call inside a loop, so that submit is called once for each
> > > > >chosen species.
> > > > >
> > > > >- The Info table should always have one record, and if the .gdb
> > > > >holds multiple species, the "Species" column should be some
> > > > >concatenation of multiple species names. In this case, you would
> > > > >still call submit only once, but the value you send into the
> > > > >"Species" column is some accumulation of all chosen species names.
> > > > >
> > > > >Admittedly I don't know which way is right (I assumed the former as
> > > > >of our Tuesday meeting, but on further examination I'm no longer
> > > > >quite so sure).
> > > > >
> > > > >For Kam --- what does GenMAPP expect to see in the Info table if the
> > > > >opened .gdb contains multiple species?
> > > > >
> > > > >John David N. Dionisio, PhD
> > > > >Associate Professor, Computer Science
> > > > >Loyola Marymount University
> > > > >
> > > > >
> > > > >On Aug 10, 2011, at 9:36 PM, Richard Brous wrote:
> > > > >
> > > > > > OK, continued to review ExportToGenMAPP and dug into the creation
> > > > > of the first TableManager tmA on line 118.
> > > > > >
> > > > > > In reading through the method, my understanding is that it
> > > > > creates a new TableManager based on the selectedDatabaseProfile
> > > > > (which is UniProt).
> > > > > >
> > > > > > This is performed by the method getInfoTableManager() which then
> > > > > calls method submit(String tableName, QueryType queryType,
> > > > > String[][] columnNamesToValues);
> > > > > >
> > > > > > the code is as follows:
> > > > > >
> > > > > > tableManager.submit("Info", QueryType.insert, new String[][] { {
> > > > > "Owner", owner }, { "Version", new
> > > > > SimpleDateFormat("yyyyMMdd").format(version) }, { "MODSystem",
> > > > > modSystem }, { "Species", speciesProfile.getSpeciesName() }, {
> > > > > "Modify", new SimpleDateFormat("yyyyMMdd").format(modify) }, {
> > > > > "DisplayOrder", displayOrder }, { "Notes", notes } });
> > > > > >
> > > > > >
> > > > > > The modification of this line centers on { "Species",
> > > > > speciesProfile.getSpeciesName() }, since it originally processed a
> > > > > single species.
> > > > > >
> > > > > > So now I need to populate the arguments with the species
> > > > > contained within selectedDatabaseprofile.selectedSpeciesProfiles.
> > > > > >
> > > > > > I think I'll start with the baseArgument up to MODSystem, then
> > > > > append as many species as necessary, and then cap off the end with
> > > > > the rest starting at Modify. (similar to your approach in
> > > > > ExportGoData, populateUniprotGoTableFromSQL(char chosenAspect,
> > > > > List<Integer> taxonIds) line 513
> > > > > >
> > > > > > Please let me know if this approach or analysis is off track.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > Richard
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, Aug 10, 2011 at 5:50 PM, Richard Brous <rbr...@gm...> wrote:
> > > > > > Updated repository to include all Gene Ontology changes discussed
> > > > > during our meeting yesterday.
> > > > > >
> > > > > > Digging into TableManager next.
> > > > > >
> > > > > > Richard
> > > > > >
> > > > > > On Fri, Aug 5, 2011 at 10:06 AM, Richard Brous <rbr...@gm...> wrote:
> > > > > > whew... thanks for the detailed reply. I will digest this a bit
> > > > > and get back to you with further questions.
> > > > > >
> > > > > > rb
> > > > > >
> > > > > > On Thu, Aug 4, 2011 at 11:18 PM, John David N. Dionisio
> > > > > <do...@lm...> wrote:
> > > > > > Greetings,
> > > > > >
> > > > > > Sorry for the delay. I wasn't able to walk through the relevant
> > > > > code until this evening.
> > > > > >
> > > > > > As Kam said, GOA serves as the link between the UniProt and GO
> > > > > IDs. It essentially determines which GO IDs get exported by using
> > > > > GOA to see which GO IDs are associated with an exported UniProt
> > > > > ID. The populateUniprotGoTableFromSQL, in its current form,
> > > > > extracts the GO association records that match the given taxon ID
> > > > > then exports, as UniProt-GO pairs, the GO and UniProt IDs
> > > > > referenced within that GO association record. Processing that
> > > > > follows this is then based on the GO IDs that got exported --- and
> > > > > that's how the current code avoids exporting the entire list of GO terms.
> > > > > >
> > > > > > The operative query is on the second line of populateUniprotGoTableFromSQL:
> > > > > >
> > > > > > String uniProtAndGOIDSQL = "select db_object_id, go_id,
> > > > > evidence_code, with_or_from from goa where db like '%UniProt%' and
> > > > > taxon = 'taxon:" + taxon + "'";
> > > > > >
> > > > > > In plain English, this selects the GOA records whose database is
> > > > > UniProt and whose taxon ID is the given taxon. An additional
> > > > > condition is added for the "aspect" (All, Component, Function, or
> > > > > Process) that is to be exported. This is another reduction filter,
> > > > > to further shrink the number of exported GO terms and thus avoid
> > > > > MAPPFinder issues later on.
> > > > > >
> > > > > > Given this, the proper expansion here is to change the taxon
> > > > > predicate to a multiple predicate. That is, this method can be
> > > > > changed to now accept a collection or array of taxon IDs, and the
> > > > > base query should then be changed so that it accepts any taxon from
> > > > > that collection. More or less, you want:
> > > > > >
> > > > > > private void populateUniprotGoTableFromSQL(char chosenAspect,
> > > > > int[] taxons) throws SQLException {
> > > > > >
> > > > > > ...then, instead of the single string, you want to iterate
> > > > > through the taxon IDs:
> > > > > >
> > > > > > StringBuilder baseQueryBuilder = new StringBuilder("select
> > > > > db_object_id, go_id, evidence_code, with_or_from from goa where db
> > > > > like '%UniProt%'");
> > > > > > boolean first = true;
> > > > > > for (int taxon: taxons) {
> > > > > > baseQueryBuilder.append(first ? " and (" : " or ");
> > > > > > baseQueryBuilder
> > > > > > .append("taxon = 'taxon:")
> > > > > > .append(taxon).append("'");
> > > > > > first = false;
> > > > > > }
> > > > > > baseQueryBuilder.append(")");
> > > > > >
> > > > > > ...and so on. I just sort of rattled this off so there may be
> > > > > little glitches, but anyway this is just to give you an overall idea.
> > > > > >
> > > > > > Put another way, no, you do not need to iterate this method for
> > > > > each taxon ID. Instead, you can still call this method once, with
> > > > > the multiplicity of taxon IDs emerging in terms of the actual
> > > > > condition used for selecting the GO terms to be exported (based on
> > > > > the available GOA records, which as you may recall are loaded from .goa files).
> > > > > >
> > > > > > As a side note, right here you have an opportunity for a little
> > > > > sanity check regarding the content of the relational database: GO
> > > > > terms will only be exported if GOA records for the desired taxon
> > > > > IDs have been imported into the database. So, as a pre-flight
> > > > > check, one can see if there are any GOA records at all for each
> > > > > chosen taxon ID. If there are none, then the .goa file for that
> > > > > species needs to be imported into the relational database.
> > > > > >
> > > > > > Hope this helps...
> > > > > >
> > > > > > John David N. Dionisio, PhD
> > > > > > Associate Professor, Computer Science
> > > > > > Loyola Marymount University
> > > > > >
> > > > > >
> > > > > > On Aug 4, 2011, at 1:00 PM, Kam Dahlquist wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > Dondi will have to chime in on this, but I think this is where
> > > > > things are going to get tricky.
> > > > > > >
> > > > > > > The final gdb does not actually contain the entire GO, it gets
> > > > > trimmed somehow based on the GO associations for a particular
> > > > > species. This is because MAPPFinder cannot handle loading the
> > > > > entire GO. Since there is some type of species-specific trimming
> > > > > going on, it's quite possible that this will need to iterate.
> > > > > > >
> > > > > > > However, I don't have the foggiest idea of how this works, so
> > > > > Dondi will have to chime in.
> > > > > > >
> > > > > > > Best,
> > > > > > > Kam
> > > > > > >
> > > > > > > At 12:09 AM 8/4/2011, you wrote:
> > > > > > >> Wednesday 8/3/11 progress:
> > > > > > >>
> > > > > > >> 1. After following the ExportPanel1.java ground zero code of:
> > > > > databaseProfile.setSelectedSpeciesProfile( selectedProfile );
> > > > > > >>
> > > > > > >> I found the method in DatabaseProfile.java plus a getter method;
> > > > > > >> SpeciesProfile setSelectedSpeciesProfile( speciesProfile ) and
> > > > > SpeciesProfile getSelectedSpeciesProfile( speciesProfile )
> > > > > > >>
> > > > > > >> I created two new methods that each handle List<Object> of
> > > > > SpeciesProfiles argument instead of a single SpeciesProfile;
> > > > > setSelectedSpeciesProfiles and getSelectedSpeciesProfiles.
> > > > > > >>
> > > > > > >> This enabled the ExportPanel1 ground zero code to become:
> > > > > databaseProfile.setSelectedSpeciesProfiles(selectedSpecies);
> > > > > > >>
> > > > > > >> 2. public static void export() on line 104 in ExportToGenMAPP.java
> > > > > > >>
> > > > > > >> On line 107 ExportGoData is instantiated which I found in
> > > > > ExportGoData.java and calls a method: public void export(char
> > > > > chosenAspect, int taxon).
> > > > > > >>
> > > > > > >> Within export, taxon id is required for another method:
> > > > > private void populateGoTables(char chosenAspect, int taxon).
> > > > > > >>
> > > > > > >> Within populateGoTables, taxon id is required for another
> > > > > method: private void populateUniprotGoTableFromSQL( char
> > > > > chosenAspect, int taxon).
> > > > > > >>
> > > > > > >> But, if the export to GDB process starts off with exporting GO
> > > > > data, doesn't it only need to do that once no matter how many
> > > > > species are selected? As you probably realize, I'm leading towards
> > > > > not having to iterate through this for each taxon id if possible.
> > > > > > >>
> > > > > > >> Also, how does the export actually work? How are GO ids and
> > > > > UniProt ids related within the table?
> > > > > > >>
> > > > > > >> Thanks!
> > > > > > >>
> > > > > > >> Richard
> > > > > > >>
> > > > > > >>
> > > > > > > <ATT00001..txt><ATT00002..txt>
> > > > > >
> > > > > >
> > > > > >
> > > > > ------------------------------------------------------------------------------
> > > > > > BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
> > > > > > The must-attend event for mobile developers. Connect with experts.
> > > > > > Get tools for creating Super Apps. See the latest technologies.
> > > > > > Sessions, hands-on labs, demos & much more. Register early & save!
> > > > > > http://p.sf.net/sfu/rim-blackberry-1
> > > > > > _______________________________________________
> > > > > > xmlpipedb-developer mailing list
> > > > > > xml...@li...
> > > > > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> > > > > >
> > > > > >
> > > > > >
> > > > > > <ATT00001..txt><ATT00002..txt>
> > > > >
> > > > >
> > > > >------------------------------------------------------------------------------
> > > > >Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
> > > > >user administration capabilities and model configuration. Take
> > > > >the hassle out of deploying and managing Subversion and the
> > > > >tools developers use with it.
> > > > >http://p.sf.net/sfu/wandisco-dev2dev
> > > > >_______________________________________________
> > > > >xmlpipedb-developer mailing list
> > > > >xml...@li...
> > > > >https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> > > >
> > > >
> > > > ------------------------------------------------------------------------------
> > > > FREE DOWNLOAD - uberSVN with Social Coding for Subversion.
> > > > Subversion made easy with a complete admin console. Easy
> > > > to use, easy to manage, easy to install, easy to extend.
> > > > Get a Free download of the new open ALM Subversion platform now.
> > > > http://p.sf.net/sfu/wandisco-dev2dev
> > > > _______________________________________________
> > > > xmlpipedb-developer mailing list
> > > > xml...@li...
> > > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> > > >
> > > > <ATT00001..txt><ATT00002..txt>
> > >
> > >
> > > ------------------------------------------------------------------------------
> > > uberSVN's rich system and user administration capabilities and model
> > > configuration take the hassle out of deploying and managing Subversion and
> > > the tools developers use with it. Learn more about uberSVN and get a free
> > > download at: http://p.sf.net/sfu/wandisco-dev2dev
> > > _______________________________________________
> > > xmlpipedb-developer mailing list
> > > xml...@li...
> > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> > >
> > >
> > >
> > > <ATT00001..txt><ATT00002..txt>
> >
> >
> > ------------------------------------------------------------------------------
> > Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
> > user administration capabilities and model configuration. Take
> > the hassle out of deploying and managing Subversion and the
> > tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2
> > _______________________________________________
> > xmlpipedb-developer mailing list
> > xml...@li...
> > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> >
> >
> > <ATT00001..txt><ATT00002..txt>
>
>
> ------------------------------------------------------------------------------
> Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
> user administration capabilities and model configuration. Take
> the hassle out of deploying and managing Subversion and the
> tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2
> _______________________________________________
> xmlpipedb-developer mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>
> <ATT00001..txt><ATT00002..txt>
|
|
From: John D. N. D. <do...@lm...> - 2011-08-17 08:55:56
|
Hi Rich,
OK, cool, the changes look decent (except for the comment that the "|"-concatenated Species string is a non-short-circuited OR --- as clarified previously, OR is not involved here at all). Looking forward to seeing how your export turns out. Good night!
John David N. Dionisio, PhD
Associate Professor, Computer Science
Loyola Marymount University
On Aug 16, 2011, at 9:12 PM, Richard Brous wrote:
> Submitted more changes to SourceForge: changes to ExportToGenMAPP, DatabaseProfile and UniProtDatabaseProfile
>
> interim change to getRelationsManager() to see first species in selectedSpeciesProfiles<SpeciesProfile>
> changed getSystemsTableManager() to iterate through each SpeciesProfile in selectedSpeciesProfiles<SpeciesProfile>
> interim change to getPrimarySystemTableManager to see first species in selectedSpeciesProfiles<SpeciesProfile>
>
> Compiles and runs. Will do a test export of Msmeg now and let it run while I sleep.
>
> Goodnight from the east coast!
>
> Richard
>
>
> On Tue, Aug 16, 2011 at 3:41 PM, John David N. Dionisio <do...@lm...> wrote:
> Yes, sounds right. Each species profile would then, in turn, call "submit" on the given table manager for each ID that should be exported. Presumably there is a taxon ID included with each record, so that when all is said and done you'll have the IDs of each chosen species, and you can tell which ID belongs to which species.
>
> If this goes well, your export should now have the Info table with the concatenated species names, with the system tables (UniProt, GeneID, OrderedLocus, etc.) having the IDs of the multiple species. The relationship tables should have just the IDs of the first species in the collection, per our gradual refactoring strategy. Presumably you can guess how the relationship tables should look once you have tackled that part of the code :)
>
> The QA check here, I would say, is to then match the ID counts *per species* against the equivalent counts in PostgreSQL (total count will not work now, since the user need not necessarily have chosen all of the species loaded in the PostgreSQL database). TallyEngine tweaks may be necessary after this; but we will not worry about that now. In the meantime, manual counts (i.e., issuing the queries manually) should do.
>
> Formulating those SQL queries will require an understanding of grouping and aggregate functions --- a fine exercise that is relevant to the database content of the course, I would say :)
>
> John David N. Dionisio, PhD
> Associate Professor, Computer Science
> Loyola Marymount University
>
>
>
> On Aug 16, 2011, at 3:32 PM, Richard Brous wrote:
>
> > OK, so its pretty much the last line of getSystemsTableManager() which needs to be modified to be aware of the collection of SpeciesProfile:
> >
> > tableManager =
> >
> > speciesProfile.getSystemsTableManagerCustomizations(tableManager, this);
> >
> > return tableManager;
> >
> > So I believe I need to iterate through each species of the collection before returning the final tableManager.
> >
> > I would enclose it in a for loop ( SpeciesProfile speciesProfile : speciesProfiles ) {
> >
> > }
> >
> > This would seem to be the correct implementation to ensure that each species customization would be applied.
> >
> >
> > Sound good??
> >
> > Richard
> >
> >
> >
> > On Tue, Aug 16, 2011 at 12:38 PM, Richard Brous <rbr...@gm...> wrote:
> > gotcha, heading to system tables.
> >
> > gotcha.
> >
> > gotcha.
> >
> > Thanks for the speedy reply =D
> >
> > Richard
> >
> > On Tue, Aug 16, 2011 at 12:21 PM, John David N. Dionisio <do...@lm...> wrote:
> > Hi Rich,
> >
> > Yeah, relations is pretty complicated. I suggest you skip that and go to just the system tables for now (i.e., the tables that hold the IDs themselves). The relationship tables hold pairs of IDs (i.e., which one from one system corresponds to the ID from another) and so is an additional level of complexity, I think.
> >
> > Regarding your prior question about the Info table --- I'm not or'ing the species names. I'm merely concatenating the names with pipes ("|") in between, as Kam specified. While the final value may look like an "or," it is ultimately just a string. The "|" could just have easily been a comma, semicolon, or other separator.
> >
> > Your prior guess as to how that would look, with the multiple { "Species", speciesName } pairs, would actually be equivalent to multiple values for the single Species column, which does not fit the relational model. The Species column can have only one value, and in this case it is a single string that is the concatenation of all selected species names, separated by "|" characters.
> >
> > Hope that clears things up...
> >
> > John David N. Dionisio, PhD
> > Associate Professor, Computer Science
> > Loyola Marymount University
> >
> >
> >
> > On Aug 16, 2011, at 12:11 PM, Richard Brous wrote:
> >
> > > OK, have moved onto getRelationsTableManager() ... this one seems pretty complicated...
> > >
> > > Have reviewed the method and submethods called and have the following questions:
> > >
> > > 1. What are the RelationshipTables that are stored in relationshipTables?
> > > aren't they species dependent? Can't seem to find where they were created?
> > > If not, then:
> > >
> > > 2. How should the if/else conditional be handled?
> > >
> > > if
> > >
> > > (speciesProfile.getSpeciesSpecificSystemTables().containsKey(stp.systemTable1) |
> > >
> > > speciesProfile.getSpeciesSpecificSystemTables().containsKey(stp.systemTable2)) {
> > > tableManager =
> > >
> > > speciesProfile.getRelationsTableManagerCustomizations(stp.systemTable1,
> > > stp.
> > >
> > > systemTable2, templateDefinedSystemToSystemCode, tableManager);
> > > This obviously needs to be made aware of Lists of SpeciesProfiles... to build the correct tableManager, do I need this conditional to run through every species or should we enclose all the code (if and else) within a loop through each species?
> > >
> > > Thanks.
> > >
> > > Richard
> > >
> > >
> > > On Tue, Aug 16, 2011 at 11:02 AM, Richard Brous <rbr...@gm...> wrote:
> > > Sorry I typed and sent that out too quickly before getting my thoughts completely together...
> > >
> > > First off i thought that the array[][] portion of the submit needed to be in the following format: { "Species", speciesName1 }, { "Species", speciesName2 }, { "Species", speciesName3 },...
> > > but I see you specified it as: { "Species", speciesName1 | speciesName2 | speciesName3 }, ... OR'ing each of the species names?
> > >
> > > I see what you did regarding the date object... yes no reason to recreate a second object...
> > >
> > > As mentioned previously I will continue through the next methods, submitting (well thought out) changes and emailing out updates...
> > >
> > > Richard
> > > On Tue, Aug 16, 2011 at 10:36 AM, Richard Brous <rbr...@gm...> wrote:
> > > Took a look at your code and realized what I had done wrong. I should have just broken out the loop as the second argument of the 3 in the submit method. That was a rookie move on my part.
> > >
> > > I am moving on to the next method and will keep my mind on the syntactic solution that I'm not thinking through prior to moving forward with an implementation.
> > >
> > > Richard
> > >
> > > On Mon, Aug 15, 2011 at 1:30 AM, John David N. Dionisio <do...@lm...> wrote:
> > > OK, everything is committed. I got up to the tweaks on the second panel in the wizard (Save As/GO Aspects), as well as the query change for exporting any combination of C, F, or P.
> > >
> > > I still have to do the GO OBO format check, plus the UI work on the remaining two wizard panels. But meanwhile, hope these latest changes work out well.
> > >
> > > John David N. Dionisio, PhD
> > > Associate Professor, Computer Science
> > > Loyola Marymount University
> > >
> > >
> > >
> > > On Aug 14, 2011, at 7:31 PM, Richard Brous wrote:
> > >
> > > > OK, using option 2 I have made changes to DatabaseProfile.java to allow for all species names to be included in the submit argument.
> > > > But I'm stuck on how to change my StringBuilder object to a type that submit wants. Help please!!
> > > >
> > > > Submitted the above and some comment changes to SourceForge this evening.
> > > >
> > > > Richard
> > > >
> > > > On Fri, Aug 12, 2011 at 12:08 PM, Kam Dahlquist <kda...@lm...> wrote:
> > > > Hi,
> > > >
> > > > I think we had better leave the info table with only one record
> > > > (option 2). The species names can be separated by pipes " | " as
> > > > they are in the Systems table where there are multiple species. To
> > > > my knowledge, the only time GenMAPP needs to access the info table is
> > > > for the "DisplayOrder" field, I don't know what would happen if there
> > > > were multiple records there. I know that the spec for the table says
> > > > that it should only be one record, but I don't know if it would crash
> > > > if there were multiple records. To be on the safe side, I think we
> > > > should just keep it to the one record.
> > > >
> > > > Best,
> > > > Kam
> > > >
> > > > At 09:45 PM 8/10/2011, John David N. Dionisio wrote:
> > > > >Greetings,
> > > > >
> > > > >I think we have to turn to Dr. Dahlquist's GenMAPP knowledge here to
> > > > >get the definitive answer. I see two choices:
> > > > >
> > > > >- The Info table should have one record for each species that the
> > > > >.gdb holds, in which case the change you need is to wrap that single
> > > > >submit call inside a loop, so that submit is called once for each
> > > > >chosen species.
> > > > >
> > > > >- The Info table should always have one record, and if the .gdb
> > > > >holds multiple species, the "Species" column should be some
> > > > >concatenation of multiple species names. In this case, you would
> > > > >still call submit only once, but the value you send into the
> > > > >"Species" column is some accumulation of all chosen species names.
> > > > >
> > > > >Admittedly I don't know which way is right (I assumed the former as
> > > > >of our Tuesday meeting, but on further examination I'm no longer
> > > > >quite so sure).
> > > > >
> > > > >For Kam --- what does GenMAPP expect to see in the Info table if the
> > > > >opened .gdb contains multiple species?
> > > > >
> > > > >John David N. Dionisio, PhD
> > > > >Associate Professor, Computer Science
> > > > >Loyola Marymount University
> > > > >
> > > > >
> > > > >On Aug 10, 2011, at 9:36 PM, Richard Brous wrote:
> > > > >
> > > > > > OK, continued to review ExportToGenMAPP and dug into the creation
> > > > > of the first TableManager tmA on line 118.
> > > > > >
> > > > > > In reading through the method, my understanding is that it
> > > > > creates a new TableManager based on the selectedDatabaseProfile
> > > > > (which is UniProt).
> > > > > >
> > > > > > This is performed by the method getInfoTableManager() which then
> > > > > calls method submit(String tableName, QueryType queryType,
> > > > > String[][] columnNamesToValues);
> > > > > >
> > > > > > the code is as follows:
> > > > > >
> > > > > > tableManager.submit("Info", QueryType.insert, new String[][] { {
> > > > > "Owner", owner }, { "Version", new
> > > > > SimpleDateFormat("yyyyMMdd").format(version) }, { "MODSystem",
> > > > > modSystem }, { "Species", speciesProfile.getSpeciesName() }, {
> > > > > "Modify", new SimpleDateFormat("yyyyMMdd").format(modify) }, {
> > > > > "DisplayOrder", displayOrder }, { "Notes", notes } });
> > > > > >
> > > > > >
> > > > > > The modification of this line centers on { "Species",
> > > > > speciesProfile.getSpeciesName() }, since it originally processed a
> > > > > single species.
> > > > > >
> > > > > > So now I need to populate the arguments with the species
> > > > > contained within selectedDatabaseprofile.selectedSpeciesProfiles.
> > > > > >
> > > > > > I think I'll start with the baseArgument up to MODSystem, then
> > > > > append as many species as necessary, and then cap off the end with
> > > > > the rest starting at Modify. (similar to your approach in
> > > > > ExportGoData, populateUniprotGoTableFromSQL(char chosenAspect,
> > > > > List<Integer> taxonIds) line 513
> > > > > >
> > > > > > Please let me know if this approach or analysis is off track.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > Richard
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, Aug 10, 2011 at 5:50 PM, Richard Brous <rbr...@gm...> wrote:
> > > > > > Updated repository to include all Gene Ontology changes discussed
> > > > > during our meeting yesterday.
> > > > > >
> > > > > > Digging into TableManager next.
> > > > > >
> > > > > > Richard
> > > > > >
> > > > > > On Fri, Aug 5, 2011 at 10:06 AM, Richard Brous <rbr...@gm...> wrote:
> > > > > > whew... thanks for the detailed reply. I will digest this a bit
> > > > > and get back to you with further questions.
> > > > > >
> > > > > > rb
> > > > > >
> > > > > > On Thu, Aug 4, 2011 at 11:18 PM, John David N. Dionisio
> > > > > <do...@lm...> wrote:
> > > > > > Greetings,
> > > > > >
> > > > > > Sorry for the delay. I wasn't able to walk through the relevant
> > > > > code until this evening.
> > > > > >
> > > > > > As Kam said, GOA serves as the link between the UniProt and GO
> > > > > IDs. It essentially determines which GO IDs get exported by using
> > > > > GOA to see which GO IDs are associated with an exported UniProt
> > > > > ID. The populateUniprotGoTableFromSQL, in its current form,
> > > > > extracts the GO association records that match the given taxon ID
> > > > > then exports, as UniProt-GO pairs, the GO and UniProt IDs
> > > > > referenced within that GO association record. Processing that
> > > > > follows this is then based on the GO IDs that got exported --- and
> > > > > that's how the current code avoids exporting the entire list of GO terms.
> > > > > >
> > > > > > The operative query is on the second line of populateUniprotGoTableFromSQL:
> > > > > >
> > > > > > String uniProtAndGOIDSQL = "select db_object_id, go_id,
> > > > > evidence_code, with_or_from from goa where db like '%UniProt%' and
> > > > > taxon = 'taxon:" + taxon + "'";
> > > > > >
> > > > > > In plain English, this selects the GOA records whose database is
> > > > > UniProt and whose taxon ID is the given taxon. An additional
> > > > > condition is added for the "aspect" (All, Component, Function, or
> > > > > Process) that is to be exported. This is another reduction filter,
> > > > > to further shrink the number of exported GO terms and thus avoid
> > > > > MAPPFinder issues later on.
> > > > > >
> > > > > > Given this, the proper expansion here is to change the taxon
> > > > > predicate to a multiple predicate. That is, this method can be
> > > > > changed to now accept a collection or array of taxon IDs, and the
> > > > > base query should then be changed so that it accepts any taxon from
> > > > > that collection. More or less, you want:
> > > > > >
> > > > > > private void populateUniprotGoTableFromSQL(char chosenAspect,
> > > > > int[] taxons) throws SQLException {
> > > > > >
> > > > > > ...then, instead of the single string, you want to iterate
> > > > > through the taxon IDs:
> > > > > >
> > > > > > StringBuilder baseQueryBuilder = new StringBuilder("select
> > > > > db_object_id, go_id, evidence_code, with_or_from from goa where db
> > > > > like '%UniProt%'");
> > > > > > boolean first = true;
> > > > > > for (int taxon: taxons) {
> > > > > > baseQueryBuilder.append(first ? " and (" : " or ");
> > > > > > baseQueryBuilder
> > > > > > .append("taxon = 'taxon:")
> > > > > > .append(taxon).append("'");
> > > > > > first = false;
> > > > > > }
> > > > > > baseQueryBuilder.append(")");
> > > > > >
> > > > > > ...and so on. I just sort of rattled this off so there may be
> > > > > little glitches, but anyway this is just to give you an overall idea.
> > > > > >
> > > > > > Put another way, no, you do not need to iterate this method for
> > > > > each taxon ID. Instead, you can still call this method once, with
> > > > > the multiplicity of taxon IDs emerging in terms of the actual
> > > > > condition used for selecting the GO terms to be exported (based on
> > > > > the available GOA records, which as you may recall are loaded from .goa files).
> > > > > >
> > > > > > As a side note, right here you have an opportunity for a little
> > > > > sanity check regarding the content of the relational database: GO
> > > > > terms will only be exported if GOA records for the desired taxon
> > > > > IDs have been imported into the database. So, as a pre-flight
> > > > > check, one can see if there are any GOA records at all for each
> > > > > chosen taxon ID. If there are none, then the .goa file for that
> > > > > species needs to be imported into the relational database.
> > > > > >
> > > > > > Hope this helps...
> > > > > >
> > > > > > John David N. Dionisio, PhD
> > > > > > Associate Professor, Computer Science
> > > > > > Loyola Marymount University
> > > > > >
> > > > > >
> > > > > > On Aug 4, 2011, at 1:00 PM, Kam Dahlquist wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > Dondi will have to chime in on this, but I think this is where
> > > > > things are going to get tricky.
> > > > > > >
> > > > > > > The final gdb does not actually contain the entire GO, it gets
> > > > > trimmed somehow based on the GO associations for a particular
> > > > > species. This is because MAPPFinder cannot handle loading the
> > > > > entire GO. Since there is some type of species-specific trimming
> > > > > going on, it's quite possible that this will need to iterate.
> > > > > > >
> > > > > > > However, I don't have the foggiest idea of how this works, so
> > > > > Dondi will have to chime in.
> > > > > > >
> > > > > > > Best,
> > > > > > > Kam
> > > > > > >
> > > > > > > At 12:09 AM 8/4/2011, you wrote:
> > > > > > >> Wednesday 8/3/11 progress:
> > > > > > >>
> > > > > > >> 1. After following the ExportPanel1.java ground zero code of:
> > > > > databaseProfile.setSelectedSpeciesProfile( selectedProfile );
> > > > > > >>
> > > > > > >> I found the method in DatabaseProfile.java plus a getter method;
> > > > > > >> SpeciesProfile setSelectedSpeciesProfile( speciesProfile ) and
> > > > > SpeciesProfile getSelectedSpeciesProfile( speciesProfile )
> > > > > > >>
> > > > > > >> I created two new methods that each handle List<Object> of
> > > > > SpeciesProfiles argument instead of a single SpeciesProfile;
> > > > > setSelectedSpeciesProfiles and getSelectedSpeciesProfiles.
> > > > > > >>
> > > > > > >> This enabled the ExportPanel1 ground zero code to become:
> > > > > databaseProfile.setSelectedSpeciesProfiles(selectedSpecies);
> > > > > > >>
> > > > > > >> 2. public static void export() on line 104 in ExportToGenMAPP.java
> > > > > > >>
> > > > > > >> On line 107 ExportGoData is instantiated which I found in
> > > > > ExportGoData.java and calls a method: public void export(char
> > > > > chosenAspect, int taxon).
> > > > > > >>
> > > > > > >> Within export, taxon id is required for another method:
> > > > > private void populateGoTables(char chosenAspect, int taxon).
> > > > > > >>
> > > > > > >> Within populateGoTables, taxon id is required for another
> > > > > method: private void populateUniprotGoTableFromSQL( char
> > > > > chosenAspect, int taxon).
> > > > > > >>
> > > > > > >> But, if the export to GDB process starts off with exporting GO
> > > > > data, doesn't it only need to do that once no matter how many
> > > > > species are selected? As you probably realize, I'm leading towards
> > > > > not having to iterate through this for each taxon id if possible.
> > > > > > >>
> > > > > > >> Also, how does the export actually work? How are GO ids and
> > > > > UniProt ids related within the table?
> > > > > > >>
> > > > > > >> Thanks!
> > > > > > >>
> > > > > > >> Richard
> > > > > > >>
> > > > > > >>
> > > > > > > <ATT00001..txt><ATT00002..txt>
> > > > > >
> > > > > >
> > > > > >
> > > > > ------------------------------------------------------------------------------
> > > > > > BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
> > > > > > The must-attend event for mobile developers. Connect with experts.
> > > > > > Get tools for creating Super Apps. See the latest technologies.
> > > > > > Sessions, hands-on labs, demos & much more. Register early & save!
> > > > > > http://p.sf.net/sfu/rim-blackberry-1
> > > > > > _______________________________________________
> > > > > > xmlpipedb-developer mailing list
> > > > > > xml...@li...
> > > > > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> > > > > >
> > > > > >
> > > > > >
> > > > > > <ATT00001..txt><ATT00002..txt>
> > > > >
> > > > >
> > > > >------------------------------------------------------------------------------
> > > > >Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
> > > > >user administration capabilities and model configuration. Take
> > > > >the hassle out of deploying and managing Subversion and the
> > > > >tools developers use with it.
> > > > >http://p.sf.net/sfu/wandisco-dev2dev
> > > > >_______________________________________________
> > > > >xmlpipedb-developer mailing list
> > > > >xml...@li...
> > > > >https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> > > >
> > > >
> > > > ------------------------------------------------------------------------------
> > > > FREE DOWNLOAD - uberSVN with Social Coding for Subversion.
> > > > Subversion made easy with a complete admin console. Easy
> > > > to use, easy to manage, easy to install, easy to extend.
> > > > Get a Free download of the new open ALM Subversion platform now.
> > > > http://p.sf.net/sfu/wandisco-dev2dev
> > > > _______________________________________________
> > > > xmlpipedb-developer mailing list
> > > > xml...@li...
> > > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> > > >
> > > > <ATT00001..txt><ATT00002..txt>
> > >
> > >
> > > ------------------------------------------------------------------------------
> > > uberSVN's rich system and user administration capabilities and model
> > > configuration take the hassle out of deploying and managing Subversion and
> > > the tools developers use with it. Learn more about uberSVN and get a free
> > > download at: http://p.sf.net/sfu/wandisco-dev2dev
> > > _______________________________________________
> > > xmlpipedb-developer mailing list
> > > xml...@li...
> > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> > >
> > >
> > >
> > > <ATT00001..txt><ATT00002..txt>
> >
> >
> > ------------------------------------------------------------------------------
> > Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
> > user administration capabilities and model configuration. Take
> > the hassle out of deploying and managing Subversion and the
> > tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2
> > _______________________________________________
> > xmlpipedb-developer mailing list
> > xml...@li...
> > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> >
> >
> > <ATT00001..txt><ATT00002..txt>
>
>
> ------------------------------------------------------------------------------
> Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
> user administration capabilities and model configuration. Take
> the hassle out of deploying and managing Subversion and the
> tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2
> _______________________________________________
> xmlpipedb-developer mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>
> <ATT00001..txt><ATT00002..txt>
|
|
From: Richard B. <rbr...@gm...> - 2011-08-17 04:12:56
|
Submitted more changes to SourceForge: changes to ExportToGenMAPP,
DatabaseProfile and UniProtDatabaseProfile
interim change to getRelationsManager() to see first species in
selectedSpeciesProfiles<SpeciesProfile>
changed getSystemsTableManager() to iterate through each SpeciesProfile in
selectedSpeciesProfiles<SpeciesProfile>
interim change to getPrimarySystemTableManager to see first species in
selectedSpeciesProfiles<SpeciesProfile>
Compiles and runs. Will do a test export of Msmeg now and let it run while I
sleep.
Goodnight from the east coast!
Richard
On Tue, Aug 16, 2011 at 3:41 PM, John David N. Dionisio <do...@lm...>wrote:
> Yes, sounds right. Each species profile would then, in turn, call "submit"
> on the given table manager for each ID that should be exported. Presumably
> there is a taxon ID included with each record, so that when all is said and
> done you'll have the IDs of each chosen species, and you can tell which ID
> belongs to which species.
>
> If this goes well, your export should now have the Info table with the
> concatenated species names, with the system tables (UniProt, GeneID,
> OrderedLocus, etc.) having the IDs of the multiple species. The
> relationship tables should have just the IDs of the first species in the
> collection, per our gradual refactoring strategy. Presumably you can guess
> how the relationship tables should look once you have tackled that part of
> the code :)
>
> The QA check here, I would say, is to then match the ID counts *per
> species* against the equivalent counts in PostgreSQL (total count will not
> work now, since the user need not necessarily have chosen all of the species
> loaded in the PostgreSQL database). TallyEngine tweaks may be necessary
> after this; but we will not worry about that now. In the meantime, manual
> counts (i.e., issuing the queries manually) should do.
>
> Formulating those SQL queries will require an understanding of grouping and
> aggregate functions --- a fine exercise that is relevant to the database
> content of the course, I would say :)
>
> John David N. Dionisio, PhD
> Associate Professor, Computer Science
> Loyola Marymount University
>
>
>
> On Aug 16, 2011, at 3:32 PM, Richard Brous wrote:
>
> > OK, so its pretty much the last line of getSystemsTableManager() which
> needs to be modified to be aware of the collection of SpeciesProfile:
> >
> > tableManager =
> >
> > speciesProfile.getSystemsTableManagerCustomizations(tableManager, this);
> >
> > return tableManager;
> >
> > So I believe I need to iterate through each species of the collection
> before returning the final tableManager.
> >
> > I would enclose it in a for loop ( SpeciesProfile speciesProfile :
> speciesProfiles ) {
> >
> > }
> >
> > This would seem to be the correct implementation to ensure that each
> species customization would be applied.
> >
> >
> > Sound good??
> >
> > Richard
> >
> >
> >
> > On Tue, Aug 16, 2011 at 12:38 PM, Richard Brous <rbr...@gm...>
> wrote:
> > gotcha, heading to system tables.
> >
> > gotcha.
> >
> > gotcha.
> >
> > Thanks for the speedy reply =D
> >
> > Richard
> >
> > On Tue, Aug 16, 2011 at 12:21 PM, John David N. Dionisio <do...@lm...>
> wrote:
> > Hi Rich,
> >
> > Yeah, relations is pretty complicated. I suggest you skip that and go to
> just the system tables for now (i.e., the tables that hold the IDs
> themselves). The relationship tables hold pairs of IDs (i.e., which one
> from one system corresponds to the ID from another) and so is an additional
> level of complexity, I think.
> >
> > Regarding your prior question about the Info table --- I'm not or'ing the
> species names. I'm merely concatenating the names with pipes ("|") in
> between, as Kam specified. While the final value may look like an "or," it
> is ultimately just a string. The "|" could just have easily been a comma,
> semicolon, or other separator.
> >
> > Your prior guess as to how that would look, with the multiple {
> "Species", speciesName } pairs, would actually be equivalent to multiple
> values for the single Species column, which does not fit the relational
> model. The Species column can have only one value, and in this case it is a
> single string that is the concatenation of all selected species names,
> separated by "|" characters.
> >
> > Hope that clears things up...
> >
> > John David N. Dionisio, PhD
> > Associate Professor, Computer Science
> > Loyola Marymount University
> >
> >
> >
> > On Aug 16, 2011, at 12:11 PM, Richard Brous wrote:
> >
> > > OK, have moved onto getRelationsTableManager() ... this one seems
> pretty complicated...
> > >
> > > Have reviewed the method and submethods called and have the following
> questions:
> > >
> > > 1. What are the RelationshipTables that are stored in
> relationshipTables?
> > > aren't they species dependent? Can't seem to find where they were
> created?
> > > If not, then:
> > >
> > > 2. How should the if/else conditional be handled?
> > >
> > > if
> > >
> > >
> (speciesProfile.getSpeciesSpecificSystemTables().containsKey(stp.systemTable1)
> |
> > >
> > >
> speciesProfile.getSpeciesSpecificSystemTables().containsKey(stp.systemTable2))
> {
> > > tableManager =
> > >
> > > speciesProfile.getRelationsTableManagerCustomizations(stp.systemTable1,
> > > stp.
> > >
> > > systemTable2, templateDefinedSystemToSystemCode, tableManager);
> > > This obviously needs to be made aware of Lists of SpeciesProfiles... to
> build the correct tableManager, do I need this conditional to run through
> every species or should we enclose all the code (if and else) within a loop
> through each species?
> > >
> > > Thanks.
> > >
> > > Richard
> > >
> > >
> > > On Tue, Aug 16, 2011 at 11:02 AM, Richard Brous <rbr...@gm...>
> wrote:
> > > Sorry I typed and sent that out too quickly before getting my thoughts
> completely together...
> > >
> > > First off i thought that the array[][] portion of the submit needed to
> be in the following format: { "Species", speciesName1 }, { "Species",
> speciesName2 }, { "Species", speciesName3 },...
> > > but I see you specified it as: { "Species", speciesName1 | speciesName2
> | speciesName3 }, ... OR'ing each of the species names?
> > >
> > > I see what you did regarding the date object... yes no reason to
> recreate a second object...
> > >
> > > As mentioned previously I will continue through the next methods,
> submitting (well thought out) changes and emailing out updates...
> > >
> > > Richard
> > > On Tue, Aug 16, 2011 at 10:36 AM, Richard Brous <rbr...@gm...>
> wrote:
> > > Took a look at your code and realized what I had done wrong. I should
> have just broken out the loop as the second argument of the 3 in the submit
> method. That was a rookie move on my part.
> > >
> > > I am moving on to the next method and will keep my mind on the
> syntactic solution that I'm not thinking through prior to moving forward
> with an implementation.
> > >
> > > Richard
> > >
> > > On Mon, Aug 15, 2011 at 1:30 AM, John David N. Dionisio <do...@lm...>
> wrote:
> > > OK, everything is committed. I got up to the tweaks on the second
> panel in the wizard (Save As/GO Aspects), as well as the query change for
> exporting any combination of C, F, or P.
> > >
> > > I still have to do the GO OBO format check, plus the UI work on the
> remaining two wizard panels. But meanwhile, hope these latest changes work
> out well.
> > >
> > > John David N. Dionisio, PhD
> > > Associate Professor, Computer Science
> > > Loyola Marymount University
> > >
> > >
> > >
> > > On Aug 14, 2011, at 7:31 PM, Richard Brous wrote:
> > >
> > > > OK, using option 2 I have made changes to DatabaseProfile.java to
> allow for all species names to be included in the submit argument.
> > > > But I'm stuck on how to change my StringBuilder object to a type that
> submit wants. Help please!!
> > > >
> > > > Submitted the above and some comment changes to SourceForge this
> evening.
> > > >
> > > > Richard
> > > >
> > > > On Fri, Aug 12, 2011 at 12:08 PM, Kam Dahlquist <kda...@lm...>
> wrote:
> > > > Hi,
> > > >
> > > > I think we had better leave the info table with only one record
> > > > (option 2). The species names can be separated by pipes " | " as
> > > > they are in the Systems table where there are multiple species. To
> > > > my knowledge, the only time GenMAPP needs to access the info table is
> > > > for the "DisplayOrder" field, I don't know what would happen if there
> > > > were multiple records there. I know that the spec for the table says
> > > > that it should only be one record, but I don't know if it would crash
> > > > if there were multiple records. To be on the safe side, I think we
> > > > should just keep it to the one record.
> > > >
> > > > Best,
> > > > Kam
> > > >
> > > > At 09:45 PM 8/10/2011, John David N. Dionisio wrote:
> > > > >Greetings,
> > > > >
> > > > >I think we have to turn to Dr. Dahlquist's GenMAPP knowledge here to
> > > > >get the definitive answer. I see two choices:
> > > > >
> > > > >- The Info table should have one record for each species that the
> > > > >.gdb holds, in which case the change you need is to wrap that single
> > > > >submit call inside a loop, so that submit is called once for each
> > > > >chosen species.
> > > > >
> > > > >- The Info table should always have one record, and if the .gdb
> > > > >holds multiple species, the "Species" column should be some
> > > > >concatenation of multiple species names. In this case, you would
> > > > >still call submit only once, but the value you send into the
> > > > >"Species" column is some accumulation of all chosen species names.
> > > > >
> > > > >Admittedly I don't know which way is right (I assumed the former as
> > > > >of our Tuesday meeting, but on further examination I'm no longer
> > > > >quite so sure).
> > > > >
> > > > >For Kam --- what does GenMAPP expect to see in the Info table if the
> > > > >opened .gdb contains multiple species?
> > > > >
> > > > >John David N. Dionisio, PhD
> > > > >Associate Professor, Computer Science
> > > > >Loyola Marymount University
> > > > >
> > > > >
> > > > >On Aug 10, 2011, at 9:36 PM, Richard Brous wrote:
> > > > >
> > > > > > OK, continued to review ExportToGenMAPP and dug into the creation
> > > > > of the first TableManager tmA on line 118.
> > > > > >
> > > > > > In reading through the method, my understanding is that it
> > > > > creates a new TableManager based on the selectedDatabaseProfile
> > > > > (which is UniProt).
> > > > > >
> > > > > > This is performed by the method getInfoTableManager() which then
> > > > > calls method submit(String tableName, QueryType queryType,
> > > > > String[][] columnNamesToValues);
> > > > > >
> > > > > > the code is as follows:
> > > > > >
> > > > > > tableManager.submit("Info", QueryType.insert, new String[][] { {
> > > > > "Owner", owner }, { "Version", new
> > > > > SimpleDateFormat("yyyyMMdd").format(version) }, { "MODSystem",
> > > > > modSystem }, { "Species", speciesProfile.getSpeciesName() }, {
> > > > > "Modify", new SimpleDateFormat("yyyyMMdd").format(modify) }, {
> > > > > "DisplayOrder", displayOrder }, { "Notes", notes } });
> > > > > >
> > > > > >
> > > > > > The modification of this line centers on { "Species",
> > > > > speciesProfile.getSpeciesName() }, since it originally processed a
> > > > > single species.
> > > > > >
> > > > > > So now I need to populate the arguments with the species
> > > > > contained within selectedDatabaseprofile.selectedSpeciesProfiles.
> > > > > >
> > > > > > I think I'll start with the baseArgument up to MODSystem, then
> > > > > append as many species as necessary, and then cap off the end with
> > > > > the rest starting at Modify. (similar to your approach in
> > > > > ExportGoData, populateUniprotGoTableFromSQL(char chosenAspect,
> > > > > List<Integer> taxonIds) line 513
> > > > > >
> > > > > > Please let me know if this approach or analysis is off track.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > Richard
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, Aug 10, 2011 at 5:50 PM, Richard Brous <
> rbr...@gm...> wrote:
> > > > > > Updated repository to include all Gene Ontology changes discussed
> > > > > during our meeting yesterday.
> > > > > >
> > > > > > Digging into TableManager next.
> > > > > >
> > > > > > Richard
> > > > > >
> > > > > > On Fri, Aug 5, 2011 at 10:06 AM, Richard Brous <
> rbr...@gm...> wrote:
> > > > > > whew... thanks for the detailed reply. I will digest this a bit
> > > > > and get back to you with further questions.
> > > > > >
> > > > > > rb
> > > > > >
> > > > > > On Thu, Aug 4, 2011 at 11:18 PM, John David N. Dionisio
> > > > > <do...@lm...> wrote:
> > > > > > Greetings,
> > > > > >
> > > > > > Sorry for the delay. I wasn't able to walk through the relevant
> > > > > code until this evening.
> > > > > >
> > > > > > As Kam said, GOA serves as the link between the UniProt and GO
> > > > > IDs. It essentially determines which GO IDs get exported by using
> > > > > GOA to see which GO IDs are associated with an exported UniProt
> > > > > ID. The populateUniprotGoTableFromSQL, in its current form,
> > > > > extracts the GO association records that match the given taxon ID
> > > > > then exports, as UniProt-GO pairs, the GO and UniProt IDs
> > > > > referenced within that GO association record. Processing that
> > > > > follows this is then based on the GO IDs that got exported --- and
> > > > > that's how the current code avoids exporting the entire list of GO
> terms.
> > > > > >
> > > > > > The operative query is on the second line of
> populateUniprotGoTableFromSQL:
> > > > > >
> > > > > > String uniProtAndGOIDSQL = "select db_object_id, go_id,
> > > > > evidence_code, with_or_from from goa where db like '%UniProt%' and
> > > > > taxon = 'taxon:" + taxon + "'";
> > > > > >
> > > > > > In plain English, this selects the GOA records whose database is
> > > > > UniProt and whose taxon ID is the given taxon. An additional
> > > > > condition is added for the "aspect" (All, Component, Function, or
> > > > > Process) that is to be exported. This is another reduction filter,
> > > > > to further shrink the number of exported GO terms and thus avoid
> > > > > MAPPFinder issues later on.
> > > > > >
> > > > > > Given this, the proper expansion here is to change the taxon
> > > > > predicate to a multiple predicate. That is, this method can be
> > > > > changed to now accept a collection or array of taxon IDs, and the
> > > > > base query should then be changed so that it accepts any taxon from
> > > > > that collection. More or less, you want:
> > > > > >
> > > > > > private void populateUniprotGoTableFromSQL(char chosenAspect,
> > > > > int[] taxons) throws SQLException {
> > > > > >
> > > > > > ...then, instead of the single string, you want to iterate
> > > > > through the taxon IDs:
> > > > > >
> > > > > > StringBuilder baseQueryBuilder = new StringBuilder("select
> > > > > db_object_id, go_id, evidence_code, with_or_from from goa where db
> > > > > like '%UniProt%'");
> > > > > > boolean first = true;
> > > > > > for (int taxon: taxons) {
> > > > > > baseQueryBuilder.append(first ? " and (" : " or ");
> > > > > > baseQueryBuilder
> > > > > > .append("taxon = 'taxon:")
> > > > > > .append(taxon).append("'");
> > > > > > first = false;
> > > > > > }
> > > > > > baseQueryBuilder.append(")");
> > > > > >
> > > > > > ...and so on. I just sort of rattled this off so there may be
> > > > > little glitches, but anyway this is just to give you an overall
> idea.
> > > > > >
> > > > > > Put another way, no, you do not need to iterate this method for
> > > > > each taxon ID. Instead, you can still call this method once, with
> > > > > the multiplicity of taxon IDs emerging in terms of the actual
> > > > > condition used for selecting the GO terms to be exported (based on
> > > > > the available GOA records, which as you may recall are loaded from
> .goa files).
> > > > > >
> > > > > > As a side note, right here you have an opportunity for a little
> > > > > sanity check regarding the content of the relational database: GO
> > > > > terms will only be exported if GOA records for the desired taxon
> > > > > IDs have been imported into the database. So, as a pre-flight
> > > > > check, one can see if there are any GOA records at all for each
> > > > > chosen taxon ID. If there are none, then the .goa file for that
> > > > > species needs to be imported into the relational database.
> > > > > >
> > > > > > Hope this helps...
> > > > > >
> > > > > > John David N. Dionisio, PhD
> > > > > > Associate Professor, Computer Science
> > > > > > Loyola Marymount University
> > > > > >
> > > > > >
> > > > > > On Aug 4, 2011, at 1:00 PM, Kam Dahlquist wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > Dondi will have to chime in on this, but I think this is where
> > > > > things are going to get tricky.
> > > > > > >
> > > > > > > The final gdb does not actually contain the entire GO, it gets
> > > > > trimmed somehow based on the GO associations for a particular
> > > > > species. This is because MAPPFinder cannot handle loading the
> > > > > entire GO. Since there is some type of species-specific trimming
> > > > > going on, it's quite possible that this will need to iterate.
> > > > > > >
> > > > > > > However, I don't have the foggiest idea of how this works, so
> > > > > Dondi will have to chime in.
> > > > > > >
> > > > > > > Best,
> > > > > > > Kam
> > > > > > >
> > > > > > > At 12:09 AM 8/4/2011, you wrote:
> > > > > > >> Wednesday 8/3/11 progress:
> > > > > > >>
> > > > > > >> 1. After following the ExportPanel1.java ground zero code of:
> > > > > databaseProfile.setSelectedSpeciesProfile( selectedProfile );
> > > > > > >>
> > > > > > >> I found the method in DatabaseProfile.java plus a getter
> method;
> > > > > > >> SpeciesProfile setSelectedSpeciesProfile( speciesProfile ) and
> > > > > SpeciesProfile getSelectedSpeciesProfile( speciesProfile )
> > > > > > >>
> > > > > > >> I created two new methods that each handle List<Object> of
> > > > > SpeciesProfiles argument instead of a single SpeciesProfile;
> > > > > setSelectedSpeciesProfiles and getSelectedSpeciesProfiles.
> > > > > > >>
> > > > > > >> This enabled the ExportPanel1 ground zero code to become:
> > > > > databaseProfile.setSelectedSpeciesProfiles(selectedSpecies);
> > > > > > >>
> > > > > > >> 2. public static void export() on line 104 in
> ExportToGenMAPP.java
> > > > > > >>
> > > > > > >> On line 107 ExportGoData is instantiated which I found in
> > > > > ExportGoData.java and calls a method: public void export(char
> > > > > chosenAspect, int taxon).
> > > > > > >>
> > > > > > >> Within export, taxon id is required for another method:
> > > > > private void populateGoTables(char chosenAspect, int taxon).
> > > > > > >>
> > > > > > >> Within populateGoTables, taxon id is required for another
> > > > > method: private void populateUniprotGoTableFromSQL( char
> > > > > chosenAspect, int taxon).
> > > > > > >>
> > > > > > >> But, if the export to GDB process starts off with exporting GO
> > > > > data, doesn't it only need to do that once no matter how many
> > > > > species are selected? As you probably realize, I'm leading towards
> > > > > not having to iterate through this for each taxon id if possible.
> > > > > > >>
> > > > > > >> Also, how does the export actually work? How are GO ids and
> > > > > UniProt ids related within the table?
> > > > > > >>
> > > > > > >> Thanks!
> > > > > > >>
> > > > > > >> Richard
> > > > > > >>
> > > > > > >>
> > > > > > > <ATT00001..txt><ATT00002..txt>
> > > > > >
> > > > > >
> > > > > >
> > > > >
> ------------------------------------------------------------------------------
> > > > > > BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
> > > > > > The must-attend event for mobile developers. Connect with
> experts.
> > > > > > Get tools for creating Super Apps. See the latest technologies.
> > > > > > Sessions, hands-on labs, demos & much more. Register early &
> save!
> > > > > > http://p.sf.net/sfu/rim-blackberry-1
> > > > > > _______________________________________________
> > > > > > xmlpipedb-developer mailing list
> > > > > > xml...@li...
> > > > > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> > > > > >
> > > > > >
> > > > > >
> > > > > > <ATT00001..txt><ATT00002..txt>
> > > > >
> > > > >
> > > >
> >------------------------------------------------------------------------------
> > > > >Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
> > > > >user administration capabilities and model configuration. Take
> > > > >the hassle out of deploying and managing Subversion and the
> > > > >tools developers use with it.
> > > > >http://p.sf.net/sfu/wandisco-dev2dev
> > > > >_______________________________________________
> > > > >xmlpipedb-developer mailing list
> > > > >xml...@li...
> > > > >https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> > > >
> > > >
> > > >
> ------------------------------------------------------------------------------
> > > > FREE DOWNLOAD - uberSVN with Social Coding for Subversion.
> > > > Subversion made easy with a complete admin console. Easy
> > > > to use, easy to manage, easy to install, easy to extend.
> > > > Get a Free download of the new open ALM Subversion platform now.
> > > > http://p.sf.net/sfu/wandisco-dev2dev
> > > > _______________________________________________
> > > > xmlpipedb-developer mailing list
> > > > xml...@li...
> > > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> > > >
> > > > <ATT00001..txt><ATT00002..txt>
> > >
> > >
> > >
> ------------------------------------------------------------------------------
> > > uberSVN's rich system and user administration capabilities and model
> > > configuration take the hassle out of deploying and managing Subversion
> and
> > > the tools developers use with it. Learn more about uberSVN and get a
> free
> > > download at: http://p.sf.net/sfu/wandisco-dev2dev
> > > _______________________________________________
> > > xmlpipedb-developer mailing list
> > > xml...@li...
> > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> > >
> > >
> > >
> > > <ATT00001..txt><ATT00002..txt>
> >
> >
> >
> ------------------------------------------------------------------------------
> > Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
> > user administration capabilities and model configuration. Take
> > the hassle out of deploying and managing Subversion and the
> > tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2
> > _______________________________________________
> > xmlpipedb-developer mailing list
> > xml...@li...
> > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> >
> >
> > <ATT00001..txt><ATT00002..txt>
>
>
>
> ------------------------------------------------------------------------------
> Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
> user administration capabilities and model configuration. Take
> the hassle out of deploying and managing Subversion and the
> tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2
> _______________________________________________
> xmlpipedb-developer mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>
|
|
From: John D. N. D. <do...@lm...> - 2011-08-16 22:41:18
|
Yes, sounds right. Each species profile would then, in turn, call "submit" on the given table manager for each ID that should be exported. Presumably there is a taxon ID included with each record, so that when all is said and done you'll have the IDs of each chosen species, and you can tell which ID belongs to which species.
If this goes well, your export should now have the Info table with the concatenated species names, with the system tables (UniProt, GeneID, OrderedLocus, etc.) having the IDs of the multiple species. The relationship tables should have just the IDs of the first species in the collection, per our gradual refactoring strategy. Presumably you can guess how the relationship tables should look once you have tackled that part of the code :)
The QA check here, I would say, is to then match the ID counts *per species* against the equivalent counts in PostgreSQL (total count will not work now, since the user need not necessarily have chosen all of the species loaded in the PostgreSQL database). TallyEngine tweaks may be necessary after this; but we will not worry about that now. In the meantime, manual counts (i.e., issuing the queries manually) should do.
Formulating those SQL queries will require an understanding of grouping and aggregate functions --- a fine exercise that is relevant to the database content of the course, I would say :)
John David N. Dionisio, PhD
Associate Professor, Computer Science
Loyola Marymount University
On Aug 16, 2011, at 3:32 PM, Richard Brous wrote:
> OK, so its pretty much the last line of getSystemsTableManager() which needs to be modified to be aware of the collection of SpeciesProfile:
>
> tableManager =
>
> speciesProfile.getSystemsTableManagerCustomizations(tableManager, this);
>
> return tableManager;
>
> So I believe I need to iterate through each species of the collection before returning the final tableManager.
>
> I would enclose it in a for loop ( SpeciesProfile speciesProfile : speciesProfiles ) {
>
> }
>
> This would seem to be the correct implementation to ensure that each species customization would be applied.
>
>
> Sound good??
>
> Richard
>
>
>
> On Tue, Aug 16, 2011 at 12:38 PM, Richard Brous <rbr...@gm...> wrote:
> gotcha, heading to system tables.
>
> gotcha.
>
> gotcha.
>
> Thanks for the speedy reply =D
>
> Richard
>
> On Tue, Aug 16, 2011 at 12:21 PM, John David N. Dionisio <do...@lm...> wrote:
> Hi Rich,
>
> Yeah, relations is pretty complicated. I suggest you skip that and go to just the system tables for now (i.e., the tables that hold the IDs themselves). The relationship tables hold pairs of IDs (i.e., which one from one system corresponds to the ID from another) and so is an additional level of complexity, I think.
>
> Regarding your prior question about the Info table --- I'm not or'ing the species names. I'm merely concatenating the names with pipes ("|") in between, as Kam specified. While the final value may look like an "or," it is ultimately just a string. The "|" could just have easily been a comma, semicolon, or other separator.
>
> Your prior guess as to how that would look, with the multiple { "Species", speciesName } pairs, would actually be equivalent to multiple values for the single Species column, which does not fit the relational model. The Species column can have only one value, and in this case it is a single string that is the concatenation of all selected species names, separated by "|" characters.
>
> Hope that clears things up...
>
> John David N. Dionisio, PhD
> Associate Professor, Computer Science
> Loyola Marymount University
>
>
>
> On Aug 16, 2011, at 12:11 PM, Richard Brous wrote:
>
> > OK, have moved onto getRelationsTableManager() ... this one seems pretty complicated...
> >
> > Have reviewed the method and submethods called and have the following questions:
> >
> > 1. What are the RelationshipTables that are stored in relationshipTables?
> > aren't they species dependent? Can't seem to find where they were created?
> > If not, then:
> >
> > 2. How should the if/else conditional be handled?
> >
> > if
> >
> > (speciesProfile.getSpeciesSpecificSystemTables().containsKey(stp.systemTable1) |
> >
> > speciesProfile.getSpeciesSpecificSystemTables().containsKey(stp.systemTable2)) {
> > tableManager =
> >
> > speciesProfile.getRelationsTableManagerCustomizations(stp.systemTable1,
> > stp.
> >
> > systemTable2, templateDefinedSystemToSystemCode, tableManager);
> > This obviously needs to be made aware of Lists of SpeciesProfiles... to build the correct tableManager, do I need this conditional to run through every species or should we enclose all the code (if and else) within a loop through each species?
> >
> > Thanks.
> >
> > Richard
> >
> >
> > On Tue, Aug 16, 2011 at 11:02 AM, Richard Brous <rbr...@gm...> wrote:
> > Sorry I typed and sent that out too quickly before getting my thoughts completely together...
> >
> > First off i thought that the array[][] portion of the submit needed to be in the following format: { "Species", speciesName1 }, { "Species", speciesName2 }, { "Species", speciesName3 },...
> > but I see you specified it as: { "Species", speciesName1 | speciesName2 | speciesName3 }, ... OR'ing each of the species names?
> >
> > I see what you did regarding the date object... yes no reason to recreate a second object...
> >
> > As mentioned previously I will continue through the next methods, submitting (well thought out) changes and emailing out updates...
> >
> > Richard
> > On Tue, Aug 16, 2011 at 10:36 AM, Richard Brous <rbr...@gm...> wrote:
> > Took a look at your code and realized what I had done wrong. I should have just broken out the loop as the second argument of the 3 in the submit method. That was a rookie move on my part.
> >
> > I am moving on to the next method and will keep my mind on the syntactic solution that I'm not thinking through prior to moving forward with an implementation.
> >
> > Richard
> >
> > On Mon, Aug 15, 2011 at 1:30 AM, John David N. Dionisio <do...@lm...> wrote:
> > OK, everything is committed. I got up to the tweaks on the second panel in the wizard (Save As/GO Aspects), as well as the query change for exporting any combination of C, F, or P.
> >
> > I still have to do the GO OBO format check, plus the UI work on the remaining two wizard panels. But meanwhile, hope these latest changes work out well.
> >
> > John David N. Dionisio, PhD
> > Associate Professor, Computer Science
> > Loyola Marymount University
> >
> >
> >
> > On Aug 14, 2011, at 7:31 PM, Richard Brous wrote:
> >
> > > OK, using option 2 I have made changes to DatabaseProfile.java to allow for all species names to be included in the submit argument.
> > > But I'm stuck on how to change my StringBuilder object to a type that submit wants. Help please!!
> > >
> > > Submitted the above and some comment changes to SourceForge this evening.
> > >
> > > Richard
> > >
> > > On Fri, Aug 12, 2011 at 12:08 PM, Kam Dahlquist <kda...@lm...> wrote:
> > > Hi,
> > >
> > > I think we had better leave the info table with only one record
> > > (option 2). The species names can be separated by pipes " | " as
> > > they are in the Systems table where there are multiple species. To
> > > my knowledge, the only time GenMAPP needs to access the info table is
> > > for the "DisplayOrder" field, I don't know what would happen if there
> > > were multiple records there. I know that the spec for the table says
> > > that it should only be one record, but I don't know if it would crash
> > > if there were multiple records. To be on the safe side, I think we
> > > should just keep it to the one record.
> > >
> > > Best,
> > > Kam
> > >
> > > At 09:45 PM 8/10/2011, John David N. Dionisio wrote:
> > > >Greetings,
> > > >
> > > >I think we have to turn to Dr. Dahlquist's GenMAPP knowledge here to
> > > >get the definitive answer. I see two choices:
> > > >
> > > >- The Info table should have one record for each species that the
> > > >.gdb holds, in which case the change you need is to wrap that single
> > > >submit call inside a loop, so that submit is called once for each
> > > >chosen species.
> > > >
> > > >- The Info table should always have one record, and if the .gdb
> > > >holds multiple species, the "Species" column should be some
> > > >concatenation of multiple species names. In this case, you would
> > > >still call submit only once, but the value you send into the
> > > >"Species" column is some accumulation of all chosen species names.
> > > >
> > > >Admittedly I don't know which way is right (I assumed the former as
> > > >of our Tuesday meeting, but on further examination I'm no longer
> > > >quite so sure).
> > > >
> > > >For Kam --- what does GenMAPP expect to see in the Info table if the
> > > >opened .gdb contains multiple species?
> > > >
> > > >John David N. Dionisio, PhD
> > > >Associate Professor, Computer Science
> > > >Loyola Marymount University
> > > >
> > > >
> > > >On Aug 10, 2011, at 9:36 PM, Richard Brous wrote:
> > > >
> > > > > OK, continued to review ExportToGenMAPP and dug into the creation
> > > > of the first TableManager tmA on line 118.
> > > > >
> > > > > In reading through the method, my understanding is that it
> > > > creates a new TableManager based on the selectedDatabaseProfile
> > > > (which is UniProt).
> > > > >
> > > > > This is performed by the method getInfoTableManager() which then
> > > > calls method submit(String tableName, QueryType queryType,
> > > > String[][] columnNamesToValues);
> > > > >
> > > > > the code is as follows:
> > > > >
> > > > > tableManager.submit("Info", QueryType.insert, new String[][] { {
> > > > "Owner", owner }, { "Version", new
> > > > SimpleDateFormat("yyyyMMdd").format(version) }, { "MODSystem",
> > > > modSystem }, { "Species", speciesProfile.getSpeciesName() }, {
> > > > "Modify", new SimpleDateFormat("yyyyMMdd").format(modify) }, {
> > > > "DisplayOrder", displayOrder }, { "Notes", notes } });
> > > > >
> > > > >
> > > > > The modification of this line centers on { "Species",
> > > > speciesProfile.getSpeciesName() }, since it originally processed a
> > > > single species.
> > > > >
> > > > > So now I need to populate the arguments with the species
> > > > contained within selectedDatabaseprofile.selectedSpeciesProfiles.
> > > > >
> > > > > I think I'll start with the baseArgument up to MODSystem, then
> > > > append as many species as necessary, and then cap off the end with
> > > > the rest starting at Modify. (similar to your approach in
> > > > ExportGoData, populateUniprotGoTableFromSQL(char chosenAspect,
> > > > List<Integer> taxonIds) line 513
> > > > >
> > > > > Please let me know if this approach or analysis is off track.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Richard
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Aug 10, 2011 at 5:50 PM, Richard Brous <rbr...@gm...> wrote:
> > > > > Updated repository to include all Gene Ontology changes discussed
> > > > during our meeting yesterday.
> > > > >
> > > > > Digging into TableManager next.
> > > > >
> > > > > Richard
> > > > >
> > > > > On Fri, Aug 5, 2011 at 10:06 AM, Richard Brous <rbr...@gm...> wrote:
> > > > > whew... thanks for the detailed reply. I will digest this a bit
> > > > and get back to you with further questions.
> > > > >
> > > > > rb
> > > > >
> > > > > On Thu, Aug 4, 2011 at 11:18 PM, John David N. Dionisio
> > > > <do...@lm...> wrote:
> > > > > Greetings,
> > > > >
> > > > > Sorry for the delay. I wasn't able to walk through the relevant
> > > > code until this evening.
> > > > >
> > > > > As Kam said, GOA serves as the link between the UniProt and GO
> > > > IDs. It essentially determines which GO IDs get exported by using
> > > > GOA to see which GO IDs are associated with an exported UniProt
> > > > ID. The populateUniprotGoTableFromSQL, in its current form,
> > > > extracts the GO association records that match the given taxon ID
> > > > then exports, as UniProt-GO pairs, the GO and UniProt IDs
> > > > referenced within that GO association record. Processing that
> > > > follows this is then based on the GO IDs that got exported --- and
> > > > that's how the current code avoids exporting the entire list of GO terms.
> > > > >
> > > > > The operative query is on the second line of populateUniprotGoTableFromSQL:
> > > > >
> > > > > String uniProtAndGOIDSQL = "select db_object_id, go_id,
> > > > evidence_code, with_or_from from goa where db like '%UniProt%' and
> > > > taxon = 'taxon:" + taxon + "'";
> > > > >
> > > > > In plain English, this selects the GOA records whose database is
> > > > UniProt and whose taxon ID is the given taxon. An additional
> > > > condition is added for the "aspect" (All, Component, Function, or
> > > > Process) that is to be exported. This is another reduction filter,
> > > > to further shrink the number of exported GO terms and thus avoid
> > > > MAPPFinder issues later on.
> > > > >
> > > > > Given this, the proper expansion here is to change the taxon
> > > > predicate to a multiple predicate. That is, this method can be
> > > > changed to now accept a collection or array of taxon IDs, and the
> > > > base query should then be changed so that it accepts any taxon from
> > > > that collection. More or less, you want:
> > > > >
> > > > > private void populateUniprotGoTableFromSQL(char chosenAspect,
> > > > int[] taxons) throws SQLException {
> > > > >
> > > > > ...then, instead of the single string, you want to iterate
> > > > through the taxon IDs:
> > > > >
> > > > > StringBuilder baseQueryBuilder = new StringBuilder("select
> > > > db_object_id, go_id, evidence_code, with_or_from from goa where db
> > > > like '%UniProt%'");
> > > > > boolean first = true;
> > > > > for (int taxon: taxons) {
> > > > > baseQueryBuilder.append(first ? " and (" : " or ");
> > > > > baseQueryBuilder
> > > > > .append("taxon = 'taxon:")
> > > > > .append(taxon).append("'");
> > > > > first = false;
> > > > > }
> > > > > baseQueryBuilder.append(")");
> > > > >
> > > > > ...and so on. I just sort of rattled this off so there may be
> > > > little glitches, but anyway this is just to give you an overall idea.
> > > > >
> > > > > Put another way, no, you do not need to iterate this method for
> > > > each taxon ID. Instead, you can still call this method once, with
> > > > the multiplicity of taxon IDs emerging in terms of the actual
> > > > condition used for selecting the GO terms to be exported (based on
> > > > the available GOA records, which as you may recall are loaded from .goa files).
> > > > >
> > > > > As a side note, right here you have an opportunity for a little
> > > > sanity check regarding the content of the relational database: GO
> > > > terms will only be exported if GOA records for the desired taxon
> > > > IDs have been imported into the database. So, as a pre-flight
> > > > check, one can see if there are any GOA records at all for each
> > > > chosen taxon ID. If there are none, then the .goa file for that
> > > > species needs to be imported into the relational database.
> > > > >
> > > > > Hope this helps...
> > > > >
> > > > > John David N. Dionisio, PhD
> > > > > Associate Professor, Computer Science
> > > > > Loyola Marymount University
> > > > >
> > > > >
> > > > > On Aug 4, 2011, at 1:00 PM, Kam Dahlquist wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Dondi will have to chime in on this, but I think this is where
> > > > things are going to get tricky.
> > > > > >
> > > > > > The final gdb does not actually contain the entire GO, it gets
> > > > trimmed somehow based on the GO associations for a particular
> > > > species. This is because MAPPFinder cannot handle loading the
> > > > entire GO. Since there is some type of species-specific trimming
> > > > going on, it's quite possible that this will need to iterate.
> > > > > >
> > > > > > However, I don't have the foggiest idea of how this works, so
> > > > Dondi will have to chime in.
> > > > > >
> > > > > > Best,
> > > > > > Kam
> > > > > >
> > > > > > At 12:09 AM 8/4/2011, you wrote:
> > > > > >> Wednesday 8/3/11 progress:
> > > > > >>
> > > > > >> 1. After following the ExportPanel1.java ground zero code of:
> > > > databaseProfile.setSelectedSpeciesProfile( selectedProfile );
> > > > > >>
> > > > > >> I found the method in DatabaseProfile.java plus a getter method;
> > > > > >> SpeciesProfile setSelectedSpeciesProfile( speciesProfile ) and
> > > > SpeciesProfile getSelectedSpeciesProfile( speciesProfile )
> > > > > >>
> > > > > >> I created two new methods that each handle List<Object> of
> > > > SpeciesProfiles argument instead of a single SpeciesProfile;
> > > > setSelectedSpeciesProfiles and getSelectedSpeciesProfiles.
> > > > > >>
> > > > > >> This enabled the ExportPanel1 ground zero code to become:
> > > > databaseProfile.setSelectedSpeciesProfiles(selectedSpecies);
> > > > > >>
> > > > > >> 2. public static void export() on line 104 in ExportToGenMAPP.java
> > > > > >>
> > > > > >> On line 107 ExportGoData is instantiated which I found in
> > > > ExportGoData.java and calls a method: public void export(char
> > > > chosenAspect, int taxon).
> > > > > >>
> > > > > >> Within export, taxon id is required for another method:
> > > > private void populateGoTables(char chosenAspect, int taxon).
> > > > > >>
> > > > > >> Within populateGoTables, taxon id is required for another
> > > > method: private void populateUniprotGoTableFromSQL( char
> > > > chosenAspect, int taxon).
> > > > > >>
> > > > > >> But, if the export to GDB process starts off with exporting GO
> > > > data, doesn't it only need to do that once no matter how many
> > > > species are selected? As you probably realize, I'm leading towards
> > > > not having to iterate through this for each taxon id if possible.
> > > > > >>
> > > > > >> Also, how does the export actually work? How are GO ids and
> > > > UniProt ids related within the table?
> > > > > >>
> > > > > >> Thanks!
> > > > > >>
> > > > > >> Richard
> > > > > >>
> > > > > >>
> > > > > > <ATT00001..txt><ATT00002..txt>
> > > > >
> > > > >
> > > > >
> > > > ------------------------------------------------------------------------------
> > > > > BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
> > > > > The must-attend event for mobile developers. Connect with experts.
> > > > > Get tools for creating Super Apps. See the latest technologies.
> > > > > Sessions, hands-on labs, demos & much more. Register early & save!
> > > > > http://p.sf.net/sfu/rim-blackberry-1
> > > > > _______________________________________________
> > > > > xmlpipedb-developer mailing list
> > > > > xml...@li...
> > > > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> > > > >
> > > > >
> > > > >
> > > > > <ATT00001..txt><ATT00002..txt>
> > > >
> > > >
> > > >------------------------------------------------------------------------------
> > > >Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
> > > >user administration capabilities and model configuration. Take
> > > >the hassle out of deploying and managing Subversion and the
> > > >tools developers use with it.
> > > >http://p.sf.net/sfu/wandisco-dev2dev
> > > >_______________________________________________
> > > >xmlpipedb-developer mailing list
> > > >xml...@li...
> > > >https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> > >
> > >
> > > ------------------------------------------------------------------------------
> > > FREE DOWNLOAD - uberSVN with Social Coding for Subversion.
> > > Subversion made easy with a complete admin console. Easy
> > > to use, easy to manage, easy to install, easy to extend.
> > > Get a Free download of the new open ALM Subversion platform now.
> > > http://p.sf.net/sfu/wandisco-dev2dev
> > > _______________________________________________
> > > xmlpipedb-developer mailing list
> > > xml...@li...
> > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> > >
> > > <ATT00001..txt><ATT00002..txt>
> >
> >
> > ------------------------------------------------------------------------------
> > uberSVN's rich system and user administration capabilities and model
> > configuration take the hassle out of deploying and managing Subversion and
> > the tools developers use with it. Learn more about uberSVN and get a free
> > download at: http://p.sf.net/sfu/wandisco-dev2dev
> > _______________________________________________
> > xmlpipedb-developer mailing list
> > xml...@li...
> > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> >
> >
> >
> > <ATT00001..txt><ATT00002..txt>
>
>
> ------------------------------------------------------------------------------
> Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
> user administration capabilities and model configuration. Take
> the hassle out of deploying and managing Subversion and the
> tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2
> _______________________________________________
> xmlpipedb-developer mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>
>
> <ATT00001..txt><ATT00002..txt>
|
|
From: Richard B. <rbr...@gm...> - 2011-08-16 22:32:52
|
OK, so its pretty much the last line of getSystemsTableManager() which needs
to be modified to be aware of the collection of SpeciesProfile:
tableManager = speciesProfile.getSystemsTableManagerCustomizations(tableManager,
*this*);
*return* tableManager;
So I believe I need to iterate through each species of the collection before
returning the final tableManager.
I would enclose it in a for loop ( SpeciesProfile speciesProfile :
speciesProfiles ) {
}
This would seem to be the correct implementation to ensure that each species
customization would be applied.
Sound good??
Richard
On Tue, Aug 16, 2011 at 12:38 PM, Richard Brous <rbr...@gm...> wrote:
> gotcha, heading to system tables.
>
> gotcha.
>
> gotcha.
>
> Thanks for the speedy reply =D
>
> Richard
>
> On Tue, Aug 16, 2011 at 12:21 PM, John David N. Dionisio <do...@lm...>wrote:
>
>> Hi Rich,
>>
>> Yeah, relations is pretty complicated. I suggest you skip that and go to
>> just the system tables for now (i.e., the tables that hold the IDs
>> themselves). The relationship tables hold pairs of IDs (i.e., which one
>> from one system corresponds to the ID from another) and so is an additional
>> level of complexity, I think.
>>
>> Regarding your prior question about the Info table --- I'm not or'ing the
>> species names. I'm merely concatenating the names with pipes ("|") in
>> between, as Kam specified. While the final value may look like an "or," it
>> is ultimately just a string. The "|" could just have easily been a comma,
>> semicolon, or other separator.
>>
>> Your prior guess as to how that would look, with the multiple { "Species",
>> speciesName } pairs, would actually be equivalent to multiple values for the
>> single Species column, which does not fit the relational model. The Species
>> column can have only one value, and in this case it is a single string that
>> is the concatenation of all selected species names, separated by "|"
>> characters.
>>
>> Hope that clears things up...
>>
>> John David N. Dionisio, PhD
>> Associate Professor, Computer Science
>> Loyola Marymount University
>>
>>
>>
>> On Aug 16, 2011, at 12:11 PM, Richard Brous wrote:
>>
>> > OK, have moved onto getRelationsTableManager() ... this one seems pretty
>> complicated...
>> >
>> > Have reviewed the method and submethods called and have the following
>> questions:
>> >
>> > 1. What are the RelationshipTables that are stored in
>> relationshipTables?
>> > aren't they species dependent? Can't seem to find where they were
>> created?
>> > If not, then:
>> >
>> > 2. How should the if/else conditional be handled?
>> >
>> > if
>> >
>> >
>> (speciesProfile.getSpeciesSpecificSystemTables().containsKey(stp.systemTable1)
>> |
>> >
>> >
>> speciesProfile.getSpeciesSpecificSystemTables().containsKey(stp.systemTable2))
>> {
>> > tableManager =
>> >
>> > speciesProfile.getRelationsTableManagerCustomizations(stp.systemTable1,
>> > stp.
>> >
>> > systemTable2, templateDefinedSystemToSystemCode, tableManager);
>> > This obviously needs to be made aware of Lists of SpeciesProfiles... to
>> build the correct tableManager, do I need this conditional to run through
>> every species or should we enclose all the code (if and else) within a loop
>> through each species?
>> >
>> > Thanks.
>> >
>> > Richard
>> >
>> >
>> > On Tue, Aug 16, 2011 at 11:02 AM, Richard Brous <rbr...@gm...>
>> wrote:
>> > Sorry I typed and sent that out too quickly before getting my thoughts
>> completely together...
>> >
>> > First off i thought that the array[][] portion of the submit needed to
>> be in the following format: { "Species", speciesName1 }, { "Species",
>> speciesName2 }, { "Species", speciesName3 },...
>> > but I see you specified it as: { "Species", speciesName1 | speciesName2
>> | speciesName3 }, ... OR'ing each of the species names?
>> >
>> > I see what you did regarding the date object... yes no reason to
>> recreate a second object...
>> >
>> > As mentioned previously I will continue through the next methods,
>> submitting (well thought out) changes and emailing out updates...
>> >
>> > Richard
>> > On Tue, Aug 16, 2011 at 10:36 AM, Richard Brous <rbr...@gm...>
>> wrote:
>> > Took a look at your code and realized what I had done wrong. I should
>> have just broken out the loop as the second argument of the 3 in the submit
>> method. That was a rookie move on my part.
>> >
>> > I am moving on to the next method and will keep my mind on the syntactic
>> solution that I'm not thinking through prior to moving forward with an
>> implementation.
>> >
>> > Richard
>> >
>> > On Mon, Aug 15, 2011 at 1:30 AM, John David N. Dionisio <do...@lm...>
>> wrote:
>> > OK, everything is committed. I got up to the tweaks on the second panel
>> in the wizard (Save As/GO Aspects), as well as the query change for
>> exporting any combination of C, F, or P.
>> >
>> > I still have to do the GO OBO format check, plus the UI work on the
>> remaining two wizard panels. But meanwhile, hope these latest changes work
>> out well.
>> >
>> > John David N. Dionisio, PhD
>> > Associate Professor, Computer Science
>> > Loyola Marymount University
>> >
>> >
>> >
>> > On Aug 14, 2011, at 7:31 PM, Richard Brous wrote:
>> >
>> > > OK, using option 2 I have made changes to DatabaseProfile.java to
>> allow for all species names to be included in the submit argument.
>> > > But I'm stuck on how to change my StringBuilder object to a type that
>> submit wants. Help please!!
>> > >
>> > > Submitted the above and some comment changes to SourceForge this
>> evening.
>> > >
>> > > Richard
>> > >
>> > > On Fri, Aug 12, 2011 at 12:08 PM, Kam Dahlquist <kda...@lm...>
>> wrote:
>> > > Hi,
>> > >
>> > > I think we had better leave the info table with only one record
>> > > (option 2). The species names can be separated by pipes " | " as
>> > > they are in the Systems table where there are multiple species. To
>> > > my knowledge, the only time GenMAPP needs to access the info table is
>> > > for the "DisplayOrder" field, I don't know what would happen if there
>> > > were multiple records there. I know that the spec for the table says
>> > > that it should only be one record, but I don't know if it would crash
>> > > if there were multiple records. To be on the safe side, I think we
>> > > should just keep it to the one record.
>> > >
>> > > Best,
>> > > Kam
>> > >
>> > > At 09:45 PM 8/10/2011, John David N. Dionisio wrote:
>> > > >Greetings,
>> > > >
>> > > >I think we have to turn to Dr. Dahlquist's GenMAPP knowledge here to
>> > > >get the definitive answer. I see two choices:
>> > > >
>> > > >- The Info table should have one record for each species that the
>> > > >.gdb holds, in which case the change you need is to wrap that single
>> > > >submit call inside a loop, so that submit is called once for each
>> > > >chosen species.
>> > > >
>> > > >- The Info table should always have one record, and if the .gdb
>> > > >holds multiple species, the "Species" column should be some
>> > > >concatenation of multiple species names. In this case, you would
>> > > >still call submit only once, but the value you send into the
>> > > >"Species" column is some accumulation of all chosen species names.
>> > > >
>> > > >Admittedly I don't know which way is right (I assumed the former as
>> > > >of our Tuesday meeting, but on further examination I'm no longer
>> > > >quite so sure).
>> > > >
>> > > >For Kam --- what does GenMAPP expect to see in the Info table if the
>> > > >opened .gdb contains multiple species?
>> > > >
>> > > >John David N. Dionisio, PhD
>> > > >Associate Professor, Computer Science
>> > > >Loyola Marymount University
>> > > >
>> > > >
>> > > >On Aug 10, 2011, at 9:36 PM, Richard Brous wrote:
>> > > >
>> > > > > OK, continued to review ExportToGenMAPP and dug into the creation
>> > > > of the first TableManager tmA on line 118.
>> > > > >
>> > > > > In reading through the method, my understanding is that it
>> > > > creates a new TableManager based on the selectedDatabaseProfile
>> > > > (which is UniProt).
>> > > > >
>> > > > > This is performed by the method getInfoTableManager() which then
>> > > > calls method submit(String tableName, QueryType queryType,
>> > > > String[][] columnNamesToValues);
>> > > > >
>> > > > > the code is as follows:
>> > > > >
>> > > > > tableManager.submit("Info", QueryType.insert, new String[][] { {
>> > > > "Owner", owner }, { "Version", new
>> > > > SimpleDateFormat("yyyyMMdd").format(version) }, { "MODSystem",
>> > > > modSystem }, { "Species", speciesProfile.getSpeciesName() }, {
>> > > > "Modify", new SimpleDateFormat("yyyyMMdd").format(modify) }, {
>> > > > "DisplayOrder", displayOrder }, { "Notes", notes } });
>> > > > >
>> > > > >
>> > > > > The modification of this line centers on { "Species",
>> > > > speciesProfile.getSpeciesName() }, since it originally processed a
>> > > > single species.
>> > > > >
>> > > > > So now I need to populate the arguments with the species
>> > > > contained within selectedDatabaseprofile.selectedSpeciesProfiles.
>> > > > >
>> > > > > I think I'll start with the baseArgument up to MODSystem, then
>> > > > append as many species as necessary, and then cap off the end with
>> > > > the rest starting at Modify. (similar to your approach in
>> > > > ExportGoData, populateUniprotGoTableFromSQL(char chosenAspect,
>> > > > List<Integer> taxonIds) line 513
>> > > > >
>> > > > > Please let me know if this approach or analysis is off track.
>> > > > >
>> > > > > Thanks!
>> > > > >
>> > > > > Richard
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Wed, Aug 10, 2011 at 5:50 PM, Richard Brous <
>> rbr...@gm...> wrote:
>> > > > > Updated repository to include all Gene Ontology changes discussed
>> > > > during our meeting yesterday.
>> > > > >
>> > > > > Digging into TableManager next.
>> > > > >
>> > > > > Richard
>> > > > >
>> > > > > On Fri, Aug 5, 2011 at 10:06 AM, Richard Brous <
>> rbr...@gm...> wrote:
>> > > > > whew... thanks for the detailed reply. I will digest this a bit
>> > > > and get back to you with further questions.
>> > > > >
>> > > > > rb
>> > > > >
>> > > > > On Thu, Aug 4, 2011 at 11:18 PM, John David N. Dionisio
>> > > > <do...@lm...> wrote:
>> > > > > Greetings,
>> > > > >
>> > > > > Sorry for the delay. I wasn't able to walk through the relevant
>> > > > code until this evening.
>> > > > >
>> > > > > As Kam said, GOA serves as the link between the UniProt and GO
>> > > > IDs. It essentially determines which GO IDs get exported by using
>> > > > GOA to see which GO IDs are associated with an exported UniProt
>> > > > ID. The populateUniprotGoTableFromSQL, in its current form,
>> > > > extracts the GO association records that match the given taxon ID
>> > > > then exports, as UniProt-GO pairs, the GO and UniProt IDs
>> > > > referenced within that GO association record. Processing that
>> > > > follows this is then based on the GO IDs that got exported --- and
>> > > > that's how the current code avoids exporting the entire list of GO
>> terms.
>> > > > >
>> > > > > The operative query is on the second line of
>> populateUniprotGoTableFromSQL:
>> > > > >
>> > > > > String uniProtAndGOIDSQL = "select db_object_id, go_id,
>> > > > evidence_code, with_or_from from goa where db like '%UniProt%' and
>> > > > taxon = 'taxon:" + taxon + "'";
>> > > > >
>> > > > > In plain English, this selects the GOA records whose database is
>> > > > UniProt and whose taxon ID is the given taxon. An additional
>> > > > condition is added for the "aspect" (All, Component, Function, or
>> > > > Process) that is to be exported. This is another reduction filter,
>> > > > to further shrink the number of exported GO terms and thus avoid
>> > > > MAPPFinder issues later on.
>> > > > >
>> > > > > Given this, the proper expansion here is to change the taxon
>> > > > predicate to a multiple predicate. That is, this method can be
>> > > > changed to now accept a collection or array of taxon IDs, and the
>> > > > base query should then be changed so that it accepts any taxon from
>> > > > that collection. More or less, you want:
>> > > > >
>> > > > > private void populateUniprotGoTableFromSQL(char chosenAspect,
>> > > > int[] taxons) throws SQLException {
>> > > > >
>> > > > > ...then, instead of the single string, you want to iterate
>> > > > through the taxon IDs:
>> > > > >
>> > > > > StringBuilder baseQueryBuilder = new StringBuilder("select
>> > > > db_object_id, go_id, evidence_code, with_or_from from goa where db
>> > > > like '%UniProt%'");
>> > > > > boolean first = true;
>> > > > > for (int taxon: taxons) {
>> > > > > baseQueryBuilder.append(first ? " and (" : " or ");
>> > > > > baseQueryBuilder
>> > > > > .append("taxon = 'taxon:")
>> > > > > .append(taxon).append("'");
>> > > > > first = false;
>> > > > > }
>> > > > > baseQueryBuilder.append(")");
>> > > > >
>> > > > > ...and so on. I just sort of rattled this off so there may be
>> > > > little glitches, but anyway this is just to give you an overall
>> idea.
>> > > > >
>> > > > > Put another way, no, you do not need to iterate this method for
>> > > > each taxon ID. Instead, you can still call this method once, with
>> > > > the multiplicity of taxon IDs emerging in terms of the actual
>> > > > condition used for selecting the GO terms to be exported (based on
>> > > > the available GOA records, which as you may recall are loaded from
>> .goa files).
>> > > > >
>> > > > > As a side note, right here you have an opportunity for a little
>> > > > sanity check regarding the content of the relational database: GO
>> > > > terms will only be exported if GOA records for the desired taxon
>> > > > IDs have been imported into the database. So, as a pre-flight
>> > > > check, one can see if there are any GOA records at all for each
>> > > > chosen taxon ID. If there are none, then the .goa file for that
>> > > > species needs to be imported into the relational database.
>> > > > >
>> > > > > Hope this helps...
>> > > > >
>> > > > > John David N. Dionisio, PhD
>> > > > > Associate Professor, Computer Science
>> > > > > Loyola Marymount University
>> > > > >
>> > > > >
>> > > > > On Aug 4, 2011, at 1:00 PM, Kam Dahlquist wrote:
>> > > > >
>> > > > > > Hi,
>> > > > > >
>> > > > > > Dondi will have to chime in on this, but I think this is where
>> > > > things are going to get tricky.
>> > > > > >
>> > > > > > The final gdb does not actually contain the entire GO, it gets
>> > > > trimmed somehow based on the GO associations for a particular
>> > > > species. This is because MAPPFinder cannot handle loading the
>> > > > entire GO. Since there is some type of species-specific trimming
>> > > > going on, it's quite possible that this will need to iterate.
>> > > > > >
>> > > > > > However, I don't have the foggiest idea of how this works, so
>> > > > Dondi will have to chime in.
>> > > > > >
>> > > > > > Best,
>> > > > > > Kam
>> > > > > >
>> > > > > > At 12:09 AM 8/4/2011, you wrote:
>> > > > > >> Wednesday 8/3/11 progress:
>> > > > > >>
>> > > > > >> 1. After following the ExportPanel1.java ground zero code of:
>> > > > databaseProfile.setSelectedSpeciesProfile( selectedProfile );
>> > > > > >>
>> > > > > >> I found the method in DatabaseProfile.java plus a getter
>> method;
>> > > > > >> SpeciesProfile setSelectedSpeciesProfile( speciesProfile ) and
>> > > > SpeciesProfile getSelectedSpeciesProfile( speciesProfile )
>> > > > > >>
>> > > > > >> I created two new methods that each handle List<Object> of
>> > > > SpeciesProfiles argument instead of a single SpeciesProfile;
>> > > > setSelectedSpeciesProfiles and getSelectedSpeciesProfiles.
>> > > > > >>
>> > > > > >> This enabled the ExportPanel1 ground zero code to become:
>> > > > databaseProfile.setSelectedSpeciesProfiles(selectedSpecies);
>> > > > > >>
>> > > > > >> 2. public static void export() on line 104 in
>> ExportToGenMAPP.java
>> > > > > >>
>> > > > > >> On line 107 ExportGoData is instantiated which I found in
>> > > > ExportGoData.java and calls a method: public void export(char
>> > > > chosenAspect, int taxon).
>> > > > > >>
>> > > > > >> Within export, taxon id is required for another method:
>> > > > private void populateGoTables(char chosenAspect, int taxon).
>> > > > > >>
>> > > > > >> Within populateGoTables, taxon id is required for another
>> > > > method: private void populateUniprotGoTableFromSQL( char
>> > > > chosenAspect, int taxon).
>> > > > > >>
>> > > > > >> But, if the export to GDB process starts off with exporting GO
>> > > > data, doesn't it only need to do that once no matter how many
>> > > > species are selected? As you probably realize, I'm leading towards
>> > > > not having to iterate through this for each taxon id if possible.
>> > > > > >>
>> > > > > >> Also, how does the export actually work? How are GO ids and
>> > > > UniProt ids related within the table?
>> > > > > >>
>> > > > > >> Thanks!
>> > > > > >>
>> > > > > >> Richard
>> > > > > >>
>> > > > > >>
>> > > > > > <ATT00001..txt><ATT00002..txt>
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> ------------------------------------------------------------------------------
>> > > > > BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
>> > > > > The must-attend event for mobile developers. Connect with experts.
>> > > > > Get tools for creating Super Apps. See the latest technologies.
>> > > > > Sessions, hands-on labs, demos & much more. Register early & save!
>> > > > > http://p.sf.net/sfu/rim-blackberry-1
>> > > > > _______________________________________________
>> > > > > xmlpipedb-developer mailing list
>> > > > > xml...@li...
>> > > > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>> > > > >
>> > > > >
>> > > > >
>> > > > > <ATT00001..txt><ATT00002..txt>
>> > > >
>> > > >
>> > >
>> >------------------------------------------------------------------------------
>> > > >Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
>> > > >user administration capabilities and model configuration. Take
>> > > >the hassle out of deploying and managing Subversion and the
>> > > >tools developers use with it.
>> > > >http://p.sf.net/sfu/wandisco-dev2dev
>> > > >_______________________________________________
>> > > >xmlpipedb-developer mailing list
>> > > >xml...@li...
>> > > >https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>> > >
>> > >
>> > >
>> ------------------------------------------------------------------------------
>> > > FREE DOWNLOAD - uberSVN with Social Coding for Subversion.
>> > > Subversion made easy with a complete admin console. Easy
>> > > to use, easy to manage, easy to install, easy to extend.
>> > > Get a Free download of the new open ALM Subversion platform now.
>> > > http://p.sf.net/sfu/wandisco-dev2dev
>> > > _______________________________________________
>> > > xmlpipedb-developer mailing list
>> > > xml...@li...
>> > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>> > >
>> > > <ATT00001..txt><ATT00002..txt>
>> >
>> >
>> >
>> ------------------------------------------------------------------------------
>> > uberSVN's rich system and user administration capabilities and model
>> > configuration take the hassle out of deploying and managing Subversion
>> and
>> > the tools developers use with it. Learn more about uberSVN and get a
>> free
>> > download at: http://p.sf.net/sfu/wandisco-dev2dev
>> > _______________________________________________
>> > xmlpipedb-developer mailing list
>> > xml...@li...
>> > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>> >
>> >
>> >
>> > <ATT00001..txt><ATT00002..txt>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
>> user administration capabilities and model configuration. Take
>> the hassle out of deploying and managing Subversion and the
>> tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2
>> _______________________________________________
>> xmlpipedb-developer mailing list
>> xml...@li...
>> https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>>
>
>
|
|
From: Richard B. <rbr...@gm...> - 2011-08-16 19:38:41
|
gotcha, heading to system tables.
gotcha.
gotcha.
Thanks for the speedy reply =D
Richard
On Tue, Aug 16, 2011 at 12:21 PM, John David N. Dionisio <do...@lm...>wrote:
> Hi Rich,
>
> Yeah, relations is pretty complicated. I suggest you skip that and go to
> just the system tables for now (i.e., the tables that hold the IDs
> themselves). The relationship tables hold pairs of IDs (i.e., which one
> from one system corresponds to the ID from another) and so is an additional
> level of complexity, I think.
>
> Regarding your prior question about the Info table --- I'm not or'ing the
> species names. I'm merely concatenating the names with pipes ("|") in
> between, as Kam specified. While the final value may look like an "or," it
> is ultimately just a string. The "|" could just have easily been a comma,
> semicolon, or other separator.
>
> Your prior guess as to how that would look, with the multiple { "Species",
> speciesName } pairs, would actually be equivalent to multiple values for the
> single Species column, which does not fit the relational model. The Species
> column can have only one value, and in this case it is a single string that
> is the concatenation of all selected species names, separated by "|"
> characters.
>
> Hope that clears things up...
>
> John David N. Dionisio, PhD
> Associate Professor, Computer Science
> Loyola Marymount University
>
>
>
> On Aug 16, 2011, at 12:11 PM, Richard Brous wrote:
>
> > OK, have moved onto getRelationsTableManager() ... this one seems pretty
> complicated...
> >
> > Have reviewed the method and submethods called and have the following
> questions:
> >
> > 1. What are the RelationshipTables that are stored in relationshipTables?
> > aren't they species dependent? Can't seem to find where they were
> created?
> > If not, then:
> >
> > 2. How should the if/else conditional be handled?
> >
> > if
> >
> >
> (speciesProfile.getSpeciesSpecificSystemTables().containsKey(stp.systemTable1)
> |
> >
> >
> speciesProfile.getSpeciesSpecificSystemTables().containsKey(stp.systemTable2))
> {
> > tableManager =
> >
> > speciesProfile.getRelationsTableManagerCustomizations(stp.systemTable1,
> > stp.
> >
> > systemTable2, templateDefinedSystemToSystemCode, tableManager);
> > This obviously needs to be made aware of Lists of SpeciesProfiles... to
> build the correct tableManager, do I need this conditional to run through
> every species or should we enclose all the code (if and else) within a loop
> through each species?
> >
> > Thanks.
> >
> > Richard
> >
> >
> > On Tue, Aug 16, 2011 at 11:02 AM, Richard Brous <rbr...@gm...>
> wrote:
> > Sorry I typed and sent that out too quickly before getting my thoughts
> completely together...
> >
> > First off i thought that the array[][] portion of the submit needed to be
> in the following format: { "Species", speciesName1 }, { "Species",
> speciesName2 }, { "Species", speciesName3 },...
> > but I see you specified it as: { "Species", speciesName1 | speciesName2 |
> speciesName3 }, ... OR'ing each of the species names?
> >
> > I see what you did regarding the date object... yes no reason to recreate
> a second object...
> >
> > As mentioned previously I will continue through the next methods,
> submitting (well thought out) changes and emailing out updates...
> >
> > Richard
> > On Tue, Aug 16, 2011 at 10:36 AM, Richard Brous <rbr...@gm...>
> wrote:
> > Took a look at your code and realized what I had done wrong. I should
> have just broken out the loop as the second argument of the 3 in the submit
> method. That was a rookie move on my part.
> >
> > I am moving on to the next method and will keep my mind on the syntactic
> solution that I'm not thinking through prior to moving forward with an
> implementation.
> >
> > Richard
> >
> > On Mon, Aug 15, 2011 at 1:30 AM, John David N. Dionisio <do...@lm...>
> wrote:
> > OK, everything is committed. I got up to the tweaks on the second panel
> in the wizard (Save As/GO Aspects), as well as the query change for
> exporting any combination of C, F, or P.
> >
> > I still have to do the GO OBO format check, plus the UI work on the
> remaining two wizard panels. But meanwhile, hope these latest changes work
> out well.
> >
> > John David N. Dionisio, PhD
> > Associate Professor, Computer Science
> > Loyola Marymount University
> >
> >
> >
> > On Aug 14, 2011, at 7:31 PM, Richard Brous wrote:
> >
> > > OK, using option 2 I have made changes to DatabaseProfile.java to allow
> for all species names to be included in the submit argument.
> > > But I'm stuck on how to change my StringBuilder object to a type that
> submit wants. Help please!!
> > >
> > > Submitted the above and some comment changes to SourceForge this
> evening.
> > >
> > > Richard
> > >
> > > On Fri, Aug 12, 2011 at 12:08 PM, Kam Dahlquist <kda...@lm...>
> wrote:
> > > Hi,
> > >
> > > I think we had better leave the info table with only one record
> > > (option 2). The species names can be separated by pipes " | " as
> > > they are in the Systems table where there are multiple species. To
> > > my knowledge, the only time GenMAPP needs to access the info table is
> > > for the "DisplayOrder" field, I don't know what would happen if there
> > > were multiple records there. I know that the spec for the table says
> > > that it should only be one record, but I don't know if it would crash
> > > if there were multiple records. To be on the safe side, I think we
> > > should just keep it to the one record.
> > >
> > > Best,
> > > Kam
> > >
> > > At 09:45 PM 8/10/2011, John David N. Dionisio wrote:
> > > >Greetings,
> > > >
> > > >I think we have to turn to Dr. Dahlquist's GenMAPP knowledge here to
> > > >get the definitive answer. I see two choices:
> > > >
> > > >- The Info table should have one record for each species that the
> > > >.gdb holds, in which case the change you need is to wrap that single
> > > >submit call inside a loop, so that submit is called once for each
> > > >chosen species.
> > > >
> > > >- The Info table should always have one record, and if the .gdb
> > > >holds multiple species, the "Species" column should be some
> > > >concatenation of multiple species names. In this case, you would
> > > >still call submit only once, but the value you send into the
> > > >"Species" column is some accumulation of all chosen species names.
> > > >
> > > >Admittedly I don't know which way is right (I assumed the former as
> > > >of our Tuesday meeting, but on further examination I'm no longer
> > > >quite so sure).
> > > >
> > > >For Kam --- what does GenMAPP expect to see in the Info table if the
> > > >opened .gdb contains multiple species?
> > > >
> > > >John David N. Dionisio, PhD
> > > >Associate Professor, Computer Science
> > > >Loyola Marymount University
> > > >
> > > >
> > > >On Aug 10, 2011, at 9:36 PM, Richard Brous wrote:
> > > >
> > > > > OK, continued to review ExportToGenMAPP and dug into the creation
> > > > of the first TableManager tmA on line 118.
> > > > >
> > > > > In reading through the method, my understanding is that it
> > > > creates a new TableManager based on the selectedDatabaseProfile
> > > > (which is UniProt).
> > > > >
> > > > > This is performed by the method getInfoTableManager() which then
> > > > calls method submit(String tableName, QueryType queryType,
> > > > String[][] columnNamesToValues);
> > > > >
> > > > > the code is as follows:
> > > > >
> > > > > tableManager.submit("Info", QueryType.insert, new String[][] { {
> > > > "Owner", owner }, { "Version", new
> > > > SimpleDateFormat("yyyyMMdd").format(version) }, { "MODSystem",
> > > > modSystem }, { "Species", speciesProfile.getSpeciesName() }, {
> > > > "Modify", new SimpleDateFormat("yyyyMMdd").format(modify) }, {
> > > > "DisplayOrder", displayOrder }, { "Notes", notes } });
> > > > >
> > > > >
> > > > > The modification of this line centers on { "Species",
> > > > speciesProfile.getSpeciesName() }, since it originally processed a
> > > > single species.
> > > > >
> > > > > So now I need to populate the arguments with the species
> > > > contained within selectedDatabaseprofile.selectedSpeciesProfiles.
> > > > >
> > > > > I think I'll start with the baseArgument up to MODSystem, then
> > > > append as many species as necessary, and then cap off the end with
> > > > the rest starting at Modify. (similar to your approach in
> > > > ExportGoData, populateUniprotGoTableFromSQL(char chosenAspect,
> > > > List<Integer> taxonIds) line 513
> > > > >
> > > > > Please let me know if this approach or analysis is off track.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Richard
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Aug 10, 2011 at 5:50 PM, Richard Brous <rbr...@gm...>
> wrote:
> > > > > Updated repository to include all Gene Ontology changes discussed
> > > > during our meeting yesterday.
> > > > >
> > > > > Digging into TableManager next.
> > > > >
> > > > > Richard
> > > > >
> > > > > On Fri, Aug 5, 2011 at 10:06 AM, Richard Brous <rbr...@gm...>
> wrote:
> > > > > whew... thanks for the detailed reply. I will digest this a bit
> > > > and get back to you with further questions.
> > > > >
> > > > > rb
> > > > >
> > > > > On Thu, Aug 4, 2011 at 11:18 PM, John David N. Dionisio
> > > > <do...@lm...> wrote:
> > > > > Greetings,
> > > > >
> > > > > Sorry for the delay. I wasn't able to walk through the relevant
> > > > code until this evening.
> > > > >
> > > > > As Kam said, GOA serves as the link between the UniProt and GO
> > > > IDs. It essentially determines which GO IDs get exported by using
> > > > GOA to see which GO IDs are associated with an exported UniProt
> > > > ID. The populateUniprotGoTableFromSQL, in its current form,
> > > > extracts the GO association records that match the given taxon ID
> > > > then exports, as UniProt-GO pairs, the GO and UniProt IDs
> > > > referenced within that GO association record. Processing that
> > > > follows this is then based on the GO IDs that got exported --- and
> > > > that's how the current code avoids exporting the entire list of GO
> terms.
> > > > >
> > > > > The operative query is on the second line of
> populateUniprotGoTableFromSQL:
> > > > >
> > > > > String uniProtAndGOIDSQL = "select db_object_id, go_id,
> > > > evidence_code, with_or_from from goa where db like '%UniProt%' and
> > > > taxon = 'taxon:" + taxon + "'";
> > > > >
> > > > > In plain English, this selects the GOA records whose database is
> > > > UniProt and whose taxon ID is the given taxon. An additional
> > > > condition is added for the "aspect" (All, Component, Function, or
> > > > Process) that is to be exported. This is another reduction filter,
> > > > to further shrink the number of exported GO terms and thus avoid
> > > > MAPPFinder issues later on.
> > > > >
> > > > > Given this, the proper expansion here is to change the taxon
> > > > predicate to a multiple predicate. That is, this method can be
> > > > changed to now accept a collection or array of taxon IDs, and the
> > > > base query should then be changed so that it accepts any taxon from
> > > > that collection. More or less, you want:
> > > > >
> > > > > private void populateUniprotGoTableFromSQL(char chosenAspect,
> > > > int[] taxons) throws SQLException {
> > > > >
> > > > > ...then, instead of the single string, you want to iterate
> > > > through the taxon IDs:
> > > > >
> > > > > StringBuilder baseQueryBuilder = new StringBuilder("select
> > > > db_object_id, go_id, evidence_code, with_or_from from goa where db
> > > > like '%UniProt%'");
> > > > > boolean first = true;
> > > > > for (int taxon: taxons) {
> > > > > baseQueryBuilder.append(first ? " and (" : " or ");
> > > > > baseQueryBuilder
> > > > > .append("taxon = 'taxon:")
> > > > > .append(taxon).append("'");
> > > > > first = false;
> > > > > }
> > > > > baseQueryBuilder.append(")");
> > > > >
> > > > > ...and so on. I just sort of rattled this off so there may be
> > > > little glitches, but anyway this is just to give you an overall idea.
> > > > >
> > > > > Put another way, no, you do not need to iterate this method for
> > > > each taxon ID. Instead, you can still call this method once, with
> > > > the multiplicity of taxon IDs emerging in terms of the actual
> > > > condition used for selecting the GO terms to be exported (based on
> > > > the available GOA records, which as you may recall are loaded from
> .goa files).
> > > > >
> > > > > As a side note, right here you have an opportunity for a little
> > > > sanity check regarding the content of the relational database: GO
> > > > terms will only be exported if GOA records for the desired taxon
> > > > IDs have been imported into the database. So, as a pre-flight
> > > > check, one can see if there are any GOA records at all for each
> > > > chosen taxon ID. If there are none, then the .goa file for that
> > > > species needs to be imported into the relational database.
> > > > >
> > > > > Hope this helps...
> > > > >
> > > > > John David N. Dionisio, PhD
> > > > > Associate Professor, Computer Science
> > > > > Loyola Marymount University
> > > > >
> > > > >
> > > > > On Aug 4, 2011, at 1:00 PM, Kam Dahlquist wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Dondi will have to chime in on this, but I think this is where
> > > > things are going to get tricky.
> > > > > >
> > > > > > The final gdb does not actually contain the entire GO, it gets
> > > > trimmed somehow based on the GO associations for a particular
> > > > species. This is because MAPPFinder cannot handle loading the
> > > > entire GO. Since there is some type of species-specific trimming
> > > > going on, it's quite possible that this will need to iterate.
> > > > > >
> > > > > > However, I don't have the foggiest idea of how this works, so
> > > > Dondi will have to chime in.
> > > > > >
> > > > > > Best,
> > > > > > Kam
> > > > > >
> > > > > > At 12:09 AM 8/4/2011, you wrote:
> > > > > >> Wednesday 8/3/11 progress:
> > > > > >>
> > > > > >> 1. After following the ExportPanel1.java ground zero code of:
> > > > databaseProfile.setSelectedSpeciesProfile( selectedProfile );
> > > > > >>
> > > > > >> I found the method in DatabaseProfile.java plus a getter method;
> > > > > >> SpeciesProfile setSelectedSpeciesProfile( speciesProfile ) and
> > > > SpeciesProfile getSelectedSpeciesProfile( speciesProfile )
> > > > > >>
> > > > > >> I created two new methods that each handle List<Object> of
> > > > SpeciesProfiles argument instead of a single SpeciesProfile;
> > > > setSelectedSpeciesProfiles and getSelectedSpeciesProfiles.
> > > > > >>
> > > > > >> This enabled the ExportPanel1 ground zero code to become:
> > > > databaseProfile.setSelectedSpeciesProfiles(selectedSpecies);
> > > > > >>
> > > > > >> 2. public static void export() on line 104 in
> ExportToGenMAPP.java
> > > > > >>
> > > > > >> On line 107 ExportGoData is instantiated which I found in
> > > > ExportGoData.java and calls a method: public void export(char
> > > > chosenAspect, int taxon).
> > > > > >>
> > > > > >> Within export, taxon id is required for another method:
> > > > private void populateGoTables(char chosenAspect, int taxon).
> > > > > >>
> > > > > >> Within populateGoTables, taxon id is required for another
> > > > method: private void populateUniprotGoTableFromSQL( char
> > > > chosenAspect, int taxon).
> > > > > >>
> > > > > >> But, if the export to GDB process starts off with exporting GO
> > > > data, doesn't it only need to do that once no matter how many
> > > > species are selected? As you probably realize, I'm leading towards
> > > > not having to iterate through this for each taxon id if possible.
> > > > > >>
> > > > > >> Also, how does the export actually work? How are GO ids and
> > > > UniProt ids related within the table?
> > > > > >>
> > > > > >> Thanks!
> > > > > >>
> > > > > >> Richard
> > > > > >>
> > > > > >>
> > > > > > <ATT00001..txt><ATT00002..txt>
> > > > >
> > > > >
> > > > >
> > > >
> ------------------------------------------------------------------------------
> > > > > BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
> > > > > The must-attend event for mobile developers. Connect with experts.
> > > > > Get tools for creating Super Apps. See the latest technologies.
> > > > > Sessions, hands-on labs, demos & much more. Register early & save!
> > > > > http://p.sf.net/sfu/rim-blackberry-1
> > > > > _______________________________________________
> > > > > xmlpipedb-developer mailing list
> > > > > xml...@li...
> > > > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> > > > >
> > > > >
> > > > >
> > > > > <ATT00001..txt><ATT00002..txt>
> > > >
> > > >
> > >
> >------------------------------------------------------------------------------
> > > >Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
> > > >user administration capabilities and model configuration. Take
> > > >the hassle out of deploying and managing Subversion and the
> > > >tools developers use with it.
> > > >http://p.sf.net/sfu/wandisco-dev2dev
> > > >_______________________________________________
> > > >xmlpipedb-developer mailing list
> > > >xml...@li...
> > > >https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> > >
> > >
> > >
> ------------------------------------------------------------------------------
> > > FREE DOWNLOAD - uberSVN with Social Coding for Subversion.
> > > Subversion made easy with a complete admin console. Easy
> > > to use, easy to manage, easy to install, easy to extend.
> > > Get a Free download of the new open ALM Subversion platform now.
> > > http://p.sf.net/sfu/wandisco-dev2dev
> > > _______________________________________________
> > > xmlpipedb-developer mailing list
> > > xml...@li...
> > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> > >
> > > <ATT00001..txt><ATT00002..txt>
> >
> >
> >
> ------------------------------------------------------------------------------
> > uberSVN's rich system and user administration capabilities and model
> > configuration take the hassle out of deploying and managing Subversion
> and
> > the tools developers use with it. Learn more about uberSVN and get a free
> > download at: http://p.sf.net/sfu/wandisco-dev2dev
> > _______________________________________________
> > xmlpipedb-developer mailing list
> > xml...@li...
> > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> >
> >
> >
> > <ATT00001..txt><ATT00002..txt>
>
>
>
> ------------------------------------------------------------------------------
> Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
> user administration capabilities and model configuration. Take
> the hassle out of deploying and managing Subversion and the
> tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2
> _______________________________________________
> xmlpipedb-developer mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>
|
|
From: John D. N. D. <do...@lm...> - 2011-08-16 19:21:43
|
Hi Rich,
Yeah, relations is pretty complicated. I suggest you skip that and go to just the system tables for now (i.e., the tables that hold the IDs themselves). The relationship tables hold pairs of IDs (i.e., which one from one system corresponds to the ID from another) and so is an additional level of complexity, I think.
Regarding your prior question about the Info table --- I'm not or'ing the species names. I'm merely concatenating the names with pipes ("|") in between, as Kam specified. While the final value may look like an "or," it is ultimately just a string. The "|" could just have easily been a comma, semicolon, or other separator.
Your prior guess as to how that would look, with the multiple { "Species", speciesName } pairs, would actually be equivalent to multiple values for the single Species column, which does not fit the relational model. The Species column can have only one value, and in this case it is a single string that is the concatenation of all selected species names, separated by "|" characters.
Hope that clears things up...
John David N. Dionisio, PhD
Associate Professor, Computer Science
Loyola Marymount University
On Aug 16, 2011, at 12:11 PM, Richard Brous wrote:
> OK, have moved onto getRelationsTableManager() ... this one seems pretty complicated...
>
> Have reviewed the method and submethods called and have the following questions:
>
> 1. What are the RelationshipTables that are stored in relationshipTables?
> aren't they species dependent? Can't seem to find where they were created?
> If not, then:
>
> 2. How should the if/else conditional be handled?
>
> if
>
> (speciesProfile.getSpeciesSpecificSystemTables().containsKey(stp.systemTable1) |
>
> speciesProfile.getSpeciesSpecificSystemTables().containsKey(stp.systemTable2)) {
> tableManager =
>
> speciesProfile.getRelationsTableManagerCustomizations(stp.systemTable1,
> stp.
>
> systemTable2, templateDefinedSystemToSystemCode, tableManager);
> This obviously needs to be made aware of Lists of SpeciesProfiles... to build the correct tableManager, do I need this conditional to run through every species or should we enclose all the code (if and else) within a loop through each species?
>
> Thanks.
>
> Richard
>
>
> On Tue, Aug 16, 2011 at 11:02 AM, Richard Brous <rbr...@gm...> wrote:
> Sorry I typed and sent that out too quickly before getting my thoughts completely together...
>
> First off i thought that the array[][] portion of the submit needed to be in the following format: { "Species", speciesName1 }, { "Species", speciesName2 }, { "Species", speciesName3 },...
> but I see you specified it as: { "Species", speciesName1 | speciesName2 | speciesName3 }, ... OR'ing each of the species names?
>
> I see what you did regarding the date object... yes no reason to recreate a second object...
>
> As mentioned previously I will continue through the next methods, submitting (well thought out) changes and emailing out updates...
>
> Richard
> On Tue, Aug 16, 2011 at 10:36 AM, Richard Brous <rbr...@gm...> wrote:
> Took a look at your code and realized what I had done wrong. I should have just broken out the loop as the second argument of the 3 in the submit method. That was a rookie move on my part.
>
> I am moving on to the next method and will keep my mind on the syntactic solution that I'm not thinking through prior to moving forward with an implementation.
>
> Richard
>
> On Mon, Aug 15, 2011 at 1:30 AM, John David N. Dionisio <do...@lm...> wrote:
> OK, everything is committed. I got up to the tweaks on the second panel in the wizard (Save As/GO Aspects), as well as the query change for exporting any combination of C, F, or P.
>
> I still have to do the GO OBO format check, plus the UI work on the remaining two wizard panels. But meanwhile, hope these latest changes work out well.
>
> John David N. Dionisio, PhD
> Associate Professor, Computer Science
> Loyola Marymount University
>
>
>
> On Aug 14, 2011, at 7:31 PM, Richard Brous wrote:
>
> > OK, using option 2 I have made changes to DatabaseProfile.java to allow for all species names to be included in the submit argument.
> > But I'm stuck on how to change my StringBuilder object to a type that submit wants. Help please!!
> >
> > Submitted the above and some comment changes to SourceForge this evening.
> >
> > Richard
> >
> > On Fri, Aug 12, 2011 at 12:08 PM, Kam Dahlquist <kda...@lm...> wrote:
> > Hi,
> >
> > I think we had better leave the info table with only one record
> > (option 2). The species names can be separated by pipes " | " as
> > they are in the Systems table where there are multiple species. To
> > my knowledge, the only time GenMAPP needs to access the info table is
> > for the "DisplayOrder" field, I don't know what would happen if there
> > were multiple records there. I know that the spec for the table says
> > that it should only be one record, but I don't know if it would crash
> > if there were multiple records. To be on the safe side, I think we
> > should just keep it to the one record.
> >
> > Best,
> > Kam
> >
> > At 09:45 PM 8/10/2011, John David N. Dionisio wrote:
> > >Greetings,
> > >
> > >I think we have to turn to Dr. Dahlquist's GenMAPP knowledge here to
> > >get the definitive answer. I see two choices:
> > >
> > >- The Info table should have one record for each species that the
> > >.gdb holds, in which case the change you need is to wrap that single
> > >submit call inside a loop, so that submit is called once for each
> > >chosen species.
> > >
> > >- The Info table should always have one record, and if the .gdb
> > >holds multiple species, the "Species" column should be some
> > >concatenation of multiple species names. In this case, you would
> > >still call submit only once, but the value you send into the
> > >"Species" column is some accumulation of all chosen species names.
> > >
> > >Admittedly I don't know which way is right (I assumed the former as
> > >of our Tuesday meeting, but on further examination I'm no longer
> > >quite so sure).
> > >
> > >For Kam --- what does GenMAPP expect to see in the Info table if the
> > >opened .gdb contains multiple species?
> > >
> > >John David N. Dionisio, PhD
> > >Associate Professor, Computer Science
> > >Loyola Marymount University
> > >
> > >
> > >On Aug 10, 2011, at 9:36 PM, Richard Brous wrote:
> > >
> > > > OK, continued to review ExportToGenMAPP and dug into the creation
> > > of the first TableManager tmA on line 118.
> > > >
> > > > In reading through the method, my understanding is that it
> > > creates a new TableManager based on the selectedDatabaseProfile
> > > (which is UniProt).
> > > >
> > > > This is performed by the method getInfoTableManager() which then
> > > calls method submit(String tableName, QueryType queryType,
> > > String[][] columnNamesToValues);
> > > >
> > > > the code is as follows:
> > > >
> > > > tableManager.submit("Info", QueryType.insert, new String[][] { {
> > > "Owner", owner }, { "Version", new
> > > SimpleDateFormat("yyyyMMdd").format(version) }, { "MODSystem",
> > > modSystem }, { "Species", speciesProfile.getSpeciesName() }, {
> > > "Modify", new SimpleDateFormat("yyyyMMdd").format(modify) }, {
> > > "DisplayOrder", displayOrder }, { "Notes", notes } });
> > > >
> > > >
> > > > The modification of this line centers on { "Species",
> > > speciesProfile.getSpeciesName() }, since it originally processed a
> > > single species.
> > > >
> > > > So now I need to populate the arguments with the species
> > > contained within selectedDatabaseprofile.selectedSpeciesProfiles.
> > > >
> > > > I think I'll start with the baseArgument up to MODSystem, then
> > > append as many species as necessary, and then cap off the end with
> > > the rest starting at Modify. (similar to your approach in
> > > ExportGoData, populateUniprotGoTableFromSQL(char chosenAspect,
> > > List<Integer> taxonIds) line 513
> > > >
> > > > Please let me know if this approach or analysis is off track.
> > > >
> > > > Thanks!
> > > >
> > > > Richard
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Wed, Aug 10, 2011 at 5:50 PM, Richard Brous <rbr...@gm...> wrote:
> > > > Updated repository to include all Gene Ontology changes discussed
> > > during our meeting yesterday.
> > > >
> > > > Digging into TableManager next.
> > > >
> > > > Richard
> > > >
> > > > On Fri, Aug 5, 2011 at 10:06 AM, Richard Brous <rbr...@gm...> wrote:
> > > > whew... thanks for the detailed reply. I will digest this a bit
> > > and get back to you with further questions.
> > > >
> > > > rb
> > > >
> > > > On Thu, Aug 4, 2011 at 11:18 PM, John David N. Dionisio
> > > <do...@lm...> wrote:
> > > > Greetings,
> > > >
> > > > Sorry for the delay. I wasn't able to walk through the relevant
> > > code until this evening.
> > > >
> > > > As Kam said, GOA serves as the link between the UniProt and GO
> > > IDs. It essentially determines which GO IDs get exported by using
> > > GOA to see which GO IDs are associated with an exported UniProt
> > > ID. The populateUniprotGoTableFromSQL, in its current form,
> > > extracts the GO association records that match the given taxon ID
> > > then exports, as UniProt-GO pairs, the GO and UniProt IDs
> > > referenced within that GO association record. Processing that
> > > follows this is then based on the GO IDs that got exported --- and
> > > that's how the current code avoids exporting the entire list of GO terms.
> > > >
> > > > The operative query is on the second line of populateUniprotGoTableFromSQL:
> > > >
> > > > String uniProtAndGOIDSQL = "select db_object_id, go_id,
> > > evidence_code, with_or_from from goa where db like '%UniProt%' and
> > > taxon = 'taxon:" + taxon + "'";
> > > >
> > > > In plain English, this selects the GOA records whose database is
> > > UniProt and whose taxon ID is the given taxon. An additional
> > > condition is added for the "aspect" (All, Component, Function, or
> > > Process) that is to be exported. This is another reduction filter,
> > > to further shrink the number of exported GO terms and thus avoid
> > > MAPPFinder issues later on.
> > > >
> > > > Given this, the proper expansion here is to change the taxon
> > > predicate to a multiple predicate. That is, this method can be
> > > changed to now accept a collection or array of taxon IDs, and the
> > > base query should then be changed so that it accepts any taxon from
> > > that collection. More or less, you want:
> > > >
> > > > private void populateUniprotGoTableFromSQL(char chosenAspect,
> > > int[] taxons) throws SQLException {
> > > >
> > > > ...then, instead of the single string, you want to iterate
> > > through the taxon IDs:
> > > >
> > > > StringBuilder baseQueryBuilder = new StringBuilder("select
> > > db_object_id, go_id, evidence_code, with_or_from from goa where db
> > > like '%UniProt%'");
> > > > boolean first = true;
> > > > for (int taxon: taxons) {
> > > > baseQueryBuilder.append(first ? " and (" : " or ");
> > > > baseQueryBuilder
> > > > .append("taxon = 'taxon:")
> > > > .append(taxon).append("'");
> > > > first = false;
> > > > }
> > > > baseQueryBuilder.append(")");
> > > >
> > > > ...and so on. I just sort of rattled this off so there may be
> > > little glitches, but anyway this is just to give you an overall idea.
> > > >
> > > > Put another way, no, you do not need to iterate this method for
> > > each taxon ID. Instead, you can still call this method once, with
> > > the multiplicity of taxon IDs emerging in terms of the actual
> > > condition used for selecting the GO terms to be exported (based on
> > > the available GOA records, which as you may recall are loaded from .goa files).
> > > >
> > > > As a side note, right here you have an opportunity for a little
> > > sanity check regarding the content of the relational database: GO
> > > terms will only be exported if GOA records for the desired taxon
> > > IDs have been imported into the database. So, as a pre-flight
> > > check, one can see if there are any GOA records at all for each
> > > chosen taxon ID. If there are none, then the .goa file for that
> > > species needs to be imported into the relational database.
> > > >
> > > > Hope this helps...
> > > >
> > > > John David N. Dionisio, PhD
> > > > Associate Professor, Computer Science
> > > > Loyola Marymount University
> > > >
> > > >
> > > > On Aug 4, 2011, at 1:00 PM, Kam Dahlquist wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > Dondi will have to chime in on this, but I think this is where
> > > things are going to get tricky.
> > > > >
> > > > > The final gdb does not actually contain the entire GO, it gets
> > > trimmed somehow based on the GO associations for a particular
> > > species. This is because MAPPFinder cannot handle loading the
> > > entire GO. Since there is some type of species-specific trimming
> > > going on, it's quite possible that this will need to iterate.
> > > > >
> > > > > However, I don't have the foggiest idea of how this works, so
> > > Dondi will have to chime in.
> > > > >
> > > > > Best,
> > > > > Kam
> > > > >
> > > > > At 12:09 AM 8/4/2011, you wrote:
> > > > >> Wednesday 8/3/11 progress:
> > > > >>
> > > > >> 1. After following the ExportPanel1.java ground zero code of:
> > > databaseProfile.setSelectedSpeciesProfile( selectedProfile );
> > > > >>
> > > > >> I found the method in DatabaseProfile.java plus a getter method;
> > > > >> SpeciesProfile setSelectedSpeciesProfile( speciesProfile ) and
> > > SpeciesProfile getSelectedSpeciesProfile( speciesProfile )
> > > > >>
> > > > >> I created two new methods that each handle List<Object> of
> > > SpeciesProfiles argument instead of a single SpeciesProfile;
> > > setSelectedSpeciesProfiles and getSelectedSpeciesProfiles.
> > > > >>
> > > > >> This enabled the ExportPanel1 ground zero code to become:
> > > databaseProfile.setSelectedSpeciesProfiles(selectedSpecies);
> > > > >>
> > > > >> 2. public static void export() on line 104 in ExportToGenMAPP.java
> > > > >>
> > > > >> On line 107 ExportGoData is instantiated which I found in
> > > ExportGoData.java and calls a method: public void export(char
> > > chosenAspect, int taxon).
> > > > >>
> > > > >> Within export, taxon id is required for another method:
> > > private void populateGoTables(char chosenAspect, int taxon).
> > > > >>
> > > > >> Within populateGoTables, taxon id is required for another
> > > method: private void populateUniprotGoTableFromSQL( char
> > > chosenAspect, int taxon).
> > > > >>
> > > > >> But, if the export to GDB process starts off with exporting GO
> > > data, doesn't it only need to do that once no matter how many
> > > species are selected? As you probably realize, I'm leading towards
> > > not having to iterate through this for each taxon id if possible.
> > > > >>
> > > > >> Also, how does the export actually work? How are GO ids and
> > > UniProt ids related within the table?
> > > > >>
> > > > >> Thanks!
> > > > >>
> > > > >> Richard
> > > > >>
> > > > >>
> > > > > <ATT00001..txt><ATT00002..txt>
> > > >
> > > >
> > > >
> > > ------------------------------------------------------------------------------
> > > > BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
> > > > The must-attend event for mobile developers. Connect with experts.
> > > > Get tools for creating Super Apps. See the latest technologies.
> > > > Sessions, hands-on labs, demos & much more. Register early & save!
> > > > http://p.sf.net/sfu/rim-blackberry-1
> > > > _______________________________________________
> > > > xmlpipedb-developer mailing list
> > > > xml...@li...
> > > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> > > >
> > > >
> > > >
> > > > <ATT00001..txt><ATT00002..txt>
> > >
> > >
> > >------------------------------------------------------------------------------
> > >Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
> > >user administration capabilities and model configuration. Take
> > >the hassle out of deploying and managing Subversion and the
> > >tools developers use with it.
> > >http://p.sf.net/sfu/wandisco-dev2dev
> > >_______________________________________________
> > >xmlpipedb-developer mailing list
> > >xml...@li...
> > >https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> >
> >
> > ------------------------------------------------------------------------------
> > FREE DOWNLOAD - uberSVN with Social Coding for Subversion.
> > Subversion made easy with a complete admin console. Easy
> > to use, easy to manage, easy to install, easy to extend.
> > Get a Free download of the new open ALM Subversion platform now.
> > http://p.sf.net/sfu/wandisco-dev2dev
> > _______________________________________________
> > xmlpipedb-developer mailing list
> > xml...@li...
> > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> >
> > <ATT00001..txt><ATT00002..txt>
>
>
> ------------------------------------------------------------------------------
> uberSVN's rich system and user administration capabilities and model
> configuration take the hassle out of deploying and managing Subversion and
> the tools developers use with it. Learn more about uberSVN and get a free
> download at: http://p.sf.net/sfu/wandisco-dev2dev
> _______________________________________________
> xmlpipedb-developer mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>
>
>
> <ATT00001..txt><ATT00002..txt>
|
|
From: Richard B. <rbr...@gm...> - 2011-08-16 19:11:50
|
OK, have moved onto getRelationsTableManager() ... this one seems pretty
complicated...
Have reviewed the method and submethods called and have the following
questions:
1. What are the RelationshipTables that are stored in relationshipTables?
aren't they species dependent? Can't seem to find where they were created?
If not, then:
2. How should the if/else conditional be handled?
*
if* (speciesProfile.getSpeciesSpecificSystemTables().containsKey(stp.
systemTable1) |
speciesProfile.getSpeciesSpecificSystemTables().containsKey(stp.systemTable2))
{
tableManager = speciesProfile.getRelationsTableManagerCustomizations(stp.
systemTable1,
stp.systemTable2, *templateDefinedSystemToSystemCode*, tableManager);
This obviously needs to be made aware of Lists of SpeciesProfiles... to
build the correct tableManager, do I need this conditional to run through
every species or should we enclose all the code (if and else) within a loop
through each species?
Thanks.
Richard
On Tue, Aug 16, 2011 at 11:02 AM, Richard Brous <rbr...@gm...> wrote:
> Sorry I typed and sent that out too quickly before getting my thoughts
> completely together...
>
> First off i thought that the array[][] portion of the submit needed to be
> in the following format: { "Species", speciesName1 }, { "Species",
> speciesName2 }, { "Species", speciesName3 },...
> but I see you specified it as: { "Species", speciesName1 | speciesName2 |
> speciesName3 }, ... OR'ing each of the species names?
>
> I see what you did regarding the date object... yes no reason to recreate a
> second object...
>
> As mentioned previously I will continue through the next methods,
> submitting (well thought out) changes and emailing out updates...
>
> Richard
> On Tue, Aug 16, 2011 at 10:36 AM, Richard Brous <rbr...@gm...>wrote:
>
>> Took a look at your code and realized what I had done wrong. I should have
>> just broken out the loop as the second argument of the 3 in the submit
>> method. That was a rookie move on my part.
>>
>> I am moving on to the next method and will keep my mind on the syntactic
>> solution that I'm not thinking through prior to moving forward with an
>> implementation.
>>
>> Richard
>>
>> On Mon, Aug 15, 2011 at 1:30 AM, John David N. Dionisio <do...@lm...>wrote:
>>
>>> OK, everything is committed. I got up to the tweaks on the second panel
>>> in the wizard (Save As/GO Aspects), as well as the query change for
>>> exporting any combination of C, F, or P.
>>>
>>> I still have to do the GO OBO format check, plus the UI work on the
>>> remaining two wizard panels. But meanwhile, hope these latest changes work
>>> out well.
>>>
>>> John David N. Dionisio, PhD
>>> Associate Professor, Computer Science
>>> Loyola Marymount University
>>>
>>>
>>>
>>> On Aug 14, 2011, at 7:31 PM, Richard Brous wrote:
>>>
>>> > OK, using option 2 I have made changes to DatabaseProfile.java to allow
>>> for all species names to be included in the submit argument.
>>> > But I'm stuck on how to change my StringBuilder object to a type that
>>> submit wants. Help please!!
>>> >
>>> > Submitted the above and some comment changes to SourceForge this
>>> evening.
>>> >
>>> > Richard
>>> >
>>> > On Fri, Aug 12, 2011 at 12:08 PM, Kam Dahlquist <kda...@lm...>
>>> wrote:
>>> > Hi,
>>> >
>>> > I think we had better leave the info table with only one record
>>> > (option 2). The species names can be separated by pipes " | " as
>>> > they are in the Systems table where there are multiple species. To
>>> > my knowledge, the only time GenMAPP needs to access the info table is
>>> > for the "DisplayOrder" field, I don't know what would happen if there
>>> > were multiple records there. I know that the spec for the table says
>>> > that it should only be one record, but I don't know if it would crash
>>> > if there were multiple records. To be on the safe side, I think we
>>> > should just keep it to the one record.
>>> >
>>> > Best,
>>> > Kam
>>> >
>>> > At 09:45 PM 8/10/2011, John David N. Dionisio wrote:
>>> > >Greetings,
>>> > >
>>> > >I think we have to turn to Dr. Dahlquist's GenMAPP knowledge here to
>>> > >get the definitive answer. I see two choices:
>>> > >
>>> > >- The Info table should have one record for each species that the
>>> > >.gdb holds, in which case the change you need is to wrap that single
>>> > >submit call inside a loop, so that submit is called once for each
>>> > >chosen species.
>>> > >
>>> > >- The Info table should always have one record, and if the .gdb
>>> > >holds multiple species, the "Species" column should be some
>>> > >concatenation of multiple species names. In this case, you would
>>> > >still call submit only once, but the value you send into the
>>> > >"Species" column is some accumulation of all chosen species names.
>>> > >
>>> > >Admittedly I don't know which way is right (I assumed the former as
>>> > >of our Tuesday meeting, but on further examination I'm no longer
>>> > >quite so sure).
>>> > >
>>> > >For Kam --- what does GenMAPP expect to see in the Info table if the
>>> > >opened .gdb contains multiple species?
>>> > >
>>> > >John David N. Dionisio, PhD
>>> > >Associate Professor, Computer Science
>>> > >Loyola Marymount University
>>> > >
>>> > >
>>> > >On Aug 10, 2011, at 9:36 PM, Richard Brous wrote:
>>> > >
>>> > > > OK, continued to review ExportToGenMAPP and dug into the creation
>>> > > of the first TableManager tmA on line 118.
>>> > > >
>>> > > > In reading through the method, my understanding is that it
>>> > > creates a new TableManager based on the selectedDatabaseProfile
>>> > > (which is UniProt).
>>> > > >
>>> > > > This is performed by the method getInfoTableManager() which then
>>> > > calls method submit(String tableName, QueryType queryType,
>>> > > String[][] columnNamesToValues);
>>> > > >
>>> > > > the code is as follows:
>>> > > >
>>> > > > tableManager.submit("Info", QueryType.insert, new String[][] { {
>>> > > "Owner", owner }, { "Version", new
>>> > > SimpleDateFormat("yyyyMMdd").format(version) }, { "MODSystem",
>>> > > modSystem }, { "Species", speciesProfile.getSpeciesName() }, {
>>> > > "Modify", new SimpleDateFormat("yyyyMMdd").format(modify) }, {
>>> > > "DisplayOrder", displayOrder }, { "Notes", notes } });
>>> > > >
>>> > > >
>>> > > > The modification of this line centers on { "Species",
>>> > > speciesProfile.getSpeciesName() }, since it originally processed a
>>> > > single species.
>>> > > >
>>> > > > So now I need to populate the arguments with the species
>>> > > contained within selectedDatabaseprofile.selectedSpeciesProfiles.
>>> > > >
>>> > > > I think I'll start with the baseArgument up to MODSystem, then
>>> > > append as many species as necessary, and then cap off the end with
>>> > > the rest starting at Modify. (similar to your approach in
>>> > > ExportGoData, populateUniprotGoTableFromSQL(char chosenAspect,
>>> > > List<Integer> taxonIds) line 513
>>> > > >
>>> > > > Please let me know if this approach or analysis is off track.
>>> > > >
>>> > > > Thanks!
>>> > > >
>>> > > > Richard
>>> > > >
>>> > > >
>>> > > >
>>> > > >
>>> > > >
>>> > > >
>>> > > > On Wed, Aug 10, 2011 at 5:50 PM, Richard Brous <rbr...@gm...>
>>> wrote:
>>> > > > Updated repository to include all Gene Ontology changes discussed
>>> > > during our meeting yesterday.
>>> > > >
>>> > > > Digging into TableManager next.
>>> > > >
>>> > > > Richard
>>> > > >
>>> > > > On Fri, Aug 5, 2011 at 10:06 AM, Richard Brous <rbr...@gm...>
>>> wrote:
>>> > > > whew... thanks for the detailed reply. I will digest this a bit
>>> > > and get back to you with further questions.
>>> > > >
>>> > > > rb
>>> > > >
>>> > > > On Thu, Aug 4, 2011 at 11:18 PM, John David N. Dionisio
>>> > > <do...@lm...> wrote:
>>> > > > Greetings,
>>> > > >
>>> > > > Sorry for the delay. I wasn't able to walk through the relevant
>>> > > code until this evening.
>>> > > >
>>> > > > As Kam said, GOA serves as the link between the UniProt and GO
>>> > > IDs. It essentially determines which GO IDs get exported by using
>>> > > GOA to see which GO IDs are associated with an exported UniProt
>>> > > ID. The populateUniprotGoTableFromSQL, in its current form,
>>> > > extracts the GO association records that match the given taxon ID
>>> > > then exports, as UniProt-GO pairs, the GO and UniProt IDs
>>> > > referenced within that GO association record. Processing that
>>> > > follows this is then based on the GO IDs that got exported --- and
>>> > > that's how the current code avoids exporting the entire list of GO
>>> terms.
>>> > > >
>>> > > > The operative query is on the second line of
>>> populateUniprotGoTableFromSQL:
>>> > > >
>>> > > > String uniProtAndGOIDSQL = "select db_object_id, go_id,
>>> > > evidence_code, with_or_from from goa where db like '%UniProt%' and
>>> > > taxon = 'taxon:" + taxon + "'";
>>> > > >
>>> > > > In plain English, this selects the GOA records whose database is
>>> > > UniProt and whose taxon ID is the given taxon. An additional
>>> > > condition is added for the "aspect" (All, Component, Function, or
>>> > > Process) that is to be exported. This is another reduction filter,
>>> > > to further shrink the number of exported GO terms and thus avoid
>>> > > MAPPFinder issues later on.
>>> > > >
>>> > > > Given this, the proper expansion here is to change the taxon
>>> > > predicate to a multiple predicate. That is, this method can be
>>> > > changed to now accept a collection or array of taxon IDs, and the
>>> > > base query should then be changed so that it accepts any taxon from
>>> > > that collection. More or less, you want:
>>> > > >
>>> > > > private void populateUniprotGoTableFromSQL(char chosenAspect,
>>> > > int[] taxons) throws SQLException {
>>> > > >
>>> > > > ...then, instead of the single string, you want to iterate
>>> > > through the taxon IDs:
>>> > > >
>>> > > > StringBuilder baseQueryBuilder = new StringBuilder("select
>>> > > db_object_id, go_id, evidence_code, with_or_from from goa where db
>>> > > like '%UniProt%'");
>>> > > > boolean first = true;
>>> > > > for (int taxon: taxons) {
>>> > > > baseQueryBuilder.append(first ? " and (" : " or ");
>>> > > > baseQueryBuilder
>>> > > > .append("taxon = 'taxon:")
>>> > > > .append(taxon).append("'");
>>> > > > first = false;
>>> > > > }
>>> > > > baseQueryBuilder.append(")");
>>> > > >
>>> > > > ...and so on. I just sort of rattled this off so there may be
>>> > > little glitches, but anyway this is just to give you an overall idea.
>>> > > >
>>> > > > Put another way, no, you do not need to iterate this method for
>>> > > each taxon ID. Instead, you can still call this method once, with
>>> > > the multiplicity of taxon IDs emerging in terms of the actual
>>> > > condition used for selecting the GO terms to be exported (based on
>>> > > the available GOA records, which as you may recall are loaded from
>>> .goa files).
>>> > > >
>>> > > > As a side note, right here you have an opportunity for a little
>>> > > sanity check regarding the content of the relational database: GO
>>> > > terms will only be exported if GOA records for the desired taxon
>>> > > IDs have been imported into the database. So, as a pre-flight
>>> > > check, one can see if there are any GOA records at all for each
>>> > > chosen taxon ID. If there are none, then the .goa file for that
>>> > > species needs to be imported into the relational database.
>>> > > >
>>> > > > Hope this helps...
>>> > > >
>>> > > > John David N. Dionisio, PhD
>>> > > > Associate Professor, Computer Science
>>> > > > Loyola Marymount University
>>> > > >
>>> > > >
>>> > > > On Aug 4, 2011, at 1:00 PM, Kam Dahlquist wrote:
>>> > > >
>>> > > > > Hi,
>>> > > > >
>>> > > > > Dondi will have to chime in on this, but I think this is where
>>> > > things are going to get tricky.
>>> > > > >
>>> > > > > The final gdb does not actually contain the entire GO, it gets
>>> > > trimmed somehow based on the GO associations for a particular
>>> > > species. This is because MAPPFinder cannot handle loading the
>>> > > entire GO. Since there is some type of species-specific trimming
>>> > > going on, it's quite possible that this will need to iterate.
>>> > > > >
>>> > > > > However, I don't have the foggiest idea of how this works, so
>>> > > Dondi will have to chime in.
>>> > > > >
>>> > > > > Best,
>>> > > > > Kam
>>> > > > >
>>> > > > > At 12:09 AM 8/4/2011, you wrote:
>>> > > > >> Wednesday 8/3/11 progress:
>>> > > > >>
>>> > > > >> 1. After following the ExportPanel1.java ground zero code of:
>>> > > databaseProfile.setSelectedSpeciesProfile( selectedProfile );
>>> > > > >>
>>> > > > >> I found the method in DatabaseProfile.java plus a getter method;
>>> > > > >> SpeciesProfile setSelectedSpeciesProfile( speciesProfile ) and
>>> > > SpeciesProfile getSelectedSpeciesProfile( speciesProfile )
>>> > > > >>
>>> > > > >> I created two new methods that each handle List<Object> of
>>> > > SpeciesProfiles argument instead of a single SpeciesProfile;
>>> > > setSelectedSpeciesProfiles and getSelectedSpeciesProfiles.
>>> > > > >>
>>> > > > >> This enabled the ExportPanel1 ground zero code to become:
>>> > > databaseProfile.setSelectedSpeciesProfiles(selectedSpecies);
>>> > > > >>
>>> > > > >> 2. public static void export() on line 104 in
>>> ExportToGenMAPP.java
>>> > > > >>
>>> > > > >> On line 107 ExportGoData is instantiated which I found in
>>> > > ExportGoData.java and calls a method: public void export(char
>>> > > chosenAspect, int taxon).
>>> > > > >>
>>> > > > >> Within export, taxon id is required for another method:
>>> > > private void populateGoTables(char chosenAspect, int taxon).
>>> > > > >>
>>> > > > >> Within populateGoTables, taxon id is required for another
>>> > > method: private void populateUniprotGoTableFromSQL( char
>>> > > chosenAspect, int taxon).
>>> > > > >>
>>> > > > >> But, if the export to GDB process starts off with exporting GO
>>> > > data, doesn't it only need to do that once no matter how many
>>> > > species are selected? As you probably realize, I'm leading towards
>>> > > not having to iterate through this for each taxon id if possible.
>>> > > > >>
>>> > > > >> Also, how does the export actually work? How are GO ids and
>>> > > UniProt ids related within the table?
>>> > > > >>
>>> > > > >> Thanks!
>>> > > > >>
>>> > > > >> Richard
>>> > > > >>
>>> > > > >>
>>> > > > > <ATT00001..txt><ATT00002..txt>
>>> > > >
>>> > > >
>>> > > >
>>> > >
>>> ------------------------------------------------------------------------------
>>> > > > BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
>>> > > > The must-attend event for mobile developers. Connect with experts.
>>> > > > Get tools for creating Super Apps. See the latest technologies.
>>> > > > Sessions, hands-on labs, demos & much more. Register early & save!
>>> > > > http://p.sf.net/sfu/rim-blackberry-1
>>> > > > _______________________________________________
>>> > > > xmlpipedb-developer mailing list
>>> > > > xml...@li...
>>> > > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>>> > > >
>>> > > >
>>> > > >
>>> > > > <ATT00001..txt><ATT00002..txt>
>>> > >
>>> > >
>>> >
>>> >------------------------------------------------------------------------------
>>> > >Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
>>> > >user administration capabilities and model configuration. Take
>>> > >the hassle out of deploying and managing Subversion and the
>>> > >tools developers use with it.
>>> > >http://p.sf.net/sfu/wandisco-dev2dev
>>> > >_______________________________________________
>>> > >xmlpipedb-developer mailing list
>>> > >xml...@li...
>>> > >https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>>> >
>>> >
>>> >
>>> ------------------------------------------------------------------------------
>>> > FREE DOWNLOAD - uberSVN with Social Coding for Subversion.
>>> > Subversion made easy with a complete admin console. Easy
>>> > to use, easy to manage, easy to install, easy to extend.
>>> > Get a Free download of the new open ALM Subversion platform now.
>>> > http://p.sf.net/sfu/wandisco-dev2dev
>>> > _______________________________________________
>>> > xmlpipedb-developer mailing list
>>> > xml...@li...
>>> > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>>> >
>>> > <ATT00001..txt><ATT00002..txt>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> uberSVN's rich system and user administration capabilities and model
>>> configuration take the hassle out of deploying and managing Subversion
>>> and
>>> the tools developers use with it. Learn more about uberSVN and get a free
>>> download at: http://p.sf.net/sfu/wandisco-dev2dev
>>> _______________________________________________
>>> xmlpipedb-developer mailing list
>>> xml...@li...
>>> https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>>>
>>
>>
>
|
|
From: Richard B. <rbr...@gm...> - 2011-08-16 18:02:15
|
Sorry I typed and sent that out too quickly before getting my thoughts
completely together...
First off i thought that the array[][] portion of the submit needed to be in
the following format: { "Species", speciesName1 }, { "Species", speciesName2
}, { "Species", speciesName3 },...
but I see you specified it as: { "Species", speciesName1 | speciesName2 |
speciesName3 }, ... OR'ing each of the species names?
I see what you did regarding the date object... yes no reason to recreate a
second object...
As mentioned previously I will continue through the next methods, submitting
(well thought out) changes and emailing out updates...
Richard
On Tue, Aug 16, 2011 at 10:36 AM, Richard Brous <rbr...@gm...> wrote:
> Took a look at your code and realized what I had done wrong. I should have
> just broken out the loop as the second argument of the 3 in the submit
> method. That was a rookie move on my part.
>
> I am moving on to the next method and will keep my mind on the syntactic
> solution that I'm not thinking through prior to moving forward with an
> implementation.
>
> Richard
>
> On Mon, Aug 15, 2011 at 1:30 AM, John David N. Dionisio <do...@lm...>wrote:
>
>> OK, everything is committed. I got up to the tweaks on the second panel
>> in the wizard (Save As/GO Aspects), as well as the query change for
>> exporting any combination of C, F, or P.
>>
>> I still have to do the GO OBO format check, plus the UI work on the
>> remaining two wizard panels. But meanwhile, hope these latest changes work
>> out well.
>>
>> John David N. Dionisio, PhD
>> Associate Professor, Computer Science
>> Loyola Marymount University
>>
>>
>>
>> On Aug 14, 2011, at 7:31 PM, Richard Brous wrote:
>>
>> > OK, using option 2 I have made changes to DatabaseProfile.java to allow
>> for all species names to be included in the submit argument.
>> > But I'm stuck on how to change my StringBuilder object to a type that
>> submit wants. Help please!!
>> >
>> > Submitted the above and some comment changes to SourceForge this
>> evening.
>> >
>> > Richard
>> >
>> > On Fri, Aug 12, 2011 at 12:08 PM, Kam Dahlquist <kda...@lm...>
>> wrote:
>> > Hi,
>> >
>> > I think we had better leave the info table with only one record
>> > (option 2). The species names can be separated by pipes " | " as
>> > they are in the Systems table where there are multiple species. To
>> > my knowledge, the only time GenMAPP needs to access the info table is
>> > for the "DisplayOrder" field, I don't know what would happen if there
>> > were multiple records there. I know that the spec for the table says
>> > that it should only be one record, but I don't know if it would crash
>> > if there were multiple records. To be on the safe side, I think we
>> > should just keep it to the one record.
>> >
>> > Best,
>> > Kam
>> >
>> > At 09:45 PM 8/10/2011, John David N. Dionisio wrote:
>> > >Greetings,
>> > >
>> > >I think we have to turn to Dr. Dahlquist's GenMAPP knowledge here to
>> > >get the definitive answer. I see two choices:
>> > >
>> > >- The Info table should have one record for each species that the
>> > >.gdb holds, in which case the change you need is to wrap that single
>> > >submit call inside a loop, so that submit is called once for each
>> > >chosen species.
>> > >
>> > >- The Info table should always have one record, and if the .gdb
>> > >holds multiple species, the "Species" column should be some
>> > >concatenation of multiple species names. In this case, you would
>> > >still call submit only once, but the value you send into the
>> > >"Species" column is some accumulation of all chosen species names.
>> > >
>> > >Admittedly I don't know which way is right (I assumed the former as
>> > >of our Tuesday meeting, but on further examination I'm no longer
>> > >quite so sure).
>> > >
>> > >For Kam --- what does GenMAPP expect to see in the Info table if the
>> > >opened .gdb contains multiple species?
>> > >
>> > >John David N. Dionisio, PhD
>> > >Associate Professor, Computer Science
>> > >Loyola Marymount University
>> > >
>> > >
>> > >On Aug 10, 2011, at 9:36 PM, Richard Brous wrote:
>> > >
>> > > > OK, continued to review ExportToGenMAPP and dug into the creation
>> > > of the first TableManager tmA on line 118.
>> > > >
>> > > > In reading through the method, my understanding is that it
>> > > creates a new TableManager based on the selectedDatabaseProfile
>> > > (which is UniProt).
>> > > >
>> > > > This is performed by the method getInfoTableManager() which then
>> > > calls method submit(String tableName, QueryType queryType,
>> > > String[][] columnNamesToValues);
>> > > >
>> > > > the code is as follows:
>> > > >
>> > > > tableManager.submit("Info", QueryType.insert, new String[][] { {
>> > > "Owner", owner }, { "Version", new
>> > > SimpleDateFormat("yyyyMMdd").format(version) }, { "MODSystem",
>> > > modSystem }, { "Species", speciesProfile.getSpeciesName() }, {
>> > > "Modify", new SimpleDateFormat("yyyyMMdd").format(modify) }, {
>> > > "DisplayOrder", displayOrder }, { "Notes", notes } });
>> > > >
>> > > >
>> > > > The modification of this line centers on { "Species",
>> > > speciesProfile.getSpeciesName() }, since it originally processed a
>> > > single species.
>> > > >
>> > > > So now I need to populate the arguments with the species
>> > > contained within selectedDatabaseprofile.selectedSpeciesProfiles.
>> > > >
>> > > > I think I'll start with the baseArgument up to MODSystem, then
>> > > append as many species as necessary, and then cap off the end with
>> > > the rest starting at Modify. (similar to your approach in
>> > > ExportGoData, populateUniprotGoTableFromSQL(char chosenAspect,
>> > > List<Integer> taxonIds) line 513
>> > > >
>> > > > Please let me know if this approach or analysis is off track.
>> > > >
>> > > > Thanks!
>> > > >
>> > > > Richard
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > On Wed, Aug 10, 2011 at 5:50 PM, Richard Brous <rbr...@gm...>
>> wrote:
>> > > > Updated repository to include all Gene Ontology changes discussed
>> > > during our meeting yesterday.
>> > > >
>> > > > Digging into TableManager next.
>> > > >
>> > > > Richard
>> > > >
>> > > > On Fri, Aug 5, 2011 at 10:06 AM, Richard Brous <rbr...@gm...>
>> wrote:
>> > > > whew... thanks for the detailed reply. I will digest this a bit
>> > > and get back to you with further questions.
>> > > >
>> > > > rb
>> > > >
>> > > > On Thu, Aug 4, 2011 at 11:18 PM, John David N. Dionisio
>> > > <do...@lm...> wrote:
>> > > > Greetings,
>> > > >
>> > > > Sorry for the delay. I wasn't able to walk through the relevant
>> > > code until this evening.
>> > > >
>> > > > As Kam said, GOA serves as the link between the UniProt and GO
>> > > IDs. It essentially determines which GO IDs get exported by using
>> > > GOA to see which GO IDs are associated with an exported UniProt
>> > > ID. The populateUniprotGoTableFromSQL, in its current form,
>> > > extracts the GO association records that match the given taxon ID
>> > > then exports, as UniProt-GO pairs, the GO and UniProt IDs
>> > > referenced within that GO association record. Processing that
>> > > follows this is then based on the GO IDs that got exported --- and
>> > > that's how the current code avoids exporting the entire list of GO
>> terms.
>> > > >
>> > > > The operative query is on the second line of
>> populateUniprotGoTableFromSQL:
>> > > >
>> > > > String uniProtAndGOIDSQL = "select db_object_id, go_id,
>> > > evidence_code, with_or_from from goa where db like '%UniProt%' and
>> > > taxon = 'taxon:" + taxon + "'";
>> > > >
>> > > > In plain English, this selects the GOA records whose database is
>> > > UniProt and whose taxon ID is the given taxon. An additional
>> > > condition is added for the "aspect" (All, Component, Function, or
>> > > Process) that is to be exported. This is another reduction filter,
>> > > to further shrink the number of exported GO terms and thus avoid
>> > > MAPPFinder issues later on.
>> > > >
>> > > > Given this, the proper expansion here is to change the taxon
>> > > predicate to a multiple predicate. That is, this method can be
>> > > changed to now accept a collection or array of taxon IDs, and the
>> > > base query should then be changed so that it accepts any taxon from
>> > > that collection. More or less, you want:
>> > > >
>> > > > private void populateUniprotGoTableFromSQL(char chosenAspect,
>> > > int[] taxons) throws SQLException {
>> > > >
>> > > > ...then, instead of the single string, you want to iterate
>> > > through the taxon IDs:
>> > > >
>> > > > StringBuilder baseQueryBuilder = new StringBuilder("select
>> > > db_object_id, go_id, evidence_code, with_or_from from goa where db
>> > > like '%UniProt%'");
>> > > > boolean first = true;
>> > > > for (int taxon: taxons) {
>> > > > baseQueryBuilder.append(first ? " and (" : " or ");
>> > > > baseQueryBuilder
>> > > > .append("taxon = 'taxon:")
>> > > > .append(taxon).append("'");
>> > > > first = false;
>> > > > }
>> > > > baseQueryBuilder.append(")");
>> > > >
>> > > > ...and so on. I just sort of rattled this off so there may be
>> > > little glitches, but anyway this is just to give you an overall idea.
>> > > >
>> > > > Put another way, no, you do not need to iterate this method for
>> > > each taxon ID. Instead, you can still call this method once, with
>> > > the multiplicity of taxon IDs emerging in terms of the actual
>> > > condition used for selecting the GO terms to be exported (based on
>> > > the available GOA records, which as you may recall are loaded from
>> .goa files).
>> > > >
>> > > > As a side note, right here you have an opportunity for a little
>> > > sanity check regarding the content of the relational database: GO
>> > > terms will only be exported if GOA records for the desired taxon
>> > > IDs have been imported into the database. So, as a pre-flight
>> > > check, one can see if there are any GOA records at all for each
>> > > chosen taxon ID. If there are none, then the .goa file for that
>> > > species needs to be imported into the relational database.
>> > > >
>> > > > Hope this helps...
>> > > >
>> > > > John David N. Dionisio, PhD
>> > > > Associate Professor, Computer Science
>> > > > Loyola Marymount University
>> > > >
>> > > >
>> > > > On Aug 4, 2011, at 1:00 PM, Kam Dahlquist wrote:
>> > > >
>> > > > > Hi,
>> > > > >
>> > > > > Dondi will have to chime in on this, but I think this is where
>> > > things are going to get tricky.
>> > > > >
>> > > > > The final gdb does not actually contain the entire GO, it gets
>> > > trimmed somehow based on the GO associations for a particular
>> > > species. This is because MAPPFinder cannot handle loading the
>> > > entire GO. Since there is some type of species-specific trimming
>> > > going on, it's quite possible that this will need to iterate.
>> > > > >
>> > > > > However, I don't have the foggiest idea of how this works, so
>> > > Dondi will have to chime in.
>> > > > >
>> > > > > Best,
>> > > > > Kam
>> > > > >
>> > > > > At 12:09 AM 8/4/2011, you wrote:
>> > > > >> Wednesday 8/3/11 progress:
>> > > > >>
>> > > > >> 1. After following the ExportPanel1.java ground zero code of:
>> > > databaseProfile.setSelectedSpeciesProfile( selectedProfile );
>> > > > >>
>> > > > >> I found the method in DatabaseProfile.java plus a getter method;
>> > > > >> SpeciesProfile setSelectedSpeciesProfile( speciesProfile ) and
>> > > SpeciesProfile getSelectedSpeciesProfile( speciesProfile )
>> > > > >>
>> > > > >> I created two new methods that each handle List<Object> of
>> > > SpeciesProfiles argument instead of a single SpeciesProfile;
>> > > setSelectedSpeciesProfiles and getSelectedSpeciesProfiles.
>> > > > >>
>> > > > >> This enabled the ExportPanel1 ground zero code to become:
>> > > databaseProfile.setSelectedSpeciesProfiles(selectedSpecies);
>> > > > >>
>> > > > >> 2. public static void export() on line 104 in
>> ExportToGenMAPP.java
>> > > > >>
>> > > > >> On line 107 ExportGoData is instantiated which I found in
>> > > ExportGoData.java and calls a method: public void export(char
>> > > chosenAspect, int taxon).
>> > > > >>
>> > > > >> Within export, taxon id is required for another method:
>> > > private void populateGoTables(char chosenAspect, int taxon).
>> > > > >>
>> > > > >> Within populateGoTables, taxon id is required for another
>> > > method: private void populateUniprotGoTableFromSQL( char
>> > > chosenAspect, int taxon).
>> > > > >>
>> > > > >> But, if the export to GDB process starts off with exporting GO
>> > > data, doesn't it only need to do that once no matter how many
>> > > species are selected? As you probably realize, I'm leading towards
>> > > not having to iterate through this for each taxon id if possible.
>> > > > >>
>> > > > >> Also, how does the export actually work? How are GO ids and
>> > > UniProt ids related within the table?
>> > > > >>
>> > > > >> Thanks!
>> > > > >>
>> > > > >> Richard
>> > > > >>
>> > > > >>
>> > > > > <ATT00001..txt><ATT00002..txt>
>> > > >
>> > > >
>> > > >
>> > >
>> ------------------------------------------------------------------------------
>> > > > BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
>> > > > The must-attend event for mobile developers. Connect with experts.
>> > > > Get tools for creating Super Apps. See the latest technologies.
>> > > > Sessions, hands-on labs, demos & much more. Register early & save!
>> > > > http://p.sf.net/sfu/rim-blackberry-1
>> > > > _______________________________________________
>> > > > xmlpipedb-developer mailing list
>> > > > xml...@li...
>> > > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>> > > >
>> > > >
>> > > >
>> > > > <ATT00001..txt><ATT00002..txt>
>> > >
>> > >
>> >
>> >------------------------------------------------------------------------------
>> > >Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
>> > >user administration capabilities and model configuration. Take
>> > >the hassle out of deploying and managing Subversion and the
>> > >tools developers use with it.
>> > >http://p.sf.net/sfu/wandisco-dev2dev
>> > >_______________________________________________
>> > >xmlpipedb-developer mailing list
>> > >xml...@li...
>> > >https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>> >
>> >
>> >
>> ------------------------------------------------------------------------------
>> > FREE DOWNLOAD - uberSVN with Social Coding for Subversion.
>> > Subversion made easy with a complete admin console. Easy
>> > to use, easy to manage, easy to install, easy to extend.
>> > Get a Free download of the new open ALM Subversion platform now.
>> > http://p.sf.net/sfu/wandisco-dev2dev
>> > _______________________________________________
>> > xmlpipedb-developer mailing list
>> > xml...@li...
>> > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>> >
>> > <ATT00001..txt><ATT00002..txt>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> uberSVN's rich system and user administration capabilities and model
>> configuration take the hassle out of deploying and managing Subversion and
>> the tools developers use with it. Learn more about uberSVN and get a free
>> download at: http://p.sf.net/sfu/wandisco-dev2dev
>> _______________________________________________
>> xmlpipedb-developer mailing list
>> xml...@li...
>> https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>>
>
>
|
|
From: Richard B. <rbr...@gm...> - 2011-08-16 17:36:55
|
Took a look at your code and realized what I had done wrong. I should have
just broken out the loop as the second argument of the 3 in the submit
method. That was a rookie move on my part.
I am moving on to the next method and will keep my mind on the syntactic
solution that I'm not thinking through prior to moving forward with an
implementation.
Richard
On Mon, Aug 15, 2011 at 1:30 AM, John David N. Dionisio <do...@lm...>wrote:
> OK, everything is committed. I got up to the tweaks on the second panel in
> the wizard (Save As/GO Aspects), as well as the query change for exporting
> any combination of C, F, or P.
>
> I still have to do the GO OBO format check, plus the UI work on the
> remaining two wizard panels. But meanwhile, hope these latest changes work
> out well.
>
> John David N. Dionisio, PhD
> Associate Professor, Computer Science
> Loyola Marymount University
>
>
>
> On Aug 14, 2011, at 7:31 PM, Richard Brous wrote:
>
> > OK, using option 2 I have made changes to DatabaseProfile.java to allow
> for all species names to be included in the submit argument.
> > But I'm stuck on how to change my StringBuilder object to a type that
> submit wants. Help please!!
> >
> > Submitted the above and some comment changes to SourceForge this evening.
> >
> > Richard
> >
> > On Fri, Aug 12, 2011 at 12:08 PM, Kam Dahlquist <kda...@lm...>
> wrote:
> > Hi,
> >
> > I think we had better leave the info table with only one record
> > (option 2). The species names can be separated by pipes " | " as
> > they are in the Systems table where there are multiple species. To
> > my knowledge, the only time GenMAPP needs to access the info table is
> > for the "DisplayOrder" field, I don't know what would happen if there
> > were multiple records there. I know that the spec for the table says
> > that it should only be one record, but I don't know if it would crash
> > if there were multiple records. To be on the safe side, I think we
> > should just keep it to the one record.
> >
> > Best,
> > Kam
> >
> > At 09:45 PM 8/10/2011, John David N. Dionisio wrote:
> > >Greetings,
> > >
> > >I think we have to turn to Dr. Dahlquist's GenMAPP knowledge here to
> > >get the definitive answer. I see two choices:
> > >
> > >- The Info table should have one record for each species that the
> > >.gdb holds, in which case the change you need is to wrap that single
> > >submit call inside a loop, so that submit is called once for each
> > >chosen species.
> > >
> > >- The Info table should always have one record, and if the .gdb
> > >holds multiple species, the "Species" column should be some
> > >concatenation of multiple species names. In this case, you would
> > >still call submit only once, but the value you send into the
> > >"Species" column is some accumulation of all chosen species names.
> > >
> > >Admittedly I don't know which way is right (I assumed the former as
> > >of our Tuesday meeting, but on further examination I'm no longer
> > >quite so sure).
> > >
> > >For Kam --- what does GenMAPP expect to see in the Info table if the
> > >opened .gdb contains multiple species?
> > >
> > >John David N. Dionisio, PhD
> > >Associate Professor, Computer Science
> > >Loyola Marymount University
> > >
> > >
> > >On Aug 10, 2011, at 9:36 PM, Richard Brous wrote:
> > >
> > > > OK, continued to review ExportToGenMAPP and dug into the creation
> > > of the first TableManager tmA on line 118.
> > > >
> > > > In reading through the method, my understanding is that it
> > > creates a new TableManager based on the selectedDatabaseProfile
> > > (which is UniProt).
> > > >
> > > > This is performed by the method getInfoTableManager() which then
> > > calls method submit(String tableName, QueryType queryType,
> > > String[][] columnNamesToValues);
> > > >
> > > > the code is as follows:
> > > >
> > > > tableManager.submit("Info", QueryType.insert, new String[][] { {
> > > "Owner", owner }, { "Version", new
> > > SimpleDateFormat("yyyyMMdd").format(version) }, { "MODSystem",
> > > modSystem }, { "Species", speciesProfile.getSpeciesName() }, {
> > > "Modify", new SimpleDateFormat("yyyyMMdd").format(modify) }, {
> > > "DisplayOrder", displayOrder }, { "Notes", notes } });
> > > >
> > > >
> > > > The modification of this line centers on { "Species",
> > > speciesProfile.getSpeciesName() }, since it originally processed a
> > > single species.
> > > >
> > > > So now I need to populate the arguments with the species
> > > contained within selectedDatabaseprofile.selectedSpeciesProfiles.
> > > >
> > > > I think I'll start with the baseArgument up to MODSystem, then
> > > append as many species as necessary, and then cap off the end with
> > > the rest starting at Modify. (similar to your approach in
> > > ExportGoData, populateUniprotGoTableFromSQL(char chosenAspect,
> > > List<Integer> taxonIds) line 513
> > > >
> > > > Please let me know if this approach or analysis is off track.
> > > >
> > > > Thanks!
> > > >
> > > > Richard
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Wed, Aug 10, 2011 at 5:50 PM, Richard Brous <rbr...@gm...>
> wrote:
> > > > Updated repository to include all Gene Ontology changes discussed
> > > during our meeting yesterday.
> > > >
> > > > Digging into TableManager next.
> > > >
> > > > Richard
> > > >
> > > > On Fri, Aug 5, 2011 at 10:06 AM, Richard Brous <rbr...@gm...>
> wrote:
> > > > whew... thanks for the detailed reply. I will digest this a bit
> > > and get back to you with further questions.
> > > >
> > > > rb
> > > >
> > > > On Thu, Aug 4, 2011 at 11:18 PM, John David N. Dionisio
> > > <do...@lm...> wrote:
> > > > Greetings,
> > > >
> > > > Sorry for the delay. I wasn't able to walk through the relevant
> > > code until this evening.
> > > >
> > > > As Kam said, GOA serves as the link between the UniProt and GO
> > > IDs. It essentially determines which GO IDs get exported by using
> > > GOA to see which GO IDs are associated with an exported UniProt
> > > ID. The populateUniprotGoTableFromSQL, in its current form,
> > > extracts the GO association records that match the given taxon ID
> > > then exports, as UniProt-GO pairs, the GO and UniProt IDs
> > > referenced within that GO association record. Processing that
> > > follows this is then based on the GO IDs that got exported --- and
> > > that's how the current code avoids exporting the entire list of GO
> terms.
> > > >
> > > > The operative query is on the second line of
> populateUniprotGoTableFromSQL:
> > > >
> > > > String uniProtAndGOIDSQL = "select db_object_id, go_id,
> > > evidence_code, with_or_from from goa where db like '%UniProt%' and
> > > taxon = 'taxon:" + taxon + "'";
> > > >
> > > > In plain English, this selects the GOA records whose database is
> > > UniProt and whose taxon ID is the given taxon. An additional
> > > condition is added for the "aspect" (All, Component, Function, or
> > > Process) that is to be exported. This is another reduction filter,
> > > to further shrink the number of exported GO terms and thus avoid
> > > MAPPFinder issues later on.
> > > >
> > > > Given this, the proper expansion here is to change the taxon
> > > predicate to a multiple predicate. That is, this method can be
> > > changed to now accept a collection or array of taxon IDs, and the
> > > base query should then be changed so that it accepts any taxon from
> > > that collection. More or less, you want:
> > > >
> > > > private void populateUniprotGoTableFromSQL(char chosenAspect,
> > > int[] taxons) throws SQLException {
> > > >
> > > > ...then, instead of the single string, you want to iterate
> > > through the taxon IDs:
> > > >
> > > > StringBuilder baseQueryBuilder = new StringBuilder("select
> > > db_object_id, go_id, evidence_code, with_or_from from goa where db
> > > like '%UniProt%'");
> > > > boolean first = true;
> > > > for (int taxon: taxons) {
> > > > baseQueryBuilder.append(first ? " and (" : " or ");
> > > > baseQueryBuilder
> > > > .append("taxon = 'taxon:")
> > > > .append(taxon).append("'");
> > > > first = false;
> > > > }
> > > > baseQueryBuilder.append(")");
> > > >
> > > > ...and so on. I just sort of rattled this off so there may be
> > > little glitches, but anyway this is just to give you an overall idea.
> > > >
> > > > Put another way, no, you do not need to iterate this method for
> > > each taxon ID. Instead, you can still call this method once, with
> > > the multiplicity of taxon IDs emerging in terms of the actual
> > > condition used for selecting the GO terms to be exported (based on
> > > the available GOA records, which as you may recall are loaded from .goa
> files).
> > > >
> > > > As a side note, right here you have an opportunity for a little
> > > sanity check regarding the content of the relational database: GO
> > > terms will only be exported if GOA records for the desired taxon
> > > IDs have been imported into the database. So, as a pre-flight
> > > check, one can see if there are any GOA records at all for each
> > > chosen taxon ID. If there are none, then the .goa file for that
> > > species needs to be imported into the relational database.
> > > >
> > > > Hope this helps...
> > > >
> > > > John David N. Dionisio, PhD
> > > > Associate Professor, Computer Science
> > > > Loyola Marymount University
> > > >
> > > >
> > > > On Aug 4, 2011, at 1:00 PM, Kam Dahlquist wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > Dondi will have to chime in on this, but I think this is where
> > > things are going to get tricky.
> > > > >
> > > > > The final gdb does not actually contain the entire GO, it gets
> > > trimmed somehow based on the GO associations for a particular
> > > species. This is because MAPPFinder cannot handle loading the
> > > entire GO. Since there is some type of species-specific trimming
> > > going on, it's quite possible that this will need to iterate.
> > > > >
> > > > > However, I don't have the foggiest idea of how this works, so
> > > Dondi will have to chime in.
> > > > >
> > > > > Best,
> > > > > Kam
> > > > >
> > > > > At 12:09 AM 8/4/2011, you wrote:
> > > > >> Wednesday 8/3/11 progress:
> > > > >>
> > > > >> 1. After following the ExportPanel1.java ground zero code of:
> > > databaseProfile.setSelectedSpeciesProfile( selectedProfile );
> > > > >>
> > > > >> I found the method in DatabaseProfile.java plus a getter method;
> > > > >> SpeciesProfile setSelectedSpeciesProfile( speciesProfile ) and
> > > SpeciesProfile getSelectedSpeciesProfile( speciesProfile )
> > > > >>
> > > > >> I created two new methods that each handle List<Object> of
> > > SpeciesProfiles argument instead of a single SpeciesProfile;
> > > setSelectedSpeciesProfiles and getSelectedSpeciesProfiles.
> > > > >>
> > > > >> This enabled the ExportPanel1 ground zero code to become:
> > > databaseProfile.setSelectedSpeciesProfiles(selectedSpecies);
> > > > >>
> > > > >> 2. public static void export() on line 104 in ExportToGenMAPP.java
> > > > >>
> > > > >> On line 107 ExportGoData is instantiated which I found in
> > > ExportGoData.java and calls a method: public void export(char
> > > chosenAspect, int taxon).
> > > > >>
> > > > >> Within export, taxon id is required for another method:
> > > private void populateGoTables(char chosenAspect, int taxon).
> > > > >>
> > > > >> Within populateGoTables, taxon id is required for another
> > > method: private void populateUniprotGoTableFromSQL( char
> > > chosenAspect, int taxon).
> > > > >>
> > > > >> But, if the export to GDB process starts off with exporting GO
> > > data, doesn't it only need to do that once no matter how many
> > > species are selected? As you probably realize, I'm leading towards
> > > not having to iterate through this for each taxon id if possible.
> > > > >>
> > > > >> Also, how does the export actually work? How are GO ids and
> > > UniProt ids related within the table?
> > > > >>
> > > > >> Thanks!
> > > > >>
> > > > >> Richard
> > > > >>
> > > > >>
> > > > > <ATT00001..txt><ATT00002..txt>
> > > >
> > > >
> > > >
> > >
> ------------------------------------------------------------------------------
> > > > BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
> > > > The must-attend event for mobile developers. Connect with experts.
> > > > Get tools for creating Super Apps. See the latest technologies.
> > > > Sessions, hands-on labs, demos & much more. Register early & save!
> > > > http://p.sf.net/sfu/rim-blackberry-1
> > > > _______________________________________________
> > > > xmlpipedb-developer mailing list
> > > > xml...@li...
> > > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> > > >
> > > >
> > > >
> > > > <ATT00001..txt><ATT00002..txt>
> > >
> > >
> >
> >------------------------------------------------------------------------------
> > >Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
> > >user administration capabilities and model configuration. Take
> > >the hassle out of deploying and managing Subversion and the
> > >tools developers use with it.
> > >http://p.sf.net/sfu/wandisco-dev2dev
> > >_______________________________________________
> > >xmlpipedb-developer mailing list
> > >xml...@li...
> > >https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> >
> >
> >
> ------------------------------------------------------------------------------
> > FREE DOWNLOAD - uberSVN with Social Coding for Subversion.
> > Subversion made easy with a complete admin console. Easy
> > to use, easy to manage, easy to install, easy to extend.
> > Get a Free download of the new open ALM Subversion platform now.
> > http://p.sf.net/sfu/wandisco-dev2dev
> > _______________________________________________
> > xmlpipedb-developer mailing list
> > xml...@li...
> > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> >
> > <ATT00001..txt><ATT00002..txt>
>
>
>
> ------------------------------------------------------------------------------
> uberSVN's rich system and user administration capabilities and model
> configuration take the hassle out of deploying and managing Subversion and
> the tools developers use with it. Learn more about uberSVN and get a free
> download at: http://p.sf.net/sfu/wandisco-dev2dev
> _______________________________________________
> xmlpipedb-developer mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>
|
|
From: John D. N. D. <do...@lm...> - 2011-08-15 08:30:47
|
OK, everything is committed. I got up to the tweaks on the second panel in the wizard (Save As/GO Aspects), as well as the query change for exporting any combination of C, F, or P.
I still have to do the GO OBO format check, plus the UI work on the remaining two wizard panels. But meanwhile, hope these latest changes work out well.
John David N. Dionisio, PhD
Associate Professor, Computer Science
Loyola Marymount University
On Aug 14, 2011, at 7:31 PM, Richard Brous wrote:
> OK, using option 2 I have made changes to DatabaseProfile.java to allow for all species names to be included in the submit argument.
> But I'm stuck on how to change my StringBuilder object to a type that submit wants. Help please!!
>
> Submitted the above and some comment changes to SourceForge this evening.
>
> Richard
>
> On Fri, Aug 12, 2011 at 12:08 PM, Kam Dahlquist <kda...@lm...> wrote:
> Hi,
>
> I think we had better leave the info table with only one record
> (option 2). The species names can be separated by pipes " | " as
> they are in the Systems table where there are multiple species. To
> my knowledge, the only time GenMAPP needs to access the info table is
> for the "DisplayOrder" field, I don't know what would happen if there
> were multiple records there. I know that the spec for the table says
> that it should only be one record, but I don't know if it would crash
> if there were multiple records. To be on the safe side, I think we
> should just keep it to the one record.
>
> Best,
> Kam
>
> At 09:45 PM 8/10/2011, John David N. Dionisio wrote:
> >Greetings,
> >
> >I think we have to turn to Dr. Dahlquist's GenMAPP knowledge here to
> >get the definitive answer. I see two choices:
> >
> >- The Info table should have one record for each species that the
> >.gdb holds, in which case the change you need is to wrap that single
> >submit call inside a loop, so that submit is called once for each
> >chosen species.
> >
> >- The Info table should always have one record, and if the .gdb
> >holds multiple species, the "Species" column should be some
> >concatenation of multiple species names. In this case, you would
> >still call submit only once, but the value you send into the
> >"Species" column is some accumulation of all chosen species names.
> >
> >Admittedly I don't know which way is right (I assumed the former as
> >of our Tuesday meeting, but on further examination I'm no longer
> >quite so sure).
> >
> >For Kam --- what does GenMAPP expect to see in the Info table if the
> >opened .gdb contains multiple species?
> >
> >John David N. Dionisio, PhD
> >Associate Professor, Computer Science
> >Loyola Marymount University
> >
> >
> >On Aug 10, 2011, at 9:36 PM, Richard Brous wrote:
> >
> > > OK, continued to review ExportToGenMAPP and dug into the creation
> > of the first TableManager tmA on line 118.
> > >
> > > In reading through the method, my understanding is that it
> > creates a new TableManager based on the selectedDatabaseProfile
> > (which is UniProt).
> > >
> > > This is performed by the method getInfoTableManager() which then
> > calls method submit(String tableName, QueryType queryType,
> > String[][] columnNamesToValues);
> > >
> > > the code is as follows:
> > >
> > > tableManager.submit("Info", QueryType.insert, new String[][] { {
> > "Owner", owner }, { "Version", new
> > SimpleDateFormat("yyyyMMdd").format(version) }, { "MODSystem",
> > modSystem }, { "Species", speciesProfile.getSpeciesName() }, {
> > "Modify", new SimpleDateFormat("yyyyMMdd").format(modify) }, {
> > "DisplayOrder", displayOrder }, { "Notes", notes } });
> > >
> > >
> > > The modification of this line centers on { "Species",
> > speciesProfile.getSpeciesName() }, since it originally processed a
> > single species.
> > >
> > > So now I need to populate the arguments with the species
> > contained within selectedDatabaseprofile.selectedSpeciesProfiles.
> > >
> > > I think I'll start with the baseArgument up to MODSystem, then
> > append as many species as necessary, and then cap off the end with
> > the rest starting at Modify. (similar to your approach in
> > ExportGoData, populateUniprotGoTableFromSQL(char chosenAspect,
> > List<Integer> taxonIds) line 513
> > >
> > > Please let me know if this approach or analysis is off track.
> > >
> > > Thanks!
> > >
> > > Richard
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Wed, Aug 10, 2011 at 5:50 PM, Richard Brous <rbr...@gm...> wrote:
> > > Updated repository to include all Gene Ontology changes discussed
> > during our meeting yesterday.
> > >
> > > Digging into TableManager next.
> > >
> > > Richard
> > >
> > > On Fri, Aug 5, 2011 at 10:06 AM, Richard Brous <rbr...@gm...> wrote:
> > > whew... thanks for the detailed reply. I will digest this a bit
> > and get back to you with further questions.
> > >
> > > rb
> > >
> > > On Thu, Aug 4, 2011 at 11:18 PM, John David N. Dionisio
> > <do...@lm...> wrote:
> > > Greetings,
> > >
> > > Sorry for the delay. I wasn't able to walk through the relevant
> > code until this evening.
> > >
> > > As Kam said, GOA serves as the link between the UniProt and GO
> > IDs. It essentially determines which GO IDs get exported by using
> > GOA to see which GO IDs are associated with an exported UniProt
> > ID. The populateUniprotGoTableFromSQL, in its current form,
> > extracts the GO association records that match the given taxon ID
> > then exports, as UniProt-GO pairs, the GO and UniProt IDs
> > referenced within that GO association record. Processing that
> > follows this is then based on the GO IDs that got exported --- and
> > that's how the current code avoids exporting the entire list of GO terms.
> > >
> > > The operative query is on the second line of populateUniprotGoTableFromSQL:
> > >
> > > String uniProtAndGOIDSQL = "select db_object_id, go_id,
> > evidence_code, with_or_from from goa where db like '%UniProt%' and
> > taxon = 'taxon:" + taxon + "'";
> > >
> > > In plain English, this selects the GOA records whose database is
> > UniProt and whose taxon ID is the given taxon. An additional
> > condition is added for the "aspect" (All, Component, Function, or
> > Process) that is to be exported. This is another reduction filter,
> > to further shrink the number of exported GO terms and thus avoid
> > MAPPFinder issues later on.
> > >
> > > Given this, the proper expansion here is to change the taxon
> > predicate to a multiple predicate. That is, this method can be
> > changed to now accept a collection or array of taxon IDs, and the
> > base query should then be changed so that it accepts any taxon from
> > that collection. More or less, you want:
> > >
> > > private void populateUniprotGoTableFromSQL(char chosenAspect,
> > int[] taxons) throws SQLException {
> > >
> > > ...then, instead of the single string, you want to iterate
> > through the taxon IDs:
> > >
> > > StringBuilder baseQueryBuilder = new StringBuilder("select
> > db_object_id, go_id, evidence_code, with_or_from from goa where db
> > like '%UniProt%'");
> > > boolean first = true;
> > > for (int taxon: taxons) {
> > > baseQueryBuilder.append(first ? " and (" : " or ");
> > > baseQueryBuilder
> > > .append("taxon = 'taxon:")
> > > .append(taxon).append("'");
> > > first = false;
> > > }
> > > baseQueryBuilder.append(")");
> > >
> > > ...and so on. I just sort of rattled this off so there may be
> > little glitches, but anyway this is just to give you an overall idea.
> > >
> > > Put another way, no, you do not need to iterate this method for
> > each taxon ID. Instead, you can still call this method once, with
> > the multiplicity of taxon IDs emerging in terms of the actual
> > condition used for selecting the GO terms to be exported (based on
> > the available GOA records, which as you may recall are loaded from .goa files).
> > >
> > > As a side note, right here you have an opportunity for a little
> > sanity check regarding the content of the relational database: GO
> > terms will only be exported if GOA records for the desired taxon
> > IDs have been imported into the database. So, as a pre-flight
> > check, one can see if there are any GOA records at all for each
> > chosen taxon ID. If there are none, then the .goa file for that
> > species needs to be imported into the relational database.
> > >
> > > Hope this helps...
> > >
> > > John David N. Dionisio, PhD
> > > Associate Professor, Computer Science
> > > Loyola Marymount University
> > >
> > >
> > > On Aug 4, 2011, at 1:00 PM, Kam Dahlquist wrote:
> > >
> > > > Hi,
> > > >
> > > > Dondi will have to chime in on this, but I think this is where
> > things are going to get tricky.
> > > >
> > > > The final gdb does not actually contain the entire GO, it gets
> > trimmed somehow based on the GO associations for a particular
> > species. This is because MAPPFinder cannot handle loading the
> > entire GO. Since there is some type of species-specific trimming
> > going on, it's quite possible that this will need to iterate.
> > > >
> > > > However, I don't have the foggiest idea of how this works, so
> > Dondi will have to chime in.
> > > >
> > > > Best,
> > > > Kam
> > > >
> > > > At 12:09 AM 8/4/2011, you wrote:
> > > >> Wednesday 8/3/11 progress:
> > > >>
> > > >> 1. After following the ExportPanel1.java ground zero code of:
> > databaseProfile.setSelectedSpeciesProfile( selectedProfile );
> > > >>
> > > >> I found the method in DatabaseProfile.java plus a getter method;
> > > >> SpeciesProfile setSelectedSpeciesProfile( speciesProfile ) and
> > SpeciesProfile getSelectedSpeciesProfile( speciesProfile )
> > > >>
> > > >> I created two new methods that each handle List<Object> of
> > SpeciesProfiles argument instead of a single SpeciesProfile;
> > setSelectedSpeciesProfiles and getSelectedSpeciesProfiles.
> > > >>
> > > >> This enabled the ExportPanel1 ground zero code to become:
> > databaseProfile.setSelectedSpeciesProfiles(selectedSpecies);
> > > >>
> > > >> 2. public static void export() on line 104 in ExportToGenMAPP.java
> > > >>
> > > >> On line 107 ExportGoData is instantiated which I found in
> > ExportGoData.java and calls a method: public void export(char
> > chosenAspect, int taxon).
> > > >>
> > > >> Within export, taxon id is required for another method:
> > private void populateGoTables(char chosenAspect, int taxon).
> > > >>
> > > >> Within populateGoTables, taxon id is required for another
> > method: private void populateUniprotGoTableFromSQL( char
> > chosenAspect, int taxon).
> > > >>
> > > >> But, if the export to GDB process starts off with exporting GO
> > data, doesn't it only need to do that once no matter how many
> > species are selected? As you probably realize, I'm leading towards
> > not having to iterate through this for each taxon id if possible.
> > > >>
> > > >> Also, how does the export actually work? How are GO ids and
> > UniProt ids related within the table?
> > > >>
> > > >> Thanks!
> > > >>
> > > >> Richard
> > > >>
> > > >>
> > > > <ATT00001..txt><ATT00002..txt>
> > >
> > >
> > >
> > ------------------------------------------------------------------------------
> > > BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
> > > The must-attend event for mobile developers. Connect with experts.
> > > Get tools for creating Super Apps. See the latest technologies.
> > > Sessions, hands-on labs, demos & much more. Register early & save!
> > > http://p.sf.net/sfu/rim-blackberry-1
> > > _______________________________________________
> > > xmlpipedb-developer mailing list
> > > xml...@li...
> > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> > >
> > >
> > >
> > > <ATT00001..txt><ATT00002..txt>
> >
> >
> >------------------------------------------------------------------------------
> >Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
> >user administration capabilities and model configuration. Take
> >the hassle out of deploying and managing Subversion and the
> >tools developers use with it.
> >http://p.sf.net/sfu/wandisco-dev2dev
> >_______________________________________________
> >xmlpipedb-developer mailing list
> >xml...@li...
> >https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>
>
> ------------------------------------------------------------------------------
> FREE DOWNLOAD - uberSVN with Social Coding for Subversion.
> Subversion made easy with a complete admin console. Easy
> to use, easy to manage, easy to install, easy to extend.
> Get a Free download of the new open ALM Subversion platform now.
> http://p.sf.net/sfu/wandisco-dev2dev
> _______________________________________________
> xmlpipedb-developer mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>
> <ATT00001..txt><ATT00002..txt>
|
|
From: John D. N. D. <do...@lm...> - 2011-08-15 04:41:58
|
Greetings,
I've taken a look at your initial Info multispecies expansion and the long and the short of it is that same issue I mentioned before: make sure that you are clear, on the syntactic level, about what the existing code is doing. The TableManager.submit method that you are trying to replace has *multiple* parameters, *only one of which* holds the species string that is to be replaced with the new, concatenated version. Your refactor attempt has lost sight of that, and is attempting to call submit with a single argument. So remember, always take a step back and ascertain that you are clear on what the prior code is doing, before moving on to the change. Version control is a tool here as well: if you have edited over the old code, and need to review it, then just call up the prior committed version.
I plan to fix this later tonight, plus get to the CFP and wizard-panel-merging stuff also, but I thought I'd let you know right off what the core issue was in case you want to take a look and see if you can "anticipate" the changes that will be made to get this working.
John David N. Dionisio, PhD
Associate Professor, Computer Science
Loyola Marymount University
On Aug 14, 2011, at 7:31 PM, Richard Brous wrote:
> OK, using option 2 I have made changes to DatabaseProfile.java to allow for all species names to be included in the submit argument.
> But I'm stuck on how to change my StringBuilder object to a type that submit wants. Help please!!
>
> Submitted the above and some comment changes to SourceForge this evening.
>
> Richard
>
> On Fri, Aug 12, 2011 at 12:08 PM, Kam Dahlquist <kda...@lm...> wrote:
> Hi,
>
> I think we had better leave the info table with only one record
> (option 2). The species names can be separated by pipes " | " as
> they are in the Systems table where there are multiple species. To
> my knowledge, the only time GenMAPP needs to access the info table is
> for the "DisplayOrder" field, I don't know what would happen if there
> were multiple records there. I know that the spec for the table says
> that it should only be one record, but I don't know if it would crash
> if there were multiple records. To be on the safe side, I think we
> should just keep it to the one record.
>
> Best,
> Kam
>
> At 09:45 PM 8/10/2011, John David N. Dionisio wrote:
> >Greetings,
> >
> >I think we have to turn to Dr. Dahlquist's GenMAPP knowledge here to
> >get the definitive answer. I see two choices:
> >
> >- The Info table should have one record for each species that the
> >.gdb holds, in which case the change you need is to wrap that single
> >submit call inside a loop, so that submit is called once for each
> >chosen species.
> >
> >- The Info table should always have one record, and if the .gdb
> >holds multiple species, the "Species" column should be some
> >concatenation of multiple species names. In this case, you would
> >still call submit only once, but the value you send into the
> >"Species" column is some accumulation of all chosen species names.
> >
> >Admittedly I don't know which way is right (I assumed the former as
> >of our Tuesday meeting, but on further examination I'm no longer
> >quite so sure).
> >
> >For Kam --- what does GenMAPP expect to see in the Info table if the
> >opened .gdb contains multiple species?
> >
> >John David N. Dionisio, PhD
> >Associate Professor, Computer Science
> >Loyola Marymount University
> >
> >
> >On Aug 10, 2011, at 9:36 PM, Richard Brous wrote:
> >
> > > OK, continued to review ExportToGenMAPP and dug into the creation
> > of the first TableManager tmA on line 118.
> > >
> > > In reading through the method, my understanding is that it
> > creates a new TableManager based on the selectedDatabaseProfile
> > (which is UniProt).
> > >
> > > This is performed by the method getInfoTableManager() which then
> > calls method submit(String tableName, QueryType queryType,
> > String[][] columnNamesToValues);
> > >
> > > the code is as follows:
> > >
> > > tableManager.submit("Info", QueryType.insert, new String[][] { {
> > "Owner", owner }, { "Version", new
> > SimpleDateFormat("yyyyMMdd").format(version) }, { "MODSystem",
> > modSystem }, { "Species", speciesProfile.getSpeciesName() }, {
> > "Modify", new SimpleDateFormat("yyyyMMdd").format(modify) }, {
> > "DisplayOrder", displayOrder }, { "Notes", notes } });
> > >
> > >
> > > The modification of this line centers on { "Species",
> > speciesProfile.getSpeciesName() }, since it originally processed a
> > single species.
> > >
> > > So now I need to populate the arguments with the species
> > contained within selectedDatabaseprofile.selectedSpeciesProfiles.
> > >
> > > I think I'll start with the baseArgument up to MODSystem, then
> > append as many species as necessary, and then cap off the end with
> > the rest starting at Modify. (similar to your approach in
> > ExportGoData, populateUniprotGoTableFromSQL(char chosenAspect,
> > List<Integer> taxonIds) line 513
> > >
> > > Please let me know if this approach or analysis is off track.
> > >
> > > Thanks!
> > >
> > > Richard
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Wed, Aug 10, 2011 at 5:50 PM, Richard Brous <rbr...@gm...> wrote:
> > > Updated repository to include all Gene Ontology changes discussed
> > during our meeting yesterday.
> > >
> > > Digging into TableManager next.
> > >
> > > Richard
> > >
> > > On Fri, Aug 5, 2011 at 10:06 AM, Richard Brous <rbr...@gm...> wrote:
> > > whew... thanks for the detailed reply. I will digest this a bit
> > and get back to you with further questions.
> > >
> > > rb
> > >
> > > On Thu, Aug 4, 2011 at 11:18 PM, John David N. Dionisio
> > <do...@lm...> wrote:
> > > Greetings,
> > >
> > > Sorry for the delay. I wasn't able to walk through the relevant
> > code until this evening.
> > >
> > > As Kam said, GOA serves as the link between the UniProt and GO
> > IDs. It essentially determines which GO IDs get exported by using
> > GOA to see which GO IDs are associated with an exported UniProt
> > ID. The populateUniprotGoTableFromSQL, in its current form,
> > extracts the GO association records that match the given taxon ID
> > then exports, as UniProt-GO pairs, the GO and UniProt IDs
> > referenced within that GO association record. Processing that
> > follows this is then based on the GO IDs that got exported --- and
> > that's how the current code avoids exporting the entire list of GO terms.
> > >
> > > The operative query is on the second line of populateUniprotGoTableFromSQL:
> > >
> > > String uniProtAndGOIDSQL = "select db_object_id, go_id,
> > evidence_code, with_or_from from goa where db like '%UniProt%' and
> > taxon = 'taxon:" + taxon + "'";
> > >
> > > In plain English, this selects the GOA records whose database is
> > UniProt and whose taxon ID is the given taxon. An additional
> > condition is added for the "aspect" (All, Component, Function, or
> > Process) that is to be exported. This is another reduction filter,
> > to further shrink the number of exported GO terms and thus avoid
> > MAPPFinder issues later on.
> > >
> > > Given this, the proper expansion here is to change the taxon
> > predicate to a multiple predicate. That is, this method can be
> > changed to now accept a collection or array of taxon IDs, and the
> > base query should then be changed so that it accepts any taxon from
> > that collection. More or less, you want:
> > >
> > > private void populateUniprotGoTableFromSQL(char chosenAspect,
> > int[] taxons) throws SQLException {
> > >
> > > ...then, instead of the single string, you want to iterate
> > through the taxon IDs:
> > >
> > > StringBuilder baseQueryBuilder = new StringBuilder("select
> > db_object_id, go_id, evidence_code, with_or_from from goa where db
> > like '%UniProt%'");
> > > boolean first = true;
> > > for (int taxon: taxons) {
> > > baseQueryBuilder.append(first ? " and (" : " or ");
> > > baseQueryBuilder
> > > .append("taxon = 'taxon:")
> > > .append(taxon).append("'");
> > > first = false;
> > > }
> > > baseQueryBuilder.append(")");
> > >
> > > ...and so on. I just sort of rattled this off so there may be
> > little glitches, but anyway this is just to give you an overall idea.
> > >
> > > Put another way, no, you do not need to iterate this method for
> > each taxon ID. Instead, you can still call this method once, with
> > the multiplicity of taxon IDs emerging in terms of the actual
> > condition used for selecting the GO terms to be exported (based on
> > the available GOA records, which as you may recall are loaded from .goa files).
> > >
> > > As a side note, right here you have an opportunity for a little
> > sanity check regarding the content of the relational database: GO
> > terms will only be exported if GOA records for the desired taxon
> > IDs have been imported into the database. So, as a pre-flight
> > check, one can see if there are any GOA records at all for each
> > chosen taxon ID. If there are none, then the .goa file for that
> > species needs to be imported into the relational database.
> > >
> > > Hope this helps...
> > >
> > > John David N. Dionisio, PhD
> > > Associate Professor, Computer Science
> > > Loyola Marymount University
> > >
> > >
> > > On Aug 4, 2011, at 1:00 PM, Kam Dahlquist wrote:
> > >
> > > > Hi,
> > > >
> > > > Dondi will have to chime in on this, but I think this is where
> > things are going to get tricky.
> > > >
> > > > The final gdb does not actually contain the entire GO, it gets
> > trimmed somehow based on the GO associations for a particular
> > species. This is because MAPPFinder cannot handle loading the
> > entire GO. Since there is some type of species-specific trimming
> > going on, it's quite possible that this will need to iterate.
> > > >
> > > > However, I don't have the foggiest idea of how this works, so
> > Dondi will have to chime in.
> > > >
> > > > Best,
> > > > Kam
> > > >
> > > > At 12:09 AM 8/4/2011, you wrote:
> > > >> Wednesday 8/3/11 progress:
> > > >>
> > > >> 1. After following the ExportPanel1.java ground zero code of:
> > databaseProfile.setSelectedSpeciesProfile( selectedProfile );
> > > >>
> > > >> I found the method in DatabaseProfile.java plus a getter method;
> > > >> SpeciesProfile setSelectedSpeciesProfile( speciesProfile ) and
> > SpeciesProfile getSelectedSpeciesProfile( speciesProfile )
> > > >>
> > > >> I created two new methods that each handle List<Object> of
> > SpeciesProfiles argument instead of a single SpeciesProfile;
> > setSelectedSpeciesProfiles and getSelectedSpeciesProfiles.
> > > >>
> > > >> This enabled the ExportPanel1 ground zero code to become:
> > databaseProfile.setSelectedSpeciesProfiles(selectedSpecies);
> > > >>
> > > >> 2. public static void export() on line 104 in ExportToGenMAPP.java
> > > >>
> > > >> On line 107 ExportGoData is instantiated which I found in
> > ExportGoData.java and calls a method: public void export(char
> > chosenAspect, int taxon).
> > > >>
> > > >> Within export, taxon id is required for another method:
> > private void populateGoTables(char chosenAspect, int taxon).
> > > >>
> > > >> Within populateGoTables, taxon id is required for another
> > method: private void populateUniprotGoTableFromSQL( char
> > chosenAspect, int taxon).
> > > >>
> > > >> But, if the export to GDB process starts off with exporting GO
> > data, doesn't it only need to do that once no matter how many
> > species are selected? As you probably realize, I'm leading towards
> > not having to iterate through this for each taxon id if possible.
> > > >>
> > > >> Also, how does the export actually work? How are GO ids and
> > UniProt ids related within the table?
> > > >>
> > > >> Thanks!
> > > >>
> > > >> Richard
> > > >>
> > > >>
> > > > <ATT00001..txt><ATT00002..txt>
> > >
> > >
> > >
> > ------------------------------------------------------------------------------
> > > BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
> > > The must-attend event for mobile developers. Connect with experts.
> > > Get tools for creating Super Apps. See the latest technologies.
> > > Sessions, hands-on labs, demos & much more. Register early & save!
> > > http://p.sf.net/sfu/rim-blackberry-1
> > > _______________________________________________
> > > xmlpipedb-developer mailing list
> > > xml...@li...
> > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> > >
> > >
> > >
> > > <ATT00001..txt><ATT00002..txt>
> >
> >
> >------------------------------------------------------------------------------
> >Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
> >user administration capabilities and model configuration. Take
> >the hassle out of deploying and managing Subversion and the
> >tools developers use with it.
> >http://p.sf.net/sfu/wandisco-dev2dev
> >_______________________________________________
> >xmlpipedb-developer mailing list
> >xml...@li...
> >https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>
>
> ------------------------------------------------------------------------------
> FREE DOWNLOAD - uberSVN with Social Coding for Subversion.
> Subversion made easy with a complete admin console. Easy
> to use, easy to manage, easy to install, easy to extend.
> Get a Free download of the new open ALM Subversion platform now.
> http://p.sf.net/sfu/wandisco-dev2dev
> _______________________________________________
> xmlpipedb-developer mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>
> <ATT00001..txt><ATT00002..txt>
|
|
From: Richard B. <rbr...@gm...> - 2011-08-15 02:31:23
|
OK, using option 2 I have made changes to DatabaseProfile.java to allow for
all species names to be included in the submit argument.
But I'm stuck on how to change my StringBuilder object to a type that submit
wants. Help please!!
Submitted the above and some comment changes to SourceForge this evening.
Richard
On Fri, Aug 12, 2011 at 12:08 PM, Kam Dahlquist <kda...@lm...> wrote:
> Hi,
>
> I think we had better leave the info table with only one record
> (option 2). The species names can be separated by pipes " | " as
> they are in the Systems table where there are multiple species. To
> my knowledge, the only time GenMAPP needs to access the info table is
> for the "DisplayOrder" field, I don't know what would happen if there
> were multiple records there. I know that the spec for the table says
> that it should only be one record, but I don't know if it would crash
> if there were multiple records. To be on the safe side, I think we
> should just keep it to the one record.
>
> Best,
> Kam
>
> At 09:45 PM 8/10/2011, John David N. Dionisio wrote:
> >Greetings,
> >
> >I think we have to turn to Dr. Dahlquist's GenMAPP knowledge here to
> >get the definitive answer. I see two choices:
> >
> >- The Info table should have one record for each species that the
> >.gdb holds, in which case the change you need is to wrap that single
> >submit call inside a loop, so that submit is called once for each
> >chosen species.
> >
> >- The Info table should always have one record, and if the .gdb
> >holds multiple species, the "Species" column should be some
> >concatenation of multiple species names. In this case, you would
> >still call submit only once, but the value you send into the
> >"Species" column is some accumulation of all chosen species names.
> >
> >Admittedly I don't know which way is right (I assumed the former as
> >of our Tuesday meeting, but on further examination I'm no longer
> >quite so sure).
> >
> >For Kam --- what does GenMAPP expect to see in the Info table if the
> >opened .gdb contains multiple species?
> >
> >John David N. Dionisio, PhD
> >Associate Professor, Computer Science
> >Loyola Marymount University
> >
> >
> >On Aug 10, 2011, at 9:36 PM, Richard Brous wrote:
> >
> > > OK, continued to review ExportToGenMAPP and dug into the creation
> > of the first TableManager tmA on line 118.
> > >
> > > In reading through the method, my understanding is that it
> > creates a new TableManager based on the selectedDatabaseProfile
> > (which is UniProt).
> > >
> > > This is performed by the method getInfoTableManager() which then
> > calls method submit(String tableName, QueryType queryType,
> > String[][] columnNamesToValues);
> > >
> > > the code is as follows:
> > >
> > > tableManager.submit("Info", QueryType.insert, new String[][] { {
> > "Owner", owner }, { "Version", new
> > SimpleDateFormat("yyyyMMdd").format(version) }, { "MODSystem",
> > modSystem }, { "Species", speciesProfile.getSpeciesName() }, {
> > "Modify", new SimpleDateFormat("yyyyMMdd").format(modify) }, {
> > "DisplayOrder", displayOrder }, { "Notes", notes } });
> > >
> > >
> > > The modification of this line centers on { "Species",
> > speciesProfile.getSpeciesName() }, since it originally processed a
> > single species.
> > >
> > > So now I need to populate the arguments with the species
> > contained within selectedDatabaseprofile.selectedSpeciesProfiles.
> > >
> > > I think I'll start with the baseArgument up to MODSystem, then
> > append as many species as necessary, and then cap off the end with
> > the rest starting at Modify. (similar to your approach in
> > ExportGoData, populateUniprotGoTableFromSQL(char chosenAspect,
> > List<Integer> taxonIds) line 513
> > >
> > > Please let me know if this approach or analysis is off track.
> > >
> > > Thanks!
> > >
> > > Richard
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Wed, Aug 10, 2011 at 5:50 PM, Richard Brous <rbr...@gm...>
> wrote:
> > > Updated repository to include all Gene Ontology changes discussed
> > during our meeting yesterday.
> > >
> > > Digging into TableManager next.
> > >
> > > Richard
> > >
> > > On Fri, Aug 5, 2011 at 10:06 AM, Richard Brous <rbr...@gm...>
> wrote:
> > > whew... thanks for the detailed reply. I will digest this a bit
> > and get back to you with further questions.
> > >
> > > rb
> > >
> > > On Thu, Aug 4, 2011 at 11:18 PM, John David N. Dionisio
> > <do...@lm...> wrote:
> > > Greetings,
> > >
> > > Sorry for the delay. I wasn't able to walk through the relevant
> > code until this evening.
> > >
> > > As Kam said, GOA serves as the link between the UniProt and GO
> > IDs. It essentially determines which GO IDs get exported by using
> > GOA to see which GO IDs are associated with an exported UniProt
> > ID. The populateUniprotGoTableFromSQL, in its current form,
> > extracts the GO association records that match the given taxon ID
> > then exports, as UniProt-GO pairs, the GO and UniProt IDs
> > referenced within that GO association record. Processing that
> > follows this is then based on the GO IDs that got exported --- and
> > that's how the current code avoids exporting the entire list of GO terms.
> > >
> > > The operative query is on the second line of
> populateUniprotGoTableFromSQL:
> > >
> > > String uniProtAndGOIDSQL = "select db_object_id, go_id,
> > evidence_code, with_or_from from goa where db like '%UniProt%' and
> > taxon = 'taxon:" + taxon + "'";
> > >
> > > In plain English, this selects the GOA records whose database is
> > UniProt and whose taxon ID is the given taxon. An additional
> > condition is added for the "aspect" (All, Component, Function, or
> > Process) that is to be exported. This is another reduction filter,
> > to further shrink the number of exported GO terms and thus avoid
> > MAPPFinder issues later on.
> > >
> > > Given this, the proper expansion here is to change the taxon
> > predicate to a multiple predicate. That is, this method can be
> > changed to now accept a collection or array of taxon IDs, and the
> > base query should then be changed so that it accepts any taxon from
> > that collection. More or less, you want:
> > >
> > > private void populateUniprotGoTableFromSQL(char chosenAspect,
> > int[] taxons) throws SQLException {
> > >
> > > ...then, instead of the single string, you want to iterate
> > through the taxon IDs:
> > >
> > > StringBuilder baseQueryBuilder = new StringBuilder("select
> > db_object_id, go_id, evidence_code, with_or_from from goa where db
> > like '%UniProt%'");
> > > boolean first = true;
> > > for (int taxon: taxons) {
> > > baseQueryBuilder.append(first ? " and (" : " or ");
> > > baseQueryBuilder
> > > .append("taxon = 'taxon:")
> > > .append(taxon).append("'");
> > > first = false;
> > > }
> > > baseQueryBuilder.append(")");
> > >
> > > ...and so on. I just sort of rattled this off so there may be
> > little glitches, but anyway this is just to give you an overall idea.
> > >
> > > Put another way, no, you do not need to iterate this method for
> > each taxon ID. Instead, you can still call this method once, with
> > the multiplicity of taxon IDs emerging in terms of the actual
> > condition used for selecting the GO terms to be exported (based on
> > the available GOA records, which as you may recall are loaded from .goa
> files).
> > >
> > > As a side note, right here you have an opportunity for a little
> > sanity check regarding the content of the relational database: GO
> > terms will only be exported if GOA records for the desired taxon
> > IDs have been imported into the database. So, as a pre-flight
> > check, one can see if there are any GOA records at all for each
> > chosen taxon ID. If there are none, then the .goa file for that
> > species needs to be imported into the relational database.
> > >
> > > Hope this helps...
> > >
> > > John David N. Dionisio, PhD
> > > Associate Professor, Computer Science
> > > Loyola Marymount University
> > >
> > >
> > > On Aug 4, 2011, at 1:00 PM, Kam Dahlquist wrote:
> > >
> > > > Hi,
> > > >
> > > > Dondi will have to chime in on this, but I think this is where
> > things are going to get tricky.
> > > >
> > > > The final gdb does not actually contain the entire GO, it gets
> > trimmed somehow based on the GO associations for a particular
> > species. This is because MAPPFinder cannot handle loading the
> > entire GO. Since there is some type of species-specific trimming
> > going on, it's quite possible that this will need to iterate.
> > > >
> > > > However, I don't have the foggiest idea of how this works, so
> > Dondi will have to chime in.
> > > >
> > > > Best,
> > > > Kam
> > > >
> > > > At 12:09 AM 8/4/2011, you wrote:
> > > >> Wednesday 8/3/11 progress:
> > > >>
> > > >> 1. After following the ExportPanel1.java ground zero code of:
> > databaseProfile.setSelectedSpeciesProfile( selectedProfile );
> > > >>
> > > >> I found the method in DatabaseProfile.java plus a getter method;
> > > >> SpeciesProfile setSelectedSpeciesProfile( speciesProfile ) and
> > SpeciesProfile getSelectedSpeciesProfile( speciesProfile )
> > > >>
> > > >> I created two new methods that each handle List<Object> of
> > SpeciesProfiles argument instead of a single SpeciesProfile;
> > setSelectedSpeciesProfiles and getSelectedSpeciesProfiles.
> > > >>
> > > >> This enabled the ExportPanel1 ground zero code to become:
> > databaseProfile.setSelectedSpeciesProfiles(selectedSpecies);
> > > >>
> > > >> 2. public static void export() on line 104 in ExportToGenMAPP.java
> > > >>
> > > >> On line 107 ExportGoData is instantiated which I found in
> > ExportGoData.java and calls a method: public void export(char
> > chosenAspect, int taxon).
> > > >>
> > > >> Within export, taxon id is required for another method:
> > private void populateGoTables(char chosenAspect, int taxon).
> > > >>
> > > >> Within populateGoTables, taxon id is required for another
> > method: private void populateUniprotGoTableFromSQL( char
> > chosenAspect, int taxon).
> > > >>
> > > >> But, if the export to GDB process starts off with exporting GO
> > data, doesn't it only need to do that once no matter how many
> > species are selected? As you probably realize, I'm leading towards
> > not having to iterate through this for each taxon id if possible.
> > > >>
> > > >> Also, how does the export actually work? How are GO ids and
> > UniProt ids related within the table?
> > > >>
> > > >> Thanks!
> > > >>
> > > >> Richard
> > > >>
> > > >>
> > > > <ATT00001..txt><ATT00002..txt>
> > >
> > >
> > >
> >
> ------------------------------------------------------------------------------
> > > BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
> > > The must-attend event for mobile developers. Connect with experts.
> > > Get tools for creating Super Apps. See the latest technologies.
> > > Sessions, hands-on labs, demos & much more. Register early & save!
> > > http://p.sf.net/sfu/rim-blackberry-1
> > > _______________________________________________
> > > xmlpipedb-developer mailing list
> > > xml...@li...
> > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> > >
> > >
> > >
> > > <ATT00001..txt><ATT00002..txt>
> >
> >
>
> >------------------------------------------------------------------------------
> >Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
> >user administration capabilities and model configuration. Take
> >the hassle out of deploying and managing Subversion and the
> >tools developers use with it.
> >http://p.sf.net/sfu/wandisco-dev2dev
> >_______________________________________________
> >xmlpipedb-developer mailing list
> >xml...@li...
> >https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>
>
>
> ------------------------------------------------------------------------------
> FREE DOWNLOAD - uberSVN with Social Coding for Subversion.
> Subversion made easy with a complete admin console. Easy
> to use, easy to manage, easy to install, easy to extend.
> Get a Free download of the new open ALM Subversion platform now.
> http://p.sf.net/sfu/wandisco-dev2dev
> _______________________________________________
> xmlpipedb-developer mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>
|
|
From: Richard B. <rbr...@gm...> - 2011-08-12 21:47:54
|
Yeah that makes sense, I am unable to open the gdb on my laptop for some reason as it just freezes MSAccess. But I can right click on each entry and export which is what I did for original row count.
Will try exporting a different species and see if it happens again. I had expected the species table to populate so that is a nice clue.
Leaving for Nantucket tonight but will do my best to keep moving this forward.
Richard
Sent from my iPhone
On Aug 12, 2011, at 1:41 PM, Kam Dahlquist <kda...@lm...> wrote:
> Hi,
>
> This is where ye olde visual inspection comes in handy. If you open up the tables in the database, you will see that the species field is blank for all of the systems tables (UniProt, RefSeq, etc.), the Info table, and the Systems table itself (for OrderedLocusNames).
>
> Because data is missing for thousands of records, that results in the file size discrepancy.
>
> Best,
> Kam
>
>
> At 09:25 AM 8/12/2011, you wrote:
>> Saw your comments, some silly mistakes you caught.
>>
>> I re-ran the export using your changes, checked original row counts and they were identical to the previous two exports. The file size was also near identical to the multi-species aware export at 21.0 MB (22,036,480 bytes). I'm stumped on this but did confirm that GO data was processed.
>>
>> Richard
>>
>> On Thu, Aug 11, 2011 at 3:21 PM, John David N. Dionisio <do...@lm...> wrote:
>> Greetings,
>>
>> Sorry for the delay; stuff kept interceding through last night and today.
>>
>> I did finally manage to look at your latest commits then committed back some tweaks and comments. Look in particular at the stuff I mentioned regarding taxonIDs. If things aren't self-explanatory, just holler.
>>
>> I also scanned your changes to see if anything might have affected the file size. Nothing jumped at me, unfortunately. Did you do the "Process GO Data" step prior to exporting? That's the only [highly slim] lead that I could think of.
>>
>> John David N. Dionisio, PhD
>> Associate Professor, Computer Science
>> Loyola Marymount University
>>
>>
>> On Aug 10, 2011, at 10:07 PM, Richard Brous wrote:
>>
>> > OK, will hold up until we discuss further...
>> >
>> >
>> >
>> > **On a side note I exported two Msmegmatis gdb's to see how they compared and found the following:
>> >
>> > pre multispecies aware build b64: gdb filesize = 24.5 MB (25,792,512 bytes)
>> >
>> > multispecies current working copy: gdb filesize = 20.8 MB (21,905,408 bytes)
>> >
>> > BUT when I checked orginal row counts for each they were identical... i even checked a second time to be sure...
>> >
>> > See attached spreadsheet and I'll post the files on the biodb wiki as well in a few.
>> >
>> >
>> > I'm highly suspicious of the file sizes being that different with identical original row counts.
>> >
>> > Dondi would you take a look at my committed code to see if there are glaring issues I am not aware of?
>> >
>> > Thanks.
>> >
>> > Richard
>> >
>> > On Wed, Aug 10, 2011 at 9:45 PM, John David N. Dionisio <do...@lm...> wrote:
>> > Greetings,
>> >
>> > I think we have to turn to Dr. Dahlquist's GenMAPP knowledge here to get the definitive answer. I see two choices:
>> >
>> > - The Info table should have one record for each species that the .gdb holds, in which case the change you need is to wrap that single submit call inside a loop, so that submit is called once for each chosen species.
>> >
>> > - The Info table should always have one record, and if the .gdb holds multiple species, the "Species" column should be some concatenation of multiple species names. In this case, you would still call submit only once, but the value you send into the "Species" column is some accumulation of all chosen species names.
>> >
>> > Admittedly I don't know which way is right (I assumed the former as of our Tuesday meeting, but on further examination I'm no longer quite so sure).
>> >
>> > For Kam --- what does GenMAPP expect to see in the Info table if the opened .gdb contains multiple species?
>> >
>> > John David N. Dionisio, PhD
>> > Associate Professor, Computer Science
>> > Loyola Marymount University
>> >
>> >
>> > On Aug 10, 2011, at 9:36 PM, Richard Brous wrote:
>> >
>> > > OK, continued to review ExportToGenMAPP and dug into the creation of the first TableManager tmA on line 118.
>> > >
>> > > In reading through the method, my understanding is that it creates a new TableManager based on the selectedDatabaseProfile (which is UniProt).
>> > >
>> > > This is performed by the method getInfoTableManager() which then calls method submit(String tableName, QueryType queryType, String[][] columnNamesToValues);
>> > >
>> > > the code is as follows:
>> > >
>> > > tableManager.submit("Info", QueryType.insert, new String[][] { { "Owner", owner }, { "Version", new SimpleDateFormat("yyyyMMdd").format(version) }, { "MODSystem", modSystem }, { "Species", speciesProfile.getSpeciesName() }, { "Modify", new SimpleDateFormat("yyyyMMdd").format(modify) }, { "DisplayOrder", displayOrder }, { "Notes", notes } });
>> > >
>> > >
>> > > The modification of this line centers on { "Species", speciesProfile.getSpeciesName() }, since it originally processed a single species.
>> > >
>> > > So now I need to populate the arguments with the species contained within selectedDatabaseprofile.selectedSpeciesProfiles.
>> > >
>> > > I think I'll start with the baseArgument up to MODSystem, then append as many species as necessary, and then cap off the end with the rest starting at Modify. (similar to your approach in ExportGoData, populateUniprotGoTableFromSQL(char chosenAspect, List<Integer> taxonIds) line 513
>> > >
>> > > Please let me know if this approach or analysis is off track.
>> > >
>> > > Thanks!
>> > >
>> > > Richard
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > On Wed, Aug 10, 2011 at 5:50 PM, Richard Brous <rbr...@gm...> wrote:
>> > > Updated repository to include all Gene Ontology changes discussed during our meeting yesterday.
>> > >
>> > > Digging into TableManager next.
>> > >
>> > > Richard
>> > >
>> > > On Fri, Aug 5, 2011 at 10:06 AM, Richard Brous <rbr...@gm...> wrote:
>> > > whew... thanks for the detailed reply. I will digest this a bit and get back to you with further questions.
>> > >
>> > > rb
>> > >
>> > > On Thu, Aug 4, 2011 at 11:18 PM, John David N. Dionisio <do...@lm...> wrote:
>> > > Greetings,
>> > >
>> > > Sorry for the delay. I wasn't able to walk through the relevant code until this evening.
>> > >
>> > > As Kam said, GOA serves as the link between the UniProt and GO IDs. It essentially determines which GO IDs get exported by using GOA to see which GO IDs are associated with an exported UniProt ID. The populateUniprotGoTableFromSQL, in its current form, extracts the GO association records that match the given taxon ID then exports, as UniProt-GO pairs, the GO and UniProt IDs referenced within that GO association record. Processing that follows this is then based on the GO IDs that got exported --- and that's how the current code avoids exporting the entire list of GO terms.
>> > >
>> > > The operative query is on the second line of populateUniprotGoTableFromSQL:
>> > >
>> > > String uniProtAndGOIDSQL = "select db_object_id, go_id, evidence_code, with_or_from from goa where db like '%UniProt%' and taxon = 'taxon:" + taxon + "'";
>> > >
>> > > In plain English, this selects the GOA records whose database is UniProt and whose taxon ID is the given taxon. An additional condition is added for the "aspect" (All, Component, Function, or Process) that is to be exported. This is another reduction filter, to further shrink the number of exported GO terms and thus avoid MAPPFinder issues later on.
>> > >
>> > > Given this, the proper expansion here is to change the taxon predicate to a multiple predicate. That is, this method can be changed to now accept a collection or array of taxon IDs, and the base query should then be changed so that it accepts any taxon from that collection. More or less, you want:
>> > >
>> > > private void populateUniprotGoTableFromSQL(char chosenAspect, int[] taxons) throws SQLException {
>> > >
>> > > ...then, instead of the single string, you want to iterate through the taxon IDs:
>> > >
>> > > StringBuilder baseQueryBuilder = new StringBuilder("select db_object_id, go_id, evidence_code, with_or_from from goa where db like '%UniProt%'");
>> > > boolean first = true;
>> > > for (int taxon: taxons) {
>> > > baseQueryBuilder.append(first ? " and (" : " or ");
>> > > baseQueryBuilder
>> > > .append("taxon = 'taxon:")
>> > > .append(taxon).append("'");
>> > > first = false;
>> > > }
>> > > baseQueryBuilder.append(")");
>> > >
>> > > ...and so on. I just sort of rattled this off so there may be little glitches, but anyway this is just to give you an overall idea.
>> > >
>> > > Put another way, no, you do not need to iterate this method for each taxon ID. Instead, you can still call this method once, with the multiplicity of taxon IDs emerging in terms of the actual condition used for selecting the GO terms to be exported (based on the available GOA records, which as you may recall are loaded from .goa files).
>> > >
>> > > As a side note, right here you have an opportunity for a little sanity check regarding the content of the relational database: GO terms will only be exported if GOA records for the desired taxon IDs have been imported into the database. So, as a pre-flight check, one can see if there are any GOA records at all for each chosen taxon ID. If there are none, then the .goa file for that species needs to be imported into the relational database.
>> > >
>> > > Hope this helps...
>> > >
>> > > John David N. Dionisio, PhD
>> > > Associate Professor, Computer Science
>> > > Loyola Marymount University
>> > >
>> > >
>> > > On Aug 4, 2011, at 1:00 PM, Kam Dahlquist wrote:
>> > >
>> > > > Hi,
>> > > >
>> > > > Dondi will have to chime in on this, but I think this is where things are going to get tricky.
>> > > >
>> > > > The final gdb does not actually contain the entire GO, it gets trimmed somehow based on the GO associations for a particular species. This is because MAPPFinder cannot handle loading the entire GO. Since there is some type of species-specific trimming going on, it's quite possible that this will need to iterate.
>> > > >
>> > > > However, I don't have the foggiest idea of how this works, so Dondi will have to chime in.
>> > > >
>> > > > Best,
>> > > > Kam
>> > > >
>> > > > At 12:09 AM 8/4/2011, you wrote:
>> > > >> Wednesday 8/3/11 progress:
>> > > >>
>> > > >> 1. After following the ExportPanel1.java ground zero code of: databaseProfile.setSelectedSpeciesProfile( selectedProfile );
>> > > >>
>> > > >> I found the method in DatabaseProfile.java plus a getter method;
>> > > >> SpeciesProfile setSelectedSpeciesProfile( speciesProfile ) and SpeciesProfile getSelectedSpeciesProfile( speciesProfile )
>> > > >>
>> > > >> I created two new methods that each handle List<Object> of SpeciesProfiles argument instead of a single SpeciesProfile; setSelectedSpeciesProfiles and getSelectedSpeciesProfiles.
>> > > >>
>> > > >> This enabled the ExportPanel1 ground zero code to become: databaseProfile.setSelectedSpeciesProfiles(selectedSpecies);
>> > > >>
>> > > >> 2. public static void export() on line 104 in ExportToGenMAPP.java
>> > > >>
>> > > >> On line 107 ExportGoData is instantiated which I found in ExportGoData.java and calls a method: public void export(char chosenAspect, int taxon).
>> > > >>
>> > > >> Within export, taxon id is required for another method: private void populateGoTables(char chosenAspect, int taxon).
>> > > >>
>> > > >> Within populateGoTables, taxon id is required for another method: private void populateUniprotGoTableFromSQL( char chosenAspect, int taxon).
>> > > >>
>> > > >> But, if the export to GDB process starts off with exporting GO data, doesn't it only need to do that once no matter how many species are selected? As you probably realize, I'm leading towards not having to iterate through this for each taxon id if possible.
>> > > >>
>> > > >> Also, how does the export actually work? How are GO ids and UniProt ids related within the table?
>> > > >>
>> > > >> Thanks!
>> > > >>
>> > > >> Richard
>> > > >>
>> > > >>
>> > > > <ATT00001..txt><ATT00002..txt>
>> > >
>> > >
>> > > ------------------------------------------------------------------------------
>> > > BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
>> > > The must-attend event for mobile developers. Connect with experts.
>> > > Get tools for creating Super Apps. See the latest technologies.
>> > > Sessions, hands-on labs, demos & much more. Register early & save!
>> > > http://p.sf.net/sfu/rim-blackberry-1
>> > > _______________________________________________
>> > > xmlpipedb-developer mailing list
>> > > xml...@li...
>> > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>> > >
>> > >
>> > >
>> > > <ATT00001..txt><ATT00002..txt>
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
>> > user administration capabilities and model configuration. Take
>> > the hassle out of deploying and managing Subversion and the
>> > tools developers use with it.
>> > http://p.sf.net/sfu/wandisco-dev2dev
>> > _______________________________________________
>> > xmlpipedb-developer mailing list
>> > xml...@li...
>> > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>> >
>> > <CompareOriginalRowCounts.xlsx><ATT00001..txt><ATT00002..txt>
>>
>>
>> ------------------------------------------------------------------------------
>> Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
>> user administration capabilities and model configuration. Take
>> the hassle out of deploying and managing Subversion and the
>> tools developers use with it.
>> http://p.sf.net/sfu/wandisco-dev2dev
>> _______________________________________________
>> xmlpipedb-developer mailing list
>> xml...@li...
>> https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>>
>>
>>
>
> ------------------------------------------------------------------------------
> FREE DOWNLOAD - uberSVN with Social Coding for Subversion.
> Subversion made easy with a complete admin console. Easy
> to use, easy to manage, easy to install, easy to extend.
> Get a Free download of the new open ALM Subversion platform now.
> http://p.sf.net/sfu/wandisco-dev2dev
> _______________________________________________
> xmlpipedb-developer mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
|
|
From: Kam D. <kda...@lm...> - 2011-08-12 20:41:41
|
Hi,
This is where ye olde visual inspection comes in handy. If you open
up the tables in the database, you will see that the species field is
blank for all of the systems tables (UniProt, RefSeq, etc.), the Info
table, and the Systems table itself (for OrderedLocusNames).
Because data is missing for thousands of records, that results in the
file size discrepancy.
Best,
Kam
At 09:25 AM 8/12/2011, you wrote:
>Saw your comments, some silly mistakes you caught.
>
>I re-ran the export using your changes, checked original row counts
>and they were identical to the previous two exports. The file size
>was also near identical to the multi-species aware export at 21.0 MB
>(22,036,480 bytes). I'm stumped on this but did confirm that GO data
>was processed.
>
>Richard
>
>On Thu, Aug 11, 2011 at 3:21 PM, John David N. Dionisio
><<mailto:do...@lm...>do...@lm...> wrote:
>Greetings,
>
>Sorry for the delay; stuff kept interceding through last night and today.
>
>I did finally manage to look at your latest commits then committed
>back some tweaks and comments. Look in particular at the stuff I
>mentioned regarding taxonIDs. If things aren't self-explanatory, just holler.
>
>I also scanned your changes to see if anything might have affected
>the file size. Nothing jumped at me, unfortunately. Did you do the
>"Process GO Data" step prior to exporting? That's the only [highly
>slim] lead that I could think of.
>
>John David N. Dionisio, PhD
>Associate Professor, Computer Science
>Loyola Marymount University
>
>
>On Aug 10, 2011, at 10:07 PM, Richard Brous wrote:
>
> > OK, will hold up until we discuss further...
> >
> >
> >
> > **On a side note I exported two Msmegmatis gdb's to see how they
> compared and found the following:
> >
> > pre multispecies aware build b64: gdb filesize = 24.5 MB (25,792,512 bytes)
> >
> > multispecies current working copy: gdb filesize = 20.8 MB
> (21,905,408 bytes)
> >
> > BUT when I checked orginal row counts for each they were
> identical... i even checked a second time to be sure...
> >
> > See attached spreadsheet and I'll post the files on the biodb
> wiki as well in a few.
> >
> >
> > I'm highly suspicious of the file sizes being that different with
> identical original row counts.
> >
> > Dondi would you take a look at my committed code to see if there
> are glaring issues I am not aware of?
> >
> > Thanks.
> >
> > Richard
> >
> > On Wed, Aug 10, 2011 at 9:45 PM, John David N. Dionisio
> <<mailto:do...@lm...>do...@lm...> wrote:
> > Greetings,
> >
> > I think we have to turn to Dr. Dahlquist's GenMAPP knowledge here
> to get the definitive answer. I see two choices:
> >
> > - The Info table should have one record for each species that the
> .gdb holds, in which case the change you need is to wrap that
> single submit call inside a loop, so that submit is called once for
> each chosen species.
> >
> > - The Info table should always have one record, and if the .gdb
> holds multiple species, the "Species" column should be some
> concatenation of multiple species names. In this case, you would
> still call submit only once, but the value you send into the
> "Species" column is some accumulation of all chosen species names.
> >
> > Admittedly I don't know which way is right (I assumed the former
> as of our Tuesday meeting, but on further examination I'm no longer
> quite so sure).
> >
> > For Kam --- what does GenMAPP expect to see in the Info table if
> the opened .gdb contains multiple species?
> >
> > John David N. Dionisio, PhD
> > Associate Professor, Computer Science
> > Loyola Marymount University
> >
> >
> > On Aug 10, 2011, at 9:36 PM, Richard Brous wrote:
> >
> > > OK, continued to review ExportToGenMAPP and dug into the
> creation of the first TableManager tmA on line 118.
> > >
> > > In reading through the method, my understanding is that it
> creates a new TableManager based on the selectedDatabaseProfile
> (which is UniProt).
> > >
> > > This is performed by the method getInfoTableManager() which
> then calls method submit(String tableName, QueryType queryType,
> String[][] columnNamesToValues);
> > >
> > > the code is as follows:
> > >
> > > tableManager.submit("Info", QueryType.insert, new String[][] {
> { "Owner", owner }, { "Version", new
> SimpleDateFormat("yyyyMMdd").format(version) }, { "MODSystem",
> modSystem }, { "Species", speciesProfile.getSpeciesName() }, {
> "Modify", new SimpleDateFormat("yyyyMMdd").format(modify) }, {
> "DisplayOrder", displayOrder }, { "Notes", notes } });
> > >
> > >
> > > The modification of this line centers on { "Species",
> speciesProfile.getSpeciesName() }, since it originally processed a
> single species.
> > >
> > > So now I need to populate the arguments with the species
> contained within selectedDatabaseprofile.selectedSpeciesProfiles.
> > >
> > > I think I'll start with the baseArgument up to MODSystem, then
> append as many species as necessary, and then cap off the end with
> the rest starting at Modify. (similar to your approach in
> ExportGoData, populateUniprotGoTableFromSQL(char chosenAspect,
> List<Integer> taxonIds) line 513
> > >
> > > Please let me know if this approach or analysis is off track.
> > >
> > > Thanks!
> > >
> > > Richard
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Wed, Aug 10, 2011 at 5:50 PM, Richard Brous
> <<mailto:rbr...@gm...>rbr...@gm...> wrote:
> > > Updated repository to include all Gene Ontology changes
> discussed during our meeting yesterday.
> > >
> > > Digging into TableManager next.
> > >
> > > Richard
> > >
> > > On Fri, Aug 5, 2011 at 10:06 AM, Richard Brous
> <<mailto:rbr...@gm...>rbr...@gm...> wrote:
> > > whew... thanks for the detailed reply. I will digest this a bit
> and get back to you with further questions.
> > >
> > > rb
> > >
> > > On Thu, Aug 4, 2011 at 11:18 PM, John David N. Dionisio
> <<mailto:do...@lm...>do...@lm...> wrote:
> > > Greetings,
> > >
> > > Sorry for the delay. I wasn't able to walk through the
> relevant code until this evening.
> > >
> > > As Kam said, GOA serves as the link between the UniProt and GO
> IDs. It essentially determines which GO IDs get exported by using
> GOA to see which GO IDs are associated with an exported UniProt
> ID. The populateUniprotGoTableFromSQL, in its current form,
> extracts the GO association records that match the given taxon ID
> then exports, as UniProt-GO pairs, the GO and UniProt IDs
> referenced within that GO association record. Processing that
> follows this is then based on the GO IDs that got exported --- and
> that's how the current code avoids exporting the entire list of GO terms.
> > >
> > > The operative query is on the second line of
> populateUniprotGoTableFromSQL:
> > >
> > > String uniProtAndGOIDSQL = "select db_object_id, go_id,
> evidence_code, with_or_from from goa where db like '%UniProt%' and
> taxon = 'taxon:" + taxon + "'";
> > >
> > > In plain English, this selects the GOA records whose database
> is UniProt and whose taxon ID is the given taxon. An additional
> condition is added for the "aspect" (All, Component, Function, or
> Process) that is to be exported. This is another reduction filter,
> to further shrink the number of exported GO terms and thus avoid
> MAPPFinder issues later on.
> > >
> > > Given this, the proper expansion here is to change the taxon
> predicate to a multiple predicate. That is, this method can be
> changed to now accept a collection or array of taxon IDs, and the
> base query should then be changed so that it accepts any taxon from
> that collection. More or less, you want:
> > >
> > > private void populateUniprotGoTableFromSQL(char
> chosenAspect, int[] taxons) throws SQLException {
> > >
> > > ...then, instead of the single string, you want to iterate
> through the taxon IDs:
> > >
> > > StringBuilder baseQueryBuilder = new StringBuilder("select
> db_object_id, go_id, evidence_code, with_or_from from goa where db
> like '%UniProt%'");
> > > boolean first = true;
> > > for (int taxon: taxons) {
> > > baseQueryBuilder.append(first ? " and (" : " or ");
> > > baseQueryBuilder
> > > .append("taxon = 'taxon:")
> > > .append(taxon).append("'");
> > > first = false;
> > > }
> > > baseQueryBuilder.append(")");
> > >
> > > ...and so on. I just sort of rattled this off so there may be
> little glitches, but anyway this is just to give you an overall idea.
> > >
> > > Put another way, no, you do not need to iterate this method for
> each taxon ID. Instead, you can still call this method once, with
> the multiplicity of taxon IDs emerging in terms of the actual
> condition used for selecting the GO terms to be exported (based on
> the available GOA records, which as you may recall are loaded from .goa files).
> > >
> > > As a side note, right here you have an opportunity for a little
> sanity check regarding the content of the relational database: GO
> terms will only be exported if GOA records for the desired taxon
> IDs have been imported into the database. So, as a pre-flight
> check, one can see if there are any GOA records at all for each
> chosen taxon ID. If there are none, then the .goa file for that
> species needs to be imported into the relational database.
> > >
> > > Hope this helps...
> > >
> > > John David N. Dionisio, PhD
> > > Associate Professor, Computer Science
> > > Loyola Marymount University
> > >
> > >
> > > On Aug 4, 2011, at 1:00 PM, Kam Dahlquist wrote:
> > >
> > > > Hi,
> > > >
> > > > Dondi will have to chime in on this, but I think this is
> where things are going to get tricky.
> > > >
> > > > The final gdb does not actually contain the entire GO, it
> gets trimmed somehow based on the GO associations for a particular
> species. This is because MAPPFinder cannot handle loading the
> entire GO. Since there is some type of species-specific trimming
> going on, it's quite possible that this will need to iterate.
> > > >
> > > > However, I don't have the foggiest idea of how this works, so
> Dondi will have to chime in.
> > > >
> > > > Best,
> > > > Kam
> > > >
> > > > At 12:09 AM 8/4/2011, you wrote:
> > > >> Wednesday 8/3/11 progress:
> > > >>
> > > >> 1. After following the ExportPanel1.java ground zero code
> of: databaseProfile.setSelectedSpeciesProfile( selectedProfile );
> > > >>
> > > >> I found the method in DatabaseProfile.java plus a getter method;
> > > >> SpeciesProfile setSelectedSpeciesProfile( speciesProfile )
> and SpeciesProfile getSelectedSpeciesProfile( speciesProfile )
> > > >>
> > > >> I created two new methods that each handle List<Object> of
> SpeciesProfiles argument instead of a single SpeciesProfile;
> setSelectedSpeciesProfiles and getSelectedSpeciesProfiles.
> > > >>
> > > >> This enabled the ExportPanel1 ground zero code to become:
> databaseProfile.setSelectedSpeciesProfiles(selectedSpecies);
> > > >>
> > > >> 2. public static void export() on line 104 in ExportToGenMAPP.java
> > > >>
> > > >> On line 107 ExportGoData is instantiated which I found in
> ExportGoData.java and calls a method: public void export(char
> chosenAspect, int taxon).
> > > >>
> > > >> Within export, taxon id is required for another method:
> private void populateGoTables(char chosenAspect, int taxon).
> > > >>
> > > >> Within populateGoTables, taxon id is required for another
> method: private void populateUniprotGoTableFromSQL( char
> chosenAspect, int taxon).
> > > >>
> > > >> But, if the export to GDB process starts off with exporting
> GO data, doesn't it only need to do that once no matter how many
> species are selected? As you probably realize, I'm leading towards
> not having to iterate through this for each taxon id if possible.
> > > >>
> > > >> Also, how does the export actually work? How are GO ids and
> UniProt ids related within the table?
> > > >>
> > > >> Thanks!
> > > >>
> > > >> Richard
> > > >>
> > > >>
> > > > <ATT00001..txt><ATT00002..txt>
> > >
> > >
> > >
> ------------------------------------------------------------------------------
> > > BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
> > > The must-attend event for mobile developers. Connect with experts.
> > > Get tools for creating Super Apps. See the latest technologies.
> > > Sessions, hands-on labs, demos & much more. Register early & save!
> > >
> <http://p.sf.net/sfu/rim-blackberry-1>http://p.sf.net/sfu/rim-blackberry-1
> > > _______________________________________________
> > > xmlpipedb-developer mailing list
> > >
> <mailto:xml...@li...>xml...@li...
> > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> > >
> > >
> > >
> > > <ATT00001..txt><ATT00002..txt>
> >
> >
> >
> ------------------------------------------------------------------------------
> > Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
> > user administration capabilities and model configuration. Take
> > the hassle out of deploying and managing Subversion and the
> > tools developers use with it.
> > <http://p.sf.net/sfu/wandisco-dev2dev>http://p.sf.net/sfu/wandisco-dev2dev
> > _______________________________________________
> > xmlpipedb-developer mailing list
> >
> <mailto:xml...@li...>xml...@li...
> > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> >
> > <CompareOriginalRowCounts.xlsx><ATT00001..txt><ATT00002..txt>
>
>
>------------------------------------------------------------------------------
>Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
>user administration capabilities and model configuration. Take
>the hassle out of deploying and managing Subversion and the
>tools developers use with it.
><http://p.sf.net/sfu/wandisco-dev2dev>http://p.sf.net/sfu/wandisco-dev2dev
>_______________________________________________
>xmlpipedb-developer mailing list
><mailto:xml...@li...>xml...@li...
>https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>
>
>
|
|
From: Kam D. <kda...@lm...> - 2011-08-12 19:08:43
|
Hi,
I think we had better leave the info table with only one record
(option 2). The species names can be separated by pipes " | " as
they are in the Systems table where there are multiple species. To
my knowledge, the only time GenMAPP needs to access the info table is
for the "DisplayOrder" field, I don't know what would happen if there
were multiple records there. I know that the spec for the table says
that it should only be one record, but I don't know if it would crash
if there were multiple records. To be on the safe side, I think we
should just keep it to the one record.
Best,
Kam
At 09:45 PM 8/10/2011, John David N. Dionisio wrote:
>Greetings,
>
>I think we have to turn to Dr. Dahlquist's GenMAPP knowledge here to
>get the definitive answer. I see two choices:
>
>- The Info table should have one record for each species that the
>.gdb holds, in which case the change you need is to wrap that single
>submit call inside a loop, so that submit is called once for each
>chosen species.
>
>- The Info table should always have one record, and if the .gdb
>holds multiple species, the "Species" column should be some
>concatenation of multiple species names. In this case, you would
>still call submit only once, but the value you send into the
>"Species" column is some accumulation of all chosen species names.
>
>Admittedly I don't know which way is right (I assumed the former as
>of our Tuesday meeting, but on further examination I'm no longer
>quite so sure).
>
>For Kam --- what does GenMAPP expect to see in the Info table if the
>opened .gdb contains multiple species?
>
>John David N. Dionisio, PhD
>Associate Professor, Computer Science
>Loyola Marymount University
>
>
>On Aug 10, 2011, at 9:36 PM, Richard Brous wrote:
>
> > OK, continued to review ExportToGenMAPP and dug into the creation
> of the first TableManager tmA on line 118.
> >
> > In reading through the method, my understanding is that it
> creates a new TableManager based on the selectedDatabaseProfile
> (which is UniProt).
> >
> > This is performed by the method getInfoTableManager() which then
> calls method submit(String tableName, QueryType queryType,
> String[][] columnNamesToValues);
> >
> > the code is as follows:
> >
> > tableManager.submit("Info", QueryType.insert, new String[][] { {
> "Owner", owner }, { "Version", new
> SimpleDateFormat("yyyyMMdd").format(version) }, { "MODSystem",
> modSystem }, { "Species", speciesProfile.getSpeciesName() }, {
> "Modify", new SimpleDateFormat("yyyyMMdd").format(modify) }, {
> "DisplayOrder", displayOrder }, { "Notes", notes } });
> >
> >
> > The modification of this line centers on { "Species",
> speciesProfile.getSpeciesName() }, since it originally processed a
> single species.
> >
> > So now I need to populate the arguments with the species
> contained within selectedDatabaseprofile.selectedSpeciesProfiles.
> >
> > I think I'll start with the baseArgument up to MODSystem, then
> append as many species as necessary, and then cap off the end with
> the rest starting at Modify. (similar to your approach in
> ExportGoData, populateUniprotGoTableFromSQL(char chosenAspect,
> List<Integer> taxonIds) line 513
> >
> > Please let me know if this approach or analysis is off track.
> >
> > Thanks!
> >
> > Richard
> >
> >
> >
> >
> >
> >
> > On Wed, Aug 10, 2011 at 5:50 PM, Richard Brous <rbr...@gm...> wrote:
> > Updated repository to include all Gene Ontology changes discussed
> during our meeting yesterday.
> >
> > Digging into TableManager next.
> >
> > Richard
> >
> > On Fri, Aug 5, 2011 at 10:06 AM, Richard Brous <rbr...@gm...> wrote:
> > whew... thanks for the detailed reply. I will digest this a bit
> and get back to you with further questions.
> >
> > rb
> >
> > On Thu, Aug 4, 2011 at 11:18 PM, John David N. Dionisio
> <do...@lm...> wrote:
> > Greetings,
> >
> > Sorry for the delay. I wasn't able to walk through the relevant
> code until this evening.
> >
> > As Kam said, GOA serves as the link between the UniProt and GO
> IDs. It essentially determines which GO IDs get exported by using
> GOA to see which GO IDs are associated with an exported UniProt
> ID. The populateUniprotGoTableFromSQL, in its current form,
> extracts the GO association records that match the given taxon ID
> then exports, as UniProt-GO pairs, the GO and UniProt IDs
> referenced within that GO association record. Processing that
> follows this is then based on the GO IDs that got exported --- and
> that's how the current code avoids exporting the entire list of GO terms.
> >
> > The operative query is on the second line of populateUniprotGoTableFromSQL:
> >
> > String uniProtAndGOIDSQL = "select db_object_id, go_id,
> evidence_code, with_or_from from goa where db like '%UniProt%' and
> taxon = 'taxon:" + taxon + "'";
> >
> > In plain English, this selects the GOA records whose database is
> UniProt and whose taxon ID is the given taxon. An additional
> condition is added for the "aspect" (All, Component, Function, or
> Process) that is to be exported. This is another reduction filter,
> to further shrink the number of exported GO terms and thus avoid
> MAPPFinder issues later on.
> >
> > Given this, the proper expansion here is to change the taxon
> predicate to a multiple predicate. That is, this method can be
> changed to now accept a collection or array of taxon IDs, and the
> base query should then be changed so that it accepts any taxon from
> that collection. More or less, you want:
> >
> > private void populateUniprotGoTableFromSQL(char chosenAspect,
> int[] taxons) throws SQLException {
> >
> > ...then, instead of the single string, you want to iterate
> through the taxon IDs:
> >
> > StringBuilder baseQueryBuilder = new StringBuilder("select
> db_object_id, go_id, evidence_code, with_or_from from goa where db
> like '%UniProt%'");
> > boolean first = true;
> > for (int taxon: taxons) {
> > baseQueryBuilder.append(first ? " and (" : " or ");
> > baseQueryBuilder
> > .append("taxon = 'taxon:")
> > .append(taxon).append("'");
> > first = false;
> > }
> > baseQueryBuilder.append(")");
> >
> > ...and so on. I just sort of rattled this off so there may be
> little glitches, but anyway this is just to give you an overall idea.
> >
> > Put another way, no, you do not need to iterate this method for
> each taxon ID. Instead, you can still call this method once, with
> the multiplicity of taxon IDs emerging in terms of the actual
> condition used for selecting the GO terms to be exported (based on
> the available GOA records, which as you may recall are loaded from .goa files).
> >
> > As a side note, right here you have an opportunity for a little
> sanity check regarding the content of the relational database: GO
> terms will only be exported if GOA records for the desired taxon
> IDs have been imported into the database. So, as a pre-flight
> check, one can see if there are any GOA records at all for each
> chosen taxon ID. If there are none, then the .goa file for that
> species needs to be imported into the relational database.
> >
> > Hope this helps...
> >
> > John David N. Dionisio, PhD
> > Associate Professor, Computer Science
> > Loyola Marymount University
> >
> >
> > On Aug 4, 2011, at 1:00 PM, Kam Dahlquist wrote:
> >
> > > Hi,
> > >
> > > Dondi will have to chime in on this, but I think this is where
> things are going to get tricky.
> > >
> > > The final gdb does not actually contain the entire GO, it gets
> trimmed somehow based on the GO associations for a particular
> species. This is because MAPPFinder cannot handle loading the
> entire GO. Since there is some type of species-specific trimming
> going on, it's quite possible that this will need to iterate.
> > >
> > > However, I don't have the foggiest idea of how this works, so
> Dondi will have to chime in.
> > >
> > > Best,
> > > Kam
> > >
> > > At 12:09 AM 8/4/2011, you wrote:
> > >> Wednesday 8/3/11 progress:
> > >>
> > >> 1. After following the ExportPanel1.java ground zero code of:
> databaseProfile.setSelectedSpeciesProfile( selectedProfile );
> > >>
> > >> I found the method in DatabaseProfile.java plus a getter method;
> > >> SpeciesProfile setSelectedSpeciesProfile( speciesProfile ) and
> SpeciesProfile getSelectedSpeciesProfile( speciesProfile )
> > >>
> > >> I created two new methods that each handle List<Object> of
> SpeciesProfiles argument instead of a single SpeciesProfile;
> setSelectedSpeciesProfiles and getSelectedSpeciesProfiles.
> > >>
> > >> This enabled the ExportPanel1 ground zero code to become:
> databaseProfile.setSelectedSpeciesProfiles(selectedSpecies);
> > >>
> > >> 2. public static void export() on line 104 in ExportToGenMAPP.java
> > >>
> > >> On line 107 ExportGoData is instantiated which I found in
> ExportGoData.java and calls a method: public void export(char
> chosenAspect, int taxon).
> > >>
> > >> Within export, taxon id is required for another method:
> private void populateGoTables(char chosenAspect, int taxon).
> > >>
> > >> Within populateGoTables, taxon id is required for another
> method: private void populateUniprotGoTableFromSQL( char
> chosenAspect, int taxon).
> > >>
> > >> But, if the export to GDB process starts off with exporting GO
> data, doesn't it only need to do that once no matter how many
> species are selected? As you probably realize, I'm leading towards
> not having to iterate through this for each taxon id if possible.
> > >>
> > >> Also, how does the export actually work? How are GO ids and
> UniProt ids related within the table?
> > >>
> > >> Thanks!
> > >>
> > >> Richard
> > >>
> > >>
> > > <ATT00001..txt><ATT00002..txt>
> >
> >
> >
> ------------------------------------------------------------------------------
> > BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
> > The must-attend event for mobile developers. Connect with experts.
> > Get tools for creating Super Apps. See the latest technologies.
> > Sessions, hands-on labs, demos & much more. Register early & save!
> > http://p.sf.net/sfu/rim-blackberry-1
> > _______________________________________________
> > xmlpipedb-developer mailing list
> > xml...@li...
> > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> >
> >
> >
> > <ATT00001..txt><ATT00002..txt>
>
>
>------------------------------------------------------------------------------
>Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
>user administration capabilities and model configuration. Take
>the hassle out of deploying and managing Subversion and the
>tools developers use with it.
>http://p.sf.net/sfu/wandisco-dev2dev
>_______________________________________________
>xmlpipedb-developer mailing list
>xml...@li...
>https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
|
16 messages has been excluded from this view by a project administrator.
