xmlpipedb-developer Mailing List for XMLPipeDB (Page 15)
Brought to you by:
kdahlquist,
zugzugglug
Menu
▾
▴
xmlpipedb-developer — Mailing list for XMLPipeDB software development questions and discussions.
You can subscribe to this list here.
| 2009 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
(16) |
Sep
|
Oct
(9) |
Nov
(3) |
Dec
(6) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2010 |
Jan
(2) |
Feb
(8) |
Mar
|
Apr
(22) |
May
(1) |
Jun
|
Jul
|
Aug
(3) |
Sep
(32) |
Oct
(2) |
Nov
|
Dec
|
| 2011 |
Jan
|
Feb
(60) |
Mar
(42) |
Apr
(35) |
May
(17) |
Jun
(2) |
Jul
(23) |
Aug
(72) |
Sep
(15) |
Oct
(10) |
Nov
(14) |
Dec
(4) |
| 2012 |
Jan
(6) |
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2013 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
(11) |
Dec
|
| 2014 |
Jan
(1) |
Feb
(12) |
Mar
(14) |
Apr
(8) |
May
|
Jun
(14) |
Jul
(2) |
Aug
|
Sep
(5) |
Oct
(6) |
Nov
|
Dec
|
| 2015 |
Jan
|
Feb
(5) |
Mar
(2) |
Apr
|
May
|
Jun
(2) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: John D. N. D. <do...@lm...> - 2011-02-18 00:34:26
|
Hi Rich,
No problem. The pertinent line you're referring to, for XML, is this, right above the line you copied:
mycobacteriumtuberculosis_element_level0=uniprot/entry/gene/name&type&ordered locus
The slash-separated section is the "path" of XML tags leading to the element of interest; then, after the ampersand, is a name/value pair for the desired attribute to count. Note that there is no hint of a *content*-based filter (nor is there the capability for one, as far as I can tell in the code). By "content," I mean that we can't specify filters based on what's *between* the tags. We can only go as far as filter by attribute value, e.g., type="ordered locus".
But anyway, as mentioned in the earlier e-mail, let's have the missing IDs in the .gdb take precedence for now. Please take a look at the tuberculosis, A. thaliana, and P. falciparum profiles to get an idea for how the ID output can be customized, then let me know if you have any questions or need to confirm anything.
John David N. Dionisio, PhD
Associate Professor, Computer Science
Loyola Marymount University
On Feb 17, 2011, at 3:04 PM, Richard Brous wrote:
> Sorry been slammed with a programming assignment that kept needing continued iteration and it has been all consuming until last night. But I did get a chance to work with your comments and review the code again with a different mind set.
>
> Yes, I examined the gmbuilder.properties file ( the query is also in the MycobacteriumTuberculosisUniProtSpeciesProfile which I mentioned in a previous email ) but I don't think I see what you mean regarding the XML count.
>
> I understood that: mycobacteriumtuberculosis_query_level0=select count(*) from genenametype where type = 'ordered locus' and value like 'Rv%'; was the db query but don't see which is the XML count... or do they share the same query and you are saying that XML count doesn't recognize and therefore cannot use the 'Rv%' parameter?
>
> Richard
>
>
>
> On Sat, Feb 12, 2011 at 11:46 PM, John David N. Dionisio <do...@lm...> wrote:
> Hi Rich,
>
> Sorry for the delay. Had some distractions coming into the weekend.
>
> You've looked at the code; have you looked at gmbuilder.properties? (I may have mentioned it a few e-mails ago, just as you were starting to dig into this)
>
> On the copy I have, the M. tuberculosis block looks like this (indentation is mine to set it apart):
>
> # Mycobacterium tuberculosis
> mycobacteriumtuberculosis_level_amount=1
>
> mycobacteriumtuberculosis_element_level0=uniprot/entry/gene/name&type&ordered locus
>
> mycobacteriumtuberculosis_query_level0=select count(*) from genenametype where type = 'ordered locus' and value like 'Rv%';
>
> mycobacteriumtuberculosis_table_name_level0=Ordered Locus
>
> There, I think, is the rub. Notice that the XML count does not filter on RV%. The SQL query does.
>
> Unfortunately, I don't think the TallyEngine can include selective filtering in the XML counts. If the need to do selective filtering on XML is necessary, then I think we're looking at a new functionality for you to implement (or, if this throws things off too much, this may have to be noted somewhere, that the XML vs. database counts may be off because the database count is doing some text-based filtering but the XML count does not).
>
> What does xmlpipedb-match say? That will at least tell you whether the 'RV%' count is indeed correct.
>
> John David N. Dionisio, PhD
> Associate Professor, Computer Science
> Loyola Marymount University
>
>
>
> On Feb 11, 2011, at 4:52 PM, Richard Brous wrote:
>
> > OK here is what I was able to put together from the past few hours of code review:
> >
> > MycobacteriumTuberculosisUniProtSpeciesProfile.java:
> > -reveals that after the 2 System table modifications are made adding species name and link, a PreparedStatement is instantiated which builds and calls the base query.
> >
> > -The base query called is: ("SELECT value, type " + "FROM genenametype INNER JOIN entrytype_genetype " + "ON(entrytype_genetype_name_hjid = entrytype_genetype.hjid) " + "WHERE type = 'ordered locus' and value like 'Rv%' and entrytype_gene_hjid = ?")
> >
> > -So its looking in 'ordered locus' table/column for any tuple that starts with Rv (followed by any substring) and entrytype_gene_hjid = ? .
> > The 'like' comparator and % usage are clear with the 'type' entrytype_gene_hjid = ?
> >
> > -To me it seems the query makes sense so the problem is likely elsewhere.
> >
> > GenMappBuilder.java:
> > -I found method doTallies() at code line 895 which:
> > Instantiates a Configuration called hibernateConfiguration and assigns to it the current hibernate configuration
> > Validates database settings by analyzing hibernateConfiguration
> > Instantiates a CriterionList for uniprot and assigns to it TallyType.UNIPROT
> > Instantiates a CriterionList for go and assigns to it TallyType.GO
> > Determines if both xml files exist
> > Then getTallyResultsXML and getTallyResultsDatabase are run on both xml files and their respective CriterionList
> > Results are then formatted for display in a table.
> >
> > -So enum TallyType which means that they are the only valid datatypes which TallyEngine accepts... go to know ...
> >
> > -Based on the screen shot of Tally Engine it would seem that both getTallyResultsXML() and getTallyResultsDatabase() are incorrectly returning. Likely due to both using an incorrect query (as we previously supposed). But where are the queries?... the more I dig the more I think they are in the criterial all the work is done against.
> >
> > continuing the review:
> > getTallyResultsXML() calls Tally Engine instance method getXmlFileCounts(xmlFile)
> > getTallyResultsDatabase() calls Tally Engine instance method getDbcounts(new QueryEngine(hibernateConfiguration)
> > Both of these instanced methods originate from TallyEngine.java...
> >
> > TallyEngine.java:
> >
> > getXmlFileCounts() calls digestXmlFile() which instantiates a digester then processes against criteria... but this quickly becomes confusing and is hard to follow
> >
> > getDbcounts() then starts a db session and executes a query but then I also get a bit lost with my limited db knowledge.
> >
> > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >
> > OVERALL I think I'm getting closer to the issues but I still feel as if I'm missing some understanding to proceed further. Can you pass along some of that Dondi insight and steer me in the right direction? =D
> >
> > -DB Tally - Not having taken databases yet certainly is limiting my ability determine where the "criteria" are being set and how they are followed during session activities. Also is the query we have been looking for this whole time in the criteria or someplace else?
> >
> > -XML Tally - again is the query contained within the criteria that digestXmlFile() uses to parse?
> >
> > Richard
> >
> >
> > On Mon, Feb 7, 2011 at 5:50 PM, John David N. Dionisio <do...@lm...> wrote:
> > Right, schema issues are unlikely. Most count discrepancies like this that I've seen have boiled down to forming the right query. Then, knowing the right query (in both XML and SQL), it's a matter of making sure that TallyEngine asks that same query.
> >
> > John David N. Dionisio, PhD
> > Associate Professor, Computer Science
> > Loyola Marymount University
> >
> >
> > On Feb 7, 2011, at 5:48 PM, Richard Brous wrote:
> >
> > > OK, so based on your approach:
> > >
> > > 1. I'll start with reviewing the queries for xmlpipedb-match and sql queries needed for the respective results as you requested.
> > >
> > > I was also thinking I may need to review the schema from xml into postgres but the issue isn't likely a schema error. The error most likely lies in how xmlpipedbutils queries the data from xml source and writes to the tables what it returns?
> > >
> > > 2. I'll review the code: trace the entrance of tally engine in the gmbuilder code then follow it through the xmlpipedbutils.
> > >
> > > Richard
> > >
> > > On Sat, Feb 5, 2011 at 10:28 AM, John David N. Dionisio <do...@lm...> wrote:
> > > Just wanted to confirm (since I wasn't sure in the first e-mail) --- the XMLPipeDB Utilities source code is in trunk/xmlpipedbutils in SourceForge's Subversion repo.
> > >
> > > John David N. Dionisio, PhD
> > > Associate Professor, Computer Science
> > > Loyola Marymount University
> > >
> > >
> > >
> > > On Feb 5, 2011, at 10:02 AM, Richard Brous wrote:
> > >
> > > > Hi Dondi,
> > > >
> > > > So I'm at the point in working with M tuberculosis that I was able to exactly reproduce Dr. Dahlquist's problematic TallyEngine results.
> > > >
> > > > gmb2b60 Results
> > > >
> > > >
> > > >
> > > > Now the proverbial question - What next to solve the Ordered Locus import/count issue?
> > > >
> > > > **********************************************
> > > > Here is my thought process:
> > > >
> > > > Step 1: How does the import process work at the high level? (obviously correct me if I'm wrong)
> > > >
> > > > I believe that basically as each XML tag is read, it is placed in the proper Postgres table(s) based on some criteria. There is also likely some sort of check that each individual tag is in valid XML format unless we don't care at this stage (care at export) or maybe the parser just skips over and goes on to the next .
> > > >
> > > > Step 2: What could be the problem?
> > > >
> > > > Either -
> > > > a. XML tags are being parsed incorrectly (ignored/skipped)?
> > > > b. Decision criteria of which table they should be added to?
> > > >
> > > > **********************************************
> > > >
> > > > I read on the sourceforge wiki:
> > > >
> > > > XMLPipeDB has a modular architecture with three components that may be used separately or together. XSD-to-DB reads an XSD (XML Schema Definition) and automatically generates an SQL schema, Java classes, and Hibernate mappings. XMLPipeDB Utilities provides functionality for configuring the database, importing data, and performing queries. GenMAPP Builder is based on the XMLPipeDB Utilities and exports GenMAPP-compatible Gene Databases based on data from UniProt and Gene Ontology (GO).
> > > >
> > > > So I should probably start with the XMLPipeDB Utilities which are where? I don't see any in the basic distribution or are they not standalone and called from the command line?
> > > >
> > > > Thanks!
> > > >
> > > > Richard
> > >
> > >
> > > <ATT00001..txt><ATT00002..txt>
> >
> >
> > ------------------------------------------------------------------------------
> > The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> > Pinpoint memory and threading errors before they happen.
> > Find and fix more than 250 security defects in the development cycle.
> > Locate bottlenecks in serial and parallel code that limit performance.
> > http://p.sf.net/sfu/intel-dev2devfeb
> > _______________________________________________
> > xmlpipedb-developer mailing list
> > xml...@li...
> > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> >
> > <ATT00001..txt><ATT00002..txt>
>
>
> ------------------------------------------------------------------------------
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> http://p.sf.net/sfu/intel-dev2devfeb
> _______________________________________________
> xmlpipedb-developer mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>
> <ATT00001..txt><ATT00002..txt>
|
|
From: Richard B. <rbr...@gm...> - 2011-02-17 23:04:18
|
Sorry been slammed with a programming assignment that kept needing continued
iteration and it has been all consuming until last night. But I did get a
chance to work with your comments and review the code again with a different
mind set.
Yes, I examined the gmbuilder.properties file ( the query is also in the
MycobacteriumTuberculosisUniProtSpeciesProfile which I mentioned in a
previous email ) but I don't think I see what you mean regarding the XML
count.
I understood that: *mycobacteriumtuberculosis_query_level0=select count(*)
from genenametype where type = 'ordered locus' and value like 'Rv%'*; was
the db query but don't see which is the XML count... or do they share the
same query and you are saying that XML count doesn't recognize and therefore
cannot use the 'Rv%' parameter?
Richard
On Sat, Feb 12, 2011 at 11:46 PM, John David N. Dionisio <do...@lm...>wrote:
> Hi Rich,
>
> Sorry for the delay. Had some distractions coming into the weekend.
>
> You've looked at the code; have you looked at gmbuilder.properties? (I may
> have mentioned it a few e-mails ago, just as you were starting to dig into
> this)
>
> On the copy I have, the M. tuberculosis block looks like this (indentation
> is mine to set it apart):
>
> # Mycobacterium tuberculosis
> mycobacteriumtuberculosis_level_amount=1
>
>
> mycobacteriumtuberculosis_element_level0=uniprot/entry/gene/name&type&ordered
> locus
>
> mycobacteriumtuberculosis_query_level0=select count(*) from
> genenametype where type = 'ordered locus' and value like 'Rv%';
>
> mycobacteriumtuberculosis_table_name_level0=Ordered Locus
>
> There, I think, is the rub. Notice that the XML count does not filter on
> RV%. The SQL query does.
>
> Unfortunately, I don't think the TallyEngine can include selective
> filtering in the XML counts. If the need to do selective filtering on XML
> is necessary, then I think we're looking at a new functionality for you to
> implement (or, if this throws things off too much, this may have to be noted
> somewhere, that the XML vs. database counts may be off because the database
> count is doing some text-based filtering but the XML count does not).
>
> What does xmlpipedb-match say? That will at least tell you whether the
> 'RV%' count is indeed correct.
>
> John David N. Dionisio, PhD
> Associate Professor, Computer Science
> Loyola Marymount University
>
>
>
> On Feb 11, 2011, at 4:52 PM, Richard Brous wrote:
>
> > OK here is what I was able to put together from the past few hours of
> code review:
> >
> > MycobacteriumTuberculosisUniProtSpeciesProfile.java:
> > -reveals that after the 2 System table modifications are made adding
> species name and link, a PreparedStatement is instantiated which builds and
> calls the base query.
> >
> > -The base query called is: ("SELECT value, type " + "FROM genenametype
> INNER JOIN entrytype_genetype " + "ON(entrytype_genetype_name_hjid =
> entrytype_genetype.hjid) " + "WHERE type = 'ordered locus' and value like
> 'Rv%' and entrytype_gene_hjid = ?")
> >
> > -So its looking in 'ordered locus' table/column for any tuple that starts
> with Rv (followed by any substring) and entrytype_gene_hjid = ? .
> > The 'like' comparator and % usage are clear with the 'type'
> entrytype_gene_hjid = ?
> >
> > -To me it seems the query makes sense so the problem is likely elsewhere.
> >
> > GenMappBuilder.java:
> > -I found method doTallies() at code line 895 which:
> > Instantiates a Configuration called hibernateConfiguration and assigns to
> it the current hibernate configuration
> > Validates database settings by analyzing hibernateConfiguration
> > Instantiates a CriterionList for uniprot and assigns to it
> TallyType.UNIPROT
> > Instantiates a CriterionList for go and assigns to it TallyType.GO
> > Determines if both xml files exist
> > Then getTallyResultsXML and getTallyResultsDatabase are run on both xml
> files and their respective CriterionList
> > Results are then formatted for display in a table.
> >
> > -So enum TallyType which means that they are the only valid datatypes
> which TallyEngine accepts... go to know ...
> >
> > -Based on the screen shot of Tally Engine it would seem that both
> getTallyResultsXML() and getTallyResultsDatabase() are incorrectly
> returning. Likely due to both using an incorrect query (as we previously
> supposed). But where are the queries?... the more I dig the more I think
> they are in the criterial all the work is done against.
> >
> > continuing the review:
> > getTallyResultsXML() calls Tally Engine instance method
> getXmlFileCounts(xmlFile)
> > getTallyResultsDatabase() calls Tally Engine instance method
> getDbcounts(new QueryEngine(hibernateConfiguration)
> > Both of these instanced methods originate from TallyEngine.java...
> >
> > TallyEngine.java:
> >
> > getXmlFileCounts() calls digestXmlFile() which instantiates a digester
> then processes against criteria... but this quickly becomes confusing and is
> hard to follow
> >
> > getDbcounts() then starts a db session and executes a query but then I
> also get a bit lost with my limited db knowledge.
> >
> >
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >
> > OVERALL I think I'm getting closer to the issues but I still feel as if
> I'm missing some understanding to proceed further. Can you pass along some
> of that Dondi insight and steer me in the right direction? =D
> >
> > -DB Tally - Not having taken databases yet certainly is limiting my
> ability determine where the "criteria" are being set and how they are
> followed during session activities. Also is the query we have been looking
> for this whole time in the criteria or someplace else?
> >
> > -XML Tally - again is the query contained within the criteria that
> digestXmlFile() uses to parse?
> >
> > Richard
> >
> >
> > On Mon, Feb 7, 2011 at 5:50 PM, John David N. Dionisio <do...@lm...>
> wrote:
> > Right, schema issues are unlikely. Most count discrepancies like this
> that I've seen have boiled down to forming the right query. Then, knowing
> the right query (in both XML and SQL), it's a matter of making sure that
> TallyEngine asks that same query.
> >
> > John David N. Dionisio, PhD
> > Associate Professor, Computer Science
> > Loyola Marymount University
> >
> >
> > On Feb 7, 2011, at 5:48 PM, Richard Brous wrote:
> >
> > > OK, so based on your approach:
> > >
> > > 1. I'll start with reviewing the queries for xmlpipedb-match and sql
> queries needed for the respective results as you requested.
> > >
> > > I was also thinking I may need to review the schema from xml into
> postgres but the issue isn't likely a schema error. The error most likely
> lies in how xmlpipedbutils queries the data from xml source and writes to
> the tables what it returns?
> > >
> > > 2. I'll review the code: trace the entrance of tally engine in the
> gmbuilder code then follow it through the xmlpipedbutils.
> > >
> > > Richard
> > >
> > > On Sat, Feb 5, 2011 at 10:28 AM, John David N. Dionisio <do...@lm...>
> wrote:
> > > Just wanted to confirm (since I wasn't sure in the first e-mail) ---
> the XMLPipeDB Utilities source code is in trunk/xmlpipedbutils in
> SourceForge's Subversion repo.
> > >
> > > John David N. Dionisio, PhD
> > > Associate Professor, Computer Science
> > > Loyola Marymount University
> > >
> > >
> > >
> > > On Feb 5, 2011, at 10:02 AM, Richard Brous wrote:
> > >
> > > > Hi Dondi,
> > > >
> > > > So I'm at the point in working with M tuberculosis that I was able to
> exactly reproduce Dr. Dahlquist's problematic TallyEngine results.
> > > >
> > > > gmb2b60 Results
> > > >
> > > >
> > > >
> > > > Now the proverbial question - What next to solve the Ordered Locus
> import/count issue?
> > > >
> > > > **********************************************
> > > > Here is my thought process:
> > > >
> > > > Step 1: How does the import process work at the high level?
> (obviously correct me if I'm wrong)
> > > >
> > > > I believe that basically as each XML tag is read, it is placed in the
> proper Postgres table(s) based on some criteria. There is also likely some
> sort of check that each individual tag is in valid XML format unless we
> don't care at this stage (care at export) or maybe the parser just skips
> over and goes on to the next .
> > > >
> > > > Step 2: What could be the problem?
> > > >
> > > > Either -
> > > > a. XML tags are being parsed incorrectly (ignored/skipped)?
> > > > b. Decision criteria of which table they should be added to?
> > > >
> > > > **********************************************
> > > >
> > > > I read on the sourceforge wiki:
> > > >
> > > > XMLPipeDB has a modular architecture with three components that may
> be used separately or together. XSD-to-DB reads an XSD (XML Schema
> Definition) and automatically generates an SQL schema, Java classes, and
> Hibernate mappings. XMLPipeDB Utilities provides functionality for
> configuring the database, importing data, and performing queries. GenMAPP
> Builder is based on the XMLPipeDB Utilities and exports GenMAPP-compatible
> Gene Databases based on data from UniProt and Gene Ontology (GO).
> > > >
> > > > So I should probably start with the XMLPipeDB Utilities which are
> where? I don't see any in the basic distribution or are they not standalone
> and called from the command line?
> > > >
> > > > Thanks!
> > > >
> > > > Richard
> > >
> > >
> > > <ATT00001..txt><ATT00002..txt>
> >
> >
> >
> ------------------------------------------------------------------------------
> > The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> > Pinpoint memory and threading errors before they happen.
> > Find and fix more than 250 security defects in the development cycle.
> > Locate bottlenecks in serial and parallel code that limit performance.
> > http://p.sf.net/sfu/intel-dev2devfeb
> > _______________________________________________
> > xmlpipedb-developer mailing list
> > xml...@li...
> > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
> >
> > <ATT00001..txt><ATT00002..txt>
>
>
>
> ------------------------------------------------------------------------------
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> http://p.sf.net/sfu/intel-dev2devfeb
> _______________________________________________
> xmlpipedb-developer mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>
|
|
From: John D. N. D. <do...@lm...> - 2011-02-17 18:58:57
|
Hi Rich, Dr. Dahlquist and I had an opportunity to go over things, and we determined that, for M. tuberculosis, between the TallyEngine issues and the missing IDs in the .gdb, the missing IDs should take precedence. With the TallyEngine business, we at least have backup, if more manual, methods for doing counts, with xmlpipedb-match for XML and direct SQL for the relational database. So for now, let's "hold these thoughts" re: TallyEngine, and focus on those IDs, plus writing the code that harvests those IDs once we know why they didn't show up in the first place. Let's start with the most recent M. tuberculosis testing report --- the one in your wiki pages was this (if there's something more recent, Kam, just send it our way): http://sourceforge.net/apps/mediawiki/xmlpipedb/index.php?title=Gene_Database_Testing_Report_M._tuberculosis_20101213 The "Match Results" section of that page gives you a concrete starting point for tracking down the missing IDs. Start by finding where those IDs are in the XML. I recommend doing the ones with low counts, like Rv####B and Rv####D, since those are easy to sort through individually. Typically, once we have located where those missing IDs are in the XML file, the others tend to pop up in similar locations. Once those locations are identified, we then need to track where they show up in the relational database. This shouldn't be too hard, as the table schema generally corresponds to the XML structure. Finally, once we know where the IDs are in the database, *then* we touch the species profile Java code. The nature of the change is to add new queries that insert these IDs into the .gdb. Of the existing code, the A. thaliana species profile illustrates this most strongly. I believe P. falciparum has a more limited version of this type of customization as well. So, let's get on those missing IDs. One more item --- things have loosened up enough for me that we can start doing weekly calls/meetings to keep the pace up. My schedule is mostly flexible, with Fridays being the only exception, and for just next week, I will be away at a conference until Wednesday night. Outside of that, let us know when we can set a regular call/meeting slot. OK, that's it for now; let us know if you have any questions. Thanks! John David N. Dionisio, PhD Associate Professor, Computer Science Loyola Marymount University On Feb 11, 2011, at 4:52 PM, Richard Brous wrote: > OK here is what I was able to put together from the past few hours of code review: > > MycobacteriumTuberculosisUniProtSpeciesProfile.java: > -reveals that after the 2 System table modifications are made adding species name and link, a PreparedStatement is instantiated which builds and calls the base query. > > -The base query called is: ("SELECT value, type " + "FROM genenametype INNER JOIN entrytype_genetype " + "ON(entrytype_genetype_name_hjid = entrytype_genetype.hjid) " + "WHERE type = 'ordered locus' and value like 'Rv%' and entrytype_gene_hjid = ?") > > -So its looking in 'ordered locus' table/column for any tuple that starts with Rv (followed by any substring) and entrytype_gene_hjid = ? . > The 'like' comparator and % usage are clear with the 'type' entrytype_gene_hjid = ? > > -To me it seems the query makes sense so the problem is likely elsewhere. > > GenMappBuilder.java: > -I found method doTallies() at code line 895 which: > Instantiates a Configuration called hibernateConfiguration and assigns to it the current hibernate configuration > Validates database settings by analyzing hibernateConfiguration > Instantiates a CriterionList for uniprot and assigns to it TallyType.UNIPROT > Instantiates a CriterionList for go and assigns to it TallyType.GO > Determines if both xml files exist > Then getTallyResultsXML and getTallyResultsDatabase are run on both xml files and their respective CriterionList > Results are then formatted for display in a table. > > -So enum TallyType which means that they are the only valid datatypes which TallyEngine accepts... go to know ... > > -Based on the screen shot of Tally Engine it would seem that both getTallyResultsXML() and getTallyResultsDatabase() are incorrectly returning. Likely due to both using an incorrect query (as we previously supposed). But where are the queries?... the more I dig the more I think they are in the criterial all the work is done against. > > continuing the review: > getTallyResultsXML() calls Tally Engine instance method getXmlFileCounts(xmlFile) > getTallyResultsDatabase() calls Tally Engine instance method getDbcounts(new QueryEngine(hibernateConfiguration) > Both of these instanced methods originate from TallyEngine.java... > > TallyEngine.java: > > getXmlFileCounts() calls digestXmlFile() which instantiates a digester then processes against criteria... but this quickly becomes confusing and is hard to follow > > getDbcounts() then starts a db session and executes a query but then I also get a bit lost with my limited db knowledge. > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > > OVERALL I think I'm getting closer to the issues but I still feel as if I'm missing some understanding to proceed further. Can you pass along some of that Dondi insight and steer me in the right direction? =D > > -DB Tally - Not having taken databases yet certainly is limiting my ability determine where the "criteria" are being set and how they are followed during session activities. Also is the query we have been looking for this whole time in the criteria or someplace else? > > -XML Tally - again is the query contained within the criteria that digestXmlFile() uses to parse? > > Richard > > > On Mon, Feb 7, 2011 at 5:50 PM, John David N. Dionisio <do...@lm...> wrote: > Right, schema issues are unlikely. Most count discrepancies like this that I've seen have boiled down to forming the right query. Then, knowing the right query (in both XML and SQL), it's a matter of making sure that TallyEngine asks that same query. > > John David N. Dionisio, PhD > Associate Professor, Computer Science > Loyola Marymount University > > > On Feb 7, 2011, at 5:48 PM, Richard Brous wrote: > > > OK, so based on your approach: > > > > 1. I'll start with reviewing the queries for xmlpipedb-match and sql queries needed for the respective results as you requested. > > > > I was also thinking I may need to review the schema from xml into postgres but the issue isn't likely a schema error. The error most likely lies in how xmlpipedbutils queries the data from xml source and writes to the tables what it returns? > > > > 2. I'll review the code: trace the entrance of tally engine in the gmbuilder code then follow it through the xmlpipedbutils. > > > > Richard > > > > On Sat, Feb 5, 2011 at 10:28 AM, John David N. Dionisio <do...@lm...> wrote: > > Just wanted to confirm (since I wasn't sure in the first e-mail) --- the XMLPipeDB Utilities source code is in trunk/xmlpipedbutils in SourceForge's Subversion repo. > > > > John David N. Dionisio, PhD > > Associate Professor, Computer Science > > Loyola Marymount University > > > > > > > > On Feb 5, 2011, at 10:02 AM, Richard Brous wrote: > > > > > Hi Dondi, > > > > > > So I'm at the point in working with M tuberculosis that I was able to exactly reproduce Dr. Dahlquist's problematic TallyEngine results. > > > > > > gmb2b60 Results > > > > > > > > > > > > Now the proverbial question - What next to solve the Ordered Locus import/count issue? > > > > > > ********************************************** > > > Here is my thought process: > > > > > > Step 1: How does the import process work at the high level? (obviously correct me if I'm wrong) > > > > > > I believe that basically as each XML tag is read, it is placed in the proper Postgres table(s) based on some criteria. There is also likely some sort of check that each individual tag is in valid XML format unless we don't care at this stage (care at export) or maybe the parser just skips over and goes on to the next . > > > > > > Step 2: What could be the problem? > > > > > > Either - > > > a. XML tags are being parsed incorrectly (ignored/skipped)? > > > b. Decision criteria of which table they should be added to? > > > > > > ********************************************** > > > > > > I read on the sourceforge wiki: > > > > > > XMLPipeDB has a modular architecture with three components that may be used separately or together. XSD-to-DB reads an XSD (XML Schema Definition) and automatically generates an SQL schema, Java classes, and Hibernate mappings. XMLPipeDB Utilities provides functionality for configuring the database, importing data, and performing queries. GenMAPP Builder is based on the XMLPipeDB Utilities and exports GenMAPP-compatible Gene Databases based on data from UniProt and Gene Ontology (GO). > > > > > > So I should probably start with the XMLPipeDB Utilities which are where? I don't see any in the basic distribution or are they not standalone and called from the command line? > > > > > > Thanks! > > > > > > Richard > > > > > > <ATT00001..txt><ATT00002..txt> > > > ------------------------------------------------------------------------------ > The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: > Pinpoint memory and threading errors before they happen. > Find and fix more than 250 security defects in the development cycle. > Locate bottlenecks in serial and parallel code that limit performance. > http://p.sf.net/sfu/intel-dev2devfeb > _______________________________________________ > xmlpipedb-developer mailing list > xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > > <ATT00001..txt><ATT00002..txt> |
|
From: John D. N. D. <do...@lm...> - 2011-02-13 07:47:08
|
Hi Rich,
Sorry for the delay. Had some distractions coming into the weekend.
You've looked at the code; have you looked at gmbuilder.properties? (I may have mentioned it a few e-mails ago, just as you were starting to dig into this)
On the copy I have, the M. tuberculosis block looks like this (indentation is mine to set it apart):
# Mycobacterium tuberculosis
mycobacteriumtuberculosis_level_amount=1
mycobacteriumtuberculosis_element_level0=uniprot/entry/gene/name&type&ordered locus
mycobacteriumtuberculosis_query_level0=select count(*) from genenametype where type = 'ordered locus' and value like 'Rv%';
mycobacteriumtuberculosis_table_name_level0=Ordered Locus
There, I think, is the rub. Notice that the XML count does not filter on RV%. The SQL query does.
Unfortunately, I don't think the TallyEngine can include selective filtering in the XML counts. If the need to do selective filtering on XML is necessary, then I think we're looking at a new functionality for you to implement (or, if this throws things off too much, this may have to be noted somewhere, that the XML vs. database counts may be off because the database count is doing some text-based filtering but the XML count does not).
What does xmlpipedb-match say? That will at least tell you whether the 'RV%' count is indeed correct.
John David N. Dionisio, PhD
Associate Professor, Computer Science
Loyola Marymount University
On Feb 11, 2011, at 4:52 PM, Richard Brous wrote:
> OK here is what I was able to put together from the past few hours of code review:
>
> MycobacteriumTuberculosisUniProtSpeciesProfile.java:
> -reveals that after the 2 System table modifications are made adding species name and link, a PreparedStatement is instantiated which builds and calls the base query.
>
> -The base query called is: ("SELECT value, type " + "FROM genenametype INNER JOIN entrytype_genetype " + "ON(entrytype_genetype_name_hjid = entrytype_genetype.hjid) " + "WHERE type = 'ordered locus' and value like 'Rv%' and entrytype_gene_hjid = ?")
>
> -So its looking in 'ordered locus' table/column for any tuple that starts with Rv (followed by any substring) and entrytype_gene_hjid = ? .
> The 'like' comparator and % usage are clear with the 'type' entrytype_gene_hjid = ?
>
> -To me it seems the query makes sense so the problem is likely elsewhere.
>
> GenMappBuilder.java:
> -I found method doTallies() at code line 895 which:
> Instantiates a Configuration called hibernateConfiguration and assigns to it the current hibernate configuration
> Validates database settings by analyzing hibernateConfiguration
> Instantiates a CriterionList for uniprot and assigns to it TallyType.UNIPROT
> Instantiates a CriterionList for go and assigns to it TallyType.GO
> Determines if both xml files exist
> Then getTallyResultsXML and getTallyResultsDatabase are run on both xml files and their respective CriterionList
> Results are then formatted for display in a table.
>
> -So enum TallyType which means that they are the only valid datatypes which TallyEngine accepts... go to know ...
>
> -Based on the screen shot of Tally Engine it would seem that both getTallyResultsXML() and getTallyResultsDatabase() are incorrectly returning. Likely due to both using an incorrect query (as we previously supposed). But where are the queries?... the more I dig the more I think they are in the criterial all the work is done against.
>
> continuing the review:
> getTallyResultsXML() calls Tally Engine instance method getXmlFileCounts(xmlFile)
> getTallyResultsDatabase() calls Tally Engine instance method getDbcounts(new QueryEngine(hibernateConfiguration)
> Both of these instanced methods originate from TallyEngine.java...
>
> TallyEngine.java:
>
> getXmlFileCounts() calls digestXmlFile() which instantiates a digester then processes against criteria... but this quickly becomes confusing and is hard to follow
>
> getDbcounts() then starts a db session and executes a query but then I also get a bit lost with my limited db knowledge.
>
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> OVERALL I think I'm getting closer to the issues but I still feel as if I'm missing some understanding to proceed further. Can you pass along some of that Dondi insight and steer me in the right direction? =D
>
> -DB Tally - Not having taken databases yet certainly is limiting my ability determine where the "criteria" are being set and how they are followed during session activities. Also is the query we have been looking for this whole time in the criteria or someplace else?
>
> -XML Tally - again is the query contained within the criteria that digestXmlFile() uses to parse?
>
> Richard
>
>
> On Mon, Feb 7, 2011 at 5:50 PM, John David N. Dionisio <do...@lm...> wrote:
> Right, schema issues are unlikely. Most count discrepancies like this that I've seen have boiled down to forming the right query. Then, knowing the right query (in both XML and SQL), it's a matter of making sure that TallyEngine asks that same query.
>
> John David N. Dionisio, PhD
> Associate Professor, Computer Science
> Loyola Marymount University
>
>
> On Feb 7, 2011, at 5:48 PM, Richard Brous wrote:
>
> > OK, so based on your approach:
> >
> > 1. I'll start with reviewing the queries for xmlpipedb-match and sql queries needed for the respective results as you requested.
> >
> > I was also thinking I may need to review the schema from xml into postgres but the issue isn't likely a schema error. The error most likely lies in how xmlpipedbutils queries the data from xml source and writes to the tables what it returns?
> >
> > 2. I'll review the code: trace the entrance of tally engine in the gmbuilder code then follow it through the xmlpipedbutils.
> >
> > Richard
> >
> > On Sat, Feb 5, 2011 at 10:28 AM, John David N. Dionisio <do...@lm...> wrote:
> > Just wanted to confirm (since I wasn't sure in the first e-mail) --- the XMLPipeDB Utilities source code is in trunk/xmlpipedbutils in SourceForge's Subversion repo.
> >
> > John David N. Dionisio, PhD
> > Associate Professor, Computer Science
> > Loyola Marymount University
> >
> >
> >
> > On Feb 5, 2011, at 10:02 AM, Richard Brous wrote:
> >
> > > Hi Dondi,
> > >
> > > So I'm at the point in working with M tuberculosis that I was able to exactly reproduce Dr. Dahlquist's problematic TallyEngine results.
> > >
> > > gmb2b60 Results
> > >
> > >
> > >
> > > Now the proverbial question - What next to solve the Ordered Locus import/count issue?
> > >
> > > **********************************************
> > > Here is my thought process:
> > >
> > > Step 1: How does the import process work at the high level? (obviously correct me if I'm wrong)
> > >
> > > I believe that basically as each XML tag is read, it is placed in the proper Postgres table(s) based on some criteria. There is also likely some sort of check that each individual tag is in valid XML format unless we don't care at this stage (care at export) or maybe the parser just skips over and goes on to the next .
> > >
> > > Step 2: What could be the problem?
> > >
> > > Either -
> > > a. XML tags are being parsed incorrectly (ignored/skipped)?
> > > b. Decision criteria of which table they should be added to?
> > >
> > > **********************************************
> > >
> > > I read on the sourceforge wiki:
> > >
> > > XMLPipeDB has a modular architecture with three components that may be used separately or together. XSD-to-DB reads an XSD (XML Schema Definition) and automatically generates an SQL schema, Java classes, and Hibernate mappings. XMLPipeDB Utilities provides functionality for configuring the database, importing data, and performing queries. GenMAPP Builder is based on the XMLPipeDB Utilities and exports GenMAPP-compatible Gene Databases based on data from UniProt and Gene Ontology (GO).
> > >
> > > So I should probably start with the XMLPipeDB Utilities which are where? I don't see any in the basic distribution or are they not standalone and called from the command line?
> > >
> > > Thanks!
> > >
> > > Richard
> >
> >
> > <ATT00001..txt><ATT00002..txt>
>
>
> ------------------------------------------------------------------------------
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> http://p.sf.net/sfu/intel-dev2devfeb
> _______________________________________________
> xmlpipedb-developer mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>
> <ATT00001..txt><ATT00002..txt>
|
|
From: Richard B. <rbr...@gm...> - 2011-02-12 00:52:13
|
*OK here is what I was able to put together from the past few hours of code
review:*
MycobacteriumTuberculosisUniProtSpeciesProfile.java:
-reveals that after the 2 System table modifications are made adding species
name and link, a PreparedStatement is instantiated which builds and calls
the base query.
-The base query called is: ("SELECT value, type " + "FROM genenametype INNER
JOIN entrytype_genetype " + "ON(entrytype_genetype_name_hjid =
entrytype_genetype.hjid) " + "WHERE type = 'ordered locus' and value like
'Rv%' and entrytype_gene_hjid = ?")
-So its looking in 'ordered locus' table/column for any tuple that starts
with Rv (followed by any substring) and entrytype_gene_hjid = ? .
The 'like' comparator and % usage are clear with the 'type'
entrytype_gene_hjid = ?
-To me it seems the query makes sense so the problem is likely elsewhere.
GenMappBuilder.java:
-I found method doTallies() at code line 895 which:
Instantiates a Configuration called hibernateConfiguration and assigns to it
the current hibernate configuration
Validates database settings by analyzing hibernateConfiguration
Instantiates a CriterionList for uniprot and assigns to it TallyType.UNIPROT
Instantiates a CriterionList for go and assigns to it TallyType.GO
Determines if both xml files exist
Then getTallyResultsXML and getTallyResultsDatabase are run on both xml
files and their respective CriterionList
Results are then formatted for display in a table.
-So enum TallyType which means that they are the only valid datatypes which
TallyEngine accepts... go to know ...
-Based on the screen shot of Tally Engine it would seem that
both getTallyResultsXML() and getTallyResultsDatabase() are incorrectly
returning. Likely due to both using an incorrect query (as we previously
supposed). But where are the queries?... the more I dig the more I think
they are in the criterial all the work is done against.
continuing the review:
getTallyResultsXML() calls Tally Engine instance method
getXmlFileCounts(xmlFile)
getTallyResultsDatabase() calls Tally Engine instance method getDbcounts(*
new* QueryEngine(hibernateConfiguration)
Both of these instanced methods originate from TallyEngine.java...
TallyEngine.java:
getXmlFileCounts() calls digestXmlFile() which instantiates a digester then
processes against criteria... but this quickly becomes confusing and is hard
to follow
getDbcounts() then starts a db session and executes a query but then I also
get a bit lost with my limited db knowledge.
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
OVERALL I think I'm getting closer to the issues but I still feel as if I'm
missing some understanding to proceed further. Can you pass along some of
that Dondi insight and steer me in the right direction? =D
-DB Tally - Not having taken databases yet certainly is limiting my ability
determine where the "criteria" are being set and how they are followed
during session activities. Also is the query we have been looking for this
whole time in the criteria or someplace else?
-XML Tally - again is the query contained within the criteria that
digestXmlFile() uses to parse?
Richard
On Mon, Feb 7, 2011 at 5:50 PM, John David N. Dionisio <do...@lm...>wrote:
> Right, schema issues are unlikely. Most count discrepancies like this that
> I've seen have boiled down to forming the right query. Then, knowing the
> right query (in both XML and SQL), it's a matter of making sure that
> TallyEngine asks that same query.
>
> John David N. Dionisio, PhD
> Associate Professor, Computer Science
> Loyola Marymount University
>
>
> On Feb 7, 2011, at 5:48 PM, Richard Brous wrote:
>
> > OK, so based on your approach:
> >
> > 1. I'll start with reviewing the queries for xmlpipedb-match and sql
> queries needed for the respective results as you requested.
> >
> > I was also thinking I may need to review the schema from xml into
> postgres but the issue isn't likely a schema error. The error most likely
> lies in how xmlpipedbutils queries the data from xml source and writes to
> the tables what it returns?
> >
> > 2. I'll review the code: trace the entrance of tally engine in the
> gmbuilder code then follow it through the xmlpipedbutils.
> >
> > Richard
> >
> > On Sat, Feb 5, 2011 at 10:28 AM, John David N. Dionisio <do...@lm...>
> wrote:
> > Just wanted to confirm (since I wasn't sure in the first e-mail) --- the
> XMLPipeDB Utilities source code is in trunk/xmlpipedbutils in SourceForge's
> Subversion repo.
> >
> > John David N. Dionisio, PhD
> > Associate Professor, Computer Science
> > Loyola Marymount University
> >
> >
> >
> > On Feb 5, 2011, at 10:02 AM, Richard Brous wrote:
> >
> > > Hi Dondi,
> > >
> > > So I'm at the point in working with M tuberculosis that I was able to
> exactly reproduce Dr. Dahlquist's problematic TallyEngine results.
> > >
> > > gmb2b60 Results
> > >
> > >
> > >
> > > Now the proverbial question - What next to solve the Ordered Locus
> import/count issue?
> > >
> > > **********************************************
> > > Here is my thought process:
> > >
> > > Step 1: How does the import process work at the high level? (obviously
> correct me if I'm wrong)
> > >
> > > I believe that basically as each XML tag is read, it is placed in the
> proper Postgres table(s) based on some criteria. There is also likely some
> sort of check that each individual tag is in valid XML format unless we
> don't care at this stage (care at export) or maybe the parser just skips
> over and goes on to the next .
> > >
> > > Step 2: What could be the problem?
> > >
> > > Either -
> > > a. XML tags are being parsed incorrectly (ignored/skipped)?
> > > b. Decision criteria of which table they should be added to?
> > >
> > > **********************************************
> > >
> > > I read on the sourceforge wiki:
> > >
> > > XMLPipeDB has a modular architecture with three components that may be
> used separately or together. XSD-to-DB reads an XSD (XML Schema Definition)
> and automatically generates an SQL schema, Java classes, and Hibernate
> mappings. XMLPipeDB Utilities provides functionality for configuring the
> database, importing data, and performing queries. GenMAPP Builder is based
> on the XMLPipeDB Utilities and exports GenMAPP-compatible Gene Databases
> based on data from UniProt and Gene Ontology (GO).
> > >
> > > So I should probably start with the XMLPipeDB Utilities which are
> where? I don't see any in the basic distribution or are they not standalone
> and called from the command line?
> > >
> > > Thanks!
> > >
> > > Richard
> >
> >
> > <ATT00001..txt><ATT00002..txt>
>
>
>
> ------------------------------------------------------------------------------
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> http://p.sf.net/sfu/intel-dev2devfeb
> _______________________________________________
> xmlpipedb-developer mailing list
> xml...@li...
> https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer
>
|
|
From: Kam D. <kda...@lm...> - 2011-02-11 21:47:18
|
Hi, Thanks for letting us know--hope he gets better soon! Best, Kam At 11:38 AM 2/11/2011, you wrote: >Have to pick up sick boy from school > >Richard. > >Sent from my iPhone > >------------------------------------------------------------------------------ >The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: >Pinpoint memory and threading errors before they happen. >Find and fix more than 250 security defects in the development cycle. >Locate bottlenecks in serial and parallel code that limit performance. >http://p.sf.net/sfu/intel-dev2devfeb >_______________________________________________ >xmlpipedb-developer mailing list >xml...@li... >https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer |
|
From: Richard B. <rbr...@gm...> - 2011-02-11 19:38:48
|
Have to pick up sick boy from school Richard. Sent from my iPhone |
|
From: John D. N. D. <do...@lm...> - 2011-02-08 21:20:36
|
Ah OK, I see where there were misunderstandings now. Yes, the mere act of uploading via the File Manager constitutes a "release" in SourceForge's eyes. Whatever shows up as the default download is then a separate setting. Sounds like we've cleared things up then. Carry on :) John David N. Dionisio, PhD Associate Professor, Computer Science Loyola Marymount University On Feb 8, 2011, at 12:37 PM, Richard Brous <rbr...@gm...> wrote: > Couple things: > > Think I may be using confusing terms so I'll try to be more specific... > > I thought that there was a separate process that needed to be triggered to increment the released version within SourceForge. If in fact all I needed to do was to zip up the new build and place it and a new readme.txt on SourceForge then I have done so. > > I also have rights to make the zip file the default download. My error was attempting the set the 2.0b61 folder as the default download folder which isn't an option. > > Sorry for my lack of experience with with SourceFoge but i'll remember for next time! > > Richard > > On Tue, Feb 8, 2011 at 9:57 AM, John David N. Dionisio <do...@lm...> wrote: > Greetings, > > The reason it didn't show up on the front page is because it wasn't marked as the default download (see attached screenshot). To see that panel, click on "i" then click on the appropriate checkbox. > > Rich, please verify if, when you mention that you didn't have rights, you meant that this panel does not appear (or is not editable) under your account. Or if you meant something else, let me know. > > Meanwhile, I went ahead and checked Windows as the default download. Should appear on the front page now. > > John David N. Dionisio, PhD > Associate Professor, Computer Science > Loyola Marymount University > > > > > On Feb 8, 2011, at 9:30 AM, Richard Brous wrote: > > > Yes, because I don't seem to have rights to officially release the new version. > > > > I mentioned that in my last email on yesterday's thread. > > > > 2b61 zip is complete and ready. > > > > Rb > > > > Sent from my iPhone > > > > On Feb 8, 2011, at 9:07 AM, Kam Dahlquist <kda...@lm...> wrote: > > > >> Hi, > >> > >> Do you guys know why the SourceForge home page for XMLPipeDB still has b60 as the latest file? To download b61, I have to show all files. > >> > >> Thanks, > >> Dr. D > >> > >> At 05:50 PM 2/7/2011, Richard Brous wrote: > >>> Meant rev the release number =D > >>> > >>> On the readme I followed the format of b60 which I may have uploaded incorrectly since I released it as well. Will correct right now. > >>> > >>> Richard > >>> > >>> On Mon, Feb 7, 2011 at 5:36 PM, John David N. Dionisio <do...@lm...> wrote: > >>> Hi Rich, > >>> > >>> Not sure what you mean by "rev on the release" --- you successfully uploaded, so that should be enough. > >>> > >>> Note though that the readme that you uploaded is called "gmbuilder-2_0b61-readme.txt" and not "gmbuilder-2.0b61-readme.txt" (period instead of underscore). > >>> > >>> AFAIK you can upload and reupload the file as much as needed, until the contents of that folder are correct. > >>> > >>> John David N. Dionisio, PhD > >>> Associate Professor, Computer Science > >>> Loyola Marymount University > >>> > >>> > >>> > >>> On Feb 7, 2011, at 5:29 PM, Richard Brous wrote: > >>> > >>> > OK, b61 has had its version updated (string constant), was zipped up and added to sourceforge along with a corresponding readme.txt. > >>> > > >>> > Should be go to go now for rev on the release. Do I have rights to do that if so where on Sourceforge should I go? (I checked project admin tab but no items display) > >>> > > >>> > Richard > >>> > > >>> > On Mon, Feb 7, 2011 at 4:41 PM, John David N. Dionisio <do...@lm...> wrote: > >>> > OK, got it; just holler if you run into issues. BTW Rich, since your "From:" address has changed, the mailing list did not recognize your last message as coming from a member. If you plan to keep using your Gmail address, be sure to subscribe to the mailing list with that address. > >>> > > >>> > John David N. Dionisio, PhD > >>> > Associate Professor, Computer Science > >>> > Loyola Marymount University > >>> > > >>> > > >>> > > >>> > On Feb 7, 2011, at 4:18 PM, Richard Brous wrote: > >>> > > >>> > > I'll give it a try. I already committed the link change up to the repository so there will be no committment issues (this time!). > >>> > > > >>> > > If I run into the same issue as before I'll let you know and you will probably have to handle it. > >>> > > > >>> > > rb > >>> > > > >>> > > On Mon, Feb 7, 2011 at 3:39 PM, John David N. Dionisio <do...@lm...> wrote: > >>> > > Hi Kam, > >>> > > > >>> > > Sure, no problem --- Rich, do you want to do the release yourself, or would you rather that I do it? Let me know. Either way, make sure you're all committed up before releasing. Especially if I'll be the one doing it :) > >>> > > > >>> > > John David N. Dionisio, PhD > >>> > > Associate Professor, Computer Science > >>> > > Loyola Marymount University > >>> > > > >>> > > > >>> > > > >>> > > On Feb 7, 2011, at 2:43 PM, Kam Dahlquist wrote: > >>> > > > >>> > > > Hi Dondi, Rich, > >>> > > > > >>> > > > I'm looking at the M smegmatis gdb that Rich created and the new link > >>> > > > out works and the gdb works with the major functions of > >>> > > > GenMAPP/MAPPFinder. Would it be possible to release a version of > >>> > > > GenMAPP Builder with this change so that I can try an export myself? > >>> > > > > >>> > > > I'm going to try to finish going over the gdb itself so that we can > >>> > > > also release it, but that will take me a little more time. > >>> > > > > >>> > > > Thanks, > >>> > > > Dr. D > >>> > > > > >> ------------------------------------------------------------------------------ > >> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: > >> Pinpoint memory and threading errors before they happen. > >> Find and fix more than 250 security defects in the development cycle. > >> Locate bottlenecks in serial and parallel code that limit performance. > >> http://p.sf.net/sfu/intel-dev2devfeb > >> _______________________________________________ > >> xmlpipedb-developer mailing list > >> xml...@li... > >> https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > > <ATT00001..txt><ATT00002..txt> > > > ------------------------------------------------------------------------------ > The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: > Pinpoint memory and threading errors before they happen. > Find and fix more than 250 security defects in the development cycle. > Locate bottlenecks in serial and parallel code that limit performance. > http://p.sf.net/sfu/intel-dev2devfeb > _______________________________________________ > xmlpipedb-developer mailing list > xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > > > ------------------------------------------------------------------------------ > The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: > Pinpoint memory and threading errors before they happen. > Find and fix more than 250 security defects in the development cycle. > Locate bottlenecks in serial and parallel code that limit performance. > http://p.sf.net/sfu/intel-dev2devfeb > _______________________________________________ > xmlpipedb-developer mailing list > xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer |
|
From: Richard B. <rbr...@gm...> - 2011-02-08 20:37:52
|
Couple things: Think I may be using confusing terms so I'll try to be more specific... I thought that there was a separate process that needed to be triggered to increment the released version within SourceForge. If in fact all I needed to do was to zip up the new build and place it and a new readme.txt on SourceForge then I have done so. I also have rights to make the zip file the default download. My error was attempting the set the 2.0b61 folder as the default download folder which isn't an option. Sorry for my lack of experience with with SourceFoge but i'll remember for next time! Richard On Tue, Feb 8, 2011 at 9:57 AM, John David N. Dionisio <do...@lm...>wrote: > Greetings, > > The reason it didn't show up on the front page is because it wasn't marked > as the default download (see attached screenshot). To see that panel, click > on "i" then click on the appropriate checkbox. > > Rich, please verify if, when you mention that you didn't have rights, you > meant that this panel does not appear (or is not editable) under your > account. Or if you meant something else, let me know. > > Meanwhile, I went ahead and checked Windows as the default download. > Should appear on the front page now. > > John David N. Dionisio, PhD > Associate Professor, Computer Science > Loyola Marymount University > > > > > On Feb 8, 2011, at 9:30 AM, Richard Brous wrote: > > > Yes, because I don't seem to have rights to officially release the new > version. > > > > I mentioned that in my last email on yesterday's thread. > > > > 2b61 zip is complete and ready. > > > > Rb > > > > Sent from my iPhone > > > > On Feb 8, 2011, at 9:07 AM, Kam Dahlquist <kda...@lm...> wrote: > > > >> Hi, > >> > >> Do you guys know why the SourceForge home page for XMLPipeDB still has > b60 as the latest file? To download b61, I have to show all files. > >> > >> Thanks, > >> Dr. D > >> > >> At 05:50 PM 2/7/2011, Richard Brous wrote: > >>> Meant rev the release number =D > >>> > >>> On the readme I followed the format of b60 which I may have uploaded > incorrectly since I released it as well. Will correct right now. > >>> > >>> Richard > >>> > >>> On Mon, Feb 7, 2011 at 5:36 PM, John David N. Dionisio <do...@lm...> > wrote: > >>> Hi Rich, > >>> > >>> Not sure what you mean by "rev on the release" --- you successfully > uploaded, so that should be enough. > >>> > >>> Note though that the readme that you uploaded is called > "gmbuilder-2_0b61-readme.txt" and not "gmbuilder-2.0b61-readme.txt" (period > instead of underscore). > >>> > >>> AFAIK you can upload and reupload the file as much as needed, until the > contents of that folder are correct. > >>> > >>> John David N. Dionisio, PhD > >>> Associate Professor, Computer Science > >>> Loyola Marymount University > >>> > >>> > >>> > >>> On Feb 7, 2011, at 5:29 PM, Richard Brous wrote: > >>> > >>> > OK, b61 has had its version updated (string constant), was zipped up > and added to sourceforge along with a corresponding readme.txt. > >>> > > >>> > Should be go to go now for rev on the release. Do I have rights to do > that if so where on Sourceforge should I go? (I checked project admin tab > but no items display) > >>> > > >>> > Richard > >>> > > >>> > On Mon, Feb 7, 2011 at 4:41 PM, John David N. Dionisio < > do...@lm...> wrote: > >>> > OK, got it; just holler if you run into issues. BTW Rich, since your > "From:" address has changed, the mailing list did not recognize your last > message as coming from a member. If you plan to keep using your Gmail > address, be sure to subscribe to the mailing list with that address. > >>> > > >>> > John David N. Dionisio, PhD > >>> > Associate Professor, Computer Science > >>> > Loyola Marymount University > >>> > > >>> > > >>> > > >>> > On Feb 7, 2011, at 4:18 PM, Richard Brous wrote: > >>> > > >>> > > I'll give it a try. I already committed the link change up to the > repository so there will be no committment issues (this time!). > >>> > > > >>> > > If I run into the same issue as before I'll let you know and you > will probably have to handle it. > >>> > > > >>> > > rb > >>> > > > >>> > > On Mon, Feb 7, 2011 at 3:39 PM, John David N. Dionisio < > do...@lm...> wrote: > >>> > > Hi Kam, > >>> > > > >>> > > Sure, no problem --- Rich, do you want to do the release yourself, > or would you rather that I do it? Let me know. Either way, make sure > you're all committed up before releasing. Especially if I'll be the one > doing it :) > >>> > > > >>> > > John David N. Dionisio, PhD > >>> > > Associate Professor, Computer Science > >>> > > Loyola Marymount University > >>> > > > >>> > > > >>> > > > >>> > > On Feb 7, 2011, at 2:43 PM, Kam Dahlquist wrote: > >>> > > > >>> > > > Hi Dondi, Rich, > >>> > > > > >>> > > > I'm looking at the M smegmatis gdb that Rich created and the new > link > >>> > > > out works and the gdb works with the major functions of > >>> > > > GenMAPP/MAPPFinder. Would it be possible to release a version of > >>> > > > GenMAPP Builder with this change so that I can try an export > myself? > >>> > > > > >>> > > > I'm going to try to finish going over the gdb itself so that we > can > >>> > > > also release it, but that will take me a little more time. > >>> > > > > >>> > > > Thanks, > >>> > > > Dr. D > >>> > > > > >> > ------------------------------------------------------------------------------ > >> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio > XE: > >> Pinpoint memory and threading errors before they happen. > >> Find and fix more than 250 security defects in the development cycle. > >> Locate bottlenecks in serial and parallel code that limit performance. > >> http://p.sf.net/sfu/intel-dev2devfeb > >> _______________________________________________ > >> xmlpipedb-developer mailing list > >> xml...@li... > >> https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > > <ATT00001..txt><ATT00002..txt> > > > > ------------------------------------------------------------------------------ > The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: > Pinpoint memory and threading errors before they happen. > Find and fix more than 250 security defects in the development cycle. > Locate bottlenecks in serial and parallel code that limit performance. > http://p.sf.net/sfu/intel-dev2devfeb > _______________________________________________ > xmlpipedb-developer mailing list > xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > > |
|
From: John D. N. D. <do...@lm...> - 2011-02-08 17:58:03
|
Greetings, The reason it didn't show up on the front page is because it wasn't marked as the default download (see attached screenshot). To see that panel, click on "i" then click on the appropriate checkbox. Rich, please verify if, when you mention that you didn't have rights, you meant that this panel does not appear (or is not editable) under your account. Or if you meant something else, let me know. Meanwhile, I went ahead and checked Windows as the default download. Should appear on the front page now. John David N. Dionisio, PhD Associate Professor, Computer Science Loyola Marymount University |
|
From: Richard B. <rbr...@gm...> - 2011-02-08 17:31:25
|
Yes, because I don't seem to have rights to officially release the new version. I mentioned that in my last email on yesterday's thread. 2b61 zip is complete and ready. Rb Sent from my iPhone On Feb 8, 2011, at 9:07 AM, Kam Dahlquist <kda...@lm...> wrote: > Hi, > > Do you guys know why the SourceForge home page for XMLPipeDB still has b60 as the latest file? To download b61, I have to show all files. > > Thanks, > Dr. D > > At 05:50 PM 2/7/2011, Richard Brous wrote: >> Meant rev the release number =D >> >> On the readme I followed the format of b60 which I may have uploaded incorrectly since I released it as well. Will correct right now. >> >> Richard >> >> On Mon, Feb 7, 2011 at 5:36 PM, John David N. Dionisio <do...@lm...> wrote: >> Hi Rich, >> >> Not sure what you mean by "rev on the release" --- you successfully uploaded, so that should be enough. >> >> Note though that the readme that you uploaded is called "gmbuilder-2_0b61-readme.txt" and not "gmbuilder-2.0b61-readme.txt" (period instead of underscore). >> >> AFAIK you can upload and reupload the file as much as needed, until the contents of that folder are correct. >> >> John David N. Dionisio, PhD >> Associate Professor, Computer Science >> Loyola Marymount University >> >> >> >> On Feb 7, 2011, at 5:29 PM, Richard Brous wrote: >> >> > OK, b61 has had its version updated (string constant), was zipped up and added to sourceforge along with a corresponding readme.txt. >> > >> > Should be go to go now for rev on the release. Do I have rights to do that if so where on Sourceforge should I go? (I checked project admin tab but no items display) >> > >> > Richard >> > >> > On Mon, Feb 7, 2011 at 4:41 PM, John David N. Dionisio <do...@lm...> wrote: >> > OK, got it; just holler if you run into issues. BTW Rich, since your "From:" address has changed, the mailing list did not recognize your last message as coming from a member. If you plan to keep using your Gmail address, be sure to subscribe to the mailing list with that address. >> > >> > John David N. Dionisio, PhD >> > Associate Professor, Computer Science >> > Loyola Marymount University >> > >> > >> > >> > On Feb 7, 2011, at 4:18 PM, Richard Brous wrote: >> > >> > > I'll give it a try. I already committed the link change up to the repository so there will be no committment issues (this time!). >> > > >> > > If I run into the same issue as before I'll let you know and you will probably have to handle it. >> > > >> > > rb >> > > >> > > On Mon, Feb 7, 2011 at 3:39 PM, John David N. Dionisio <do...@lm...> wrote: >> > > Hi Kam, >> > > >> > > Sure, no problem --- Rich, do you want to do the release yourself, or would you rather that I do it? Let me know. Either way, make sure you're all committed up before releasing. Especially if I'll be the one doing it :) >> > > >> > > John David N. Dionisio, PhD >> > > Associate Professor, Computer Science >> > > Loyola Marymount University >> > > >> > > >> > > >> > > On Feb 7, 2011, at 2:43 PM, Kam Dahlquist wrote: >> > > >> > > > Hi Dondi, Rich, >> > > > >> > > > I'm looking at the M smegmatis gdb that Rich created and the new link >> > > > out works and the gdb works with the major functions of >> > > > GenMAPP/MAPPFinder. Would it be possible to release a version of >> > > > GenMAPP Builder with this change so that I can try an export myself? >> > > > >> > > > I'm going to try to finish going over the gdb itself so that we can >> > > > also release it, but that will take me a little more time. >> > > > >> > > > Thanks, >> > > > Dr. D >> > > > > > ------------------------------------------------------------------------------ > The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: > Pinpoint memory and threading errors before they happen. > Find and fix more than 250 security defects in the development cycle. > Locate bottlenecks in serial and parallel code that limit performance. > http://p.sf.net/sfu/intel-dev2devfeb > _______________________________________________ > xmlpipedb-developer mailing list > xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer |
|
From: Kam D. <kda...@lm...> - 2011-02-08 17:07:58
|
Hi, Do you guys know why the SourceForge home page for XMLPipeDB still has b60 as the latest file? To download b61, I have to show all files. Thanks, Dr. D At 05:50 PM 2/7/2011, Richard Brous wrote: >Meant rev the release number =D > >On the readme I followed the format of b60 which I may have uploaded >incorrectly since I released it as well. Will correct right now. > >Richard > >On Mon, Feb 7, 2011 at 5:36 PM, John David N. Dionisio ><<mailto:do...@lm...>do...@lm...> wrote: >Hi Rich, > >Not sure what you mean by "rev on the release" --- you successfully >uploaded, so that should be enough. > >Note though that the readme that you uploaded is called >"gmbuilder-2_0b61-readme.txt" and not "gmbuilder-2.0b61-readme.txt" >(period instead of underscore). > >AFAIK you can upload and reupload the file as much as needed, until >the contents of that folder are correct. > >John David N. Dionisio, PhD >Associate Professor, Computer Science >Loyola Marymount University > > > >On Feb 7, 2011, at 5:29 PM, Richard Brous wrote: > > > OK, b61 has had its version updated (string constant), was zipped > up and added to sourceforge along with a corresponding readme.txt. > > > > Should be go to go now for rev on the release. Do I have rights > to do that if so where on Sourceforge should I go? (I checked > project admin tab but no items display) > > > > Richard > > > > On Mon, Feb 7, 2011 at 4:41 PM, John David N. Dionisio > <<mailto:do...@lm...>do...@lm...> wrote: > > OK, got it; just holler if you run into issues. BTW Rich, since > your "From:" address has changed, the mailing list did not > recognize your last message as coming from a member. If you plan > to keep using your Gmail address, be sure to subscribe to the > mailing list with that address. > > > > John David N. Dionisio, PhD > > Associate Professor, Computer Science > > Loyola Marymount University > > > > > > > > On Feb 7, 2011, at 4:18 PM, Richard Brous wrote: > > > > > I'll give it a try. I already committed the link change up to > the repository so there will be no committment issues (this time!). > > > > > > If I run into the same issue as before I'll let you know and > you will probably have to handle it. > > > > > > rb > > > > > > On Mon, Feb 7, 2011 at 3:39 PM, John David N. Dionisio > <<mailto:do...@lm...>do...@lm...> wrote: > > > Hi Kam, > > > > > > Sure, no problem --- Rich, do you want to do the release > yourself, or would you rather that I do it? Let me know. Either > way, make sure you're all committed up before > releasing. Especially if I'll be the one doing it :) > > > > > > John David N. Dionisio, PhD > > > Associate Professor, Computer Science > > > Loyola Marymount University > > > > > > > > > > > > On Feb 7, 2011, at 2:43 PM, Kam Dahlquist wrote: > > > > > > > Hi Dondi, Rich, > > > > > > > > I'm looking at the M smegmatis gdb that Rich created and the new link > > > > out works and the gdb works with the major functions of > > > > GenMAPP/MAPPFinder. Would it be possible to release a version of > > > > GenMAPP Builder with this change so that I can try an export myself? > > > > > > > > I'm going to try to finish going over the gdb itself so that we can > > > > also release it, but that will take me a little more time. > > > > > > > > Thanks, > > > > Dr. D > > > > |
|
From: Richard B. <rbr...@gm...> - 2011-02-08 01:50:42
|
Meant rev the release number =D On the readme I followed the format of b60 which I may have uploaded incorrectly since I released it as well. Will correct right now. Richard On Mon, Feb 7, 2011 at 5:36 PM, John David N. Dionisio <do...@lm...>wrote: > Hi Rich, > > Not sure what you mean by "rev on the release" --- you successfully > uploaded, so that should be enough. > > Note though that the readme that you uploaded is called > "gmbuilder-2_0b61-readme.txt" and not "gmbuilder-2.0b61-readme.txt" (period > instead of underscore). > > AFAIK you can upload and reupload the file as much as needed, until the > contents of that folder are correct. > > John David N. Dionisio, PhD > Associate Professor, Computer Science > Loyola Marymount University > > > > On Feb 7, 2011, at 5:29 PM, Richard Brous wrote: > > > OK, b61 has had its version updated (string constant), was zipped up and > added to sourceforge along with a corresponding readme.txt. > > > > Should be go to go now for rev on the release. Do I have rights to do > that if so where on Sourceforge should I go? (I checked project admin tab > but no items display) > > > > Richard > > > > On Mon, Feb 7, 2011 at 4:41 PM, John David N. Dionisio <do...@lm...> > wrote: > > OK, got it; just holler if you run into issues. BTW Rich, since your > "From:" address has changed, the mailing list did not recognize your last > message as coming from a member. If you plan to keep using your Gmail > address, be sure to subscribe to the mailing list with that address. > > > > John David N. Dionisio, PhD > > Associate Professor, Computer Science > > Loyola Marymount University > > > > > > > > On Feb 7, 2011, at 4:18 PM, Richard Brous wrote: > > > > > I'll give it a try. I already committed the link change up to the > repository so there will be no committment issues (this time!). > > > > > > If I run into the same issue as before I'll let you know and you will > probably have to handle it. > > > > > > rb > > > > > > On Mon, Feb 7, 2011 at 3:39 PM, John David N. Dionisio <do...@lm...> > wrote: > > > Hi Kam, > > > > > > Sure, no problem --- Rich, do you want to do the release yourself, or > would you rather that I do it? Let me know. Either way, make sure you're > all committed up before releasing. Especially if I'll be the one doing it > :) > > > > > > John David N. Dionisio, PhD > > > Associate Professor, Computer Science > > > Loyola Marymount University > > > > > > > > > > > > On Feb 7, 2011, at 2:43 PM, Kam Dahlquist wrote: > > > > > > > Hi Dondi, Rich, > > > > > > > > I'm looking at the M smegmatis gdb that Rich created and the new link > > > > out works and the gdb works with the major functions of > > > > GenMAPP/MAPPFinder. Would it be possible to release a version of > > > > GenMAPP Builder with this change so that I can try an export myself? > > > > > > > > I'm going to try to finish going over the gdb itself so that we can > > > > also release it, but that will take me a little more time. > > > > > > > > Thanks, > > > > Dr. D > > > > > > > > > > > > > ------------------------------------------------------------------------------ > > > > The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio > XE: > > > > Pinpoint memory and threading errors before they happen. > > > > Find and fix more than 250 security defects in the development cycle. > > > > Locate bottlenecks in serial and parallel code that limit > performance. > > > > http://p.sf.net/sfu/intel-dev2devfeb > > > > _______________________________________________ > > > > xmlpipedb-developer mailing list > > > > xml...@li... > > > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > > > > > > > > > > ------------------------------------------------------------------------------ > > > The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio > XE: > > > Pinpoint memory and threading errors before they happen. > > > Find and fix more than 250 security defects in the development cycle. > > > Locate bottlenecks in serial and parallel code that limit performance. > > > http://p.sf.net/sfu/intel-dev2devfeb > > > _______________________________________________ > > > xmlpipedb-developer mailing list > > > xml...@li... > > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > > > > > > <ATT00001..txt><ATT00002..txt> > > > > > > > ------------------------------------------------------------------------------ > > The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: > > Pinpoint memory and threading errors before they happen. > > Find and fix more than 250 security defects in the development cycle. > > Locate bottlenecks in serial and parallel code that limit performance. > > http://p.sf.net/sfu/intel-dev2devfeb > > _______________________________________________ > > xmlpipedb-developer mailing list > > xml...@li... > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > > > > <ATT00001..txt><ATT00002..txt> > > > > ------------------------------------------------------------------------------ > The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: > Pinpoint memory and threading errors before they happen. > Find and fix more than 250 security defects in the development cycle. > Locate bottlenecks in serial and parallel code that limit performance. > http://p.sf.net/sfu/intel-dev2devfeb > _______________________________________________ > xmlpipedb-developer mailing list > xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > |
|
From: John D. N. D. <do...@lm...> - 2011-02-08 01:50:18
|
Right, schema issues are unlikely. Most count discrepancies like this that I've seen have boiled down to forming the right query. Then, knowing the right query (in both XML and SQL), it's a matter of making sure that TallyEngine asks that same query. John David N. Dionisio, PhD Associate Professor, Computer Science Loyola Marymount University On Feb 7, 2011, at 5:48 PM, Richard Brous wrote: > OK, so based on your approach: > > 1. I'll start with reviewing the queries for xmlpipedb-match and sql queries needed for the respective results as you requested. > > I was also thinking I may need to review the schema from xml into postgres but the issue isn't likely a schema error. The error most likely lies in how xmlpipedbutils queries the data from xml source and writes to the tables what it returns? > > 2. I'll review the code: trace the entrance of tally engine in the gmbuilder code then follow it through the xmlpipedbutils. > > Richard > > On Sat, Feb 5, 2011 at 10:28 AM, John David N. Dionisio <do...@lm...> wrote: > Just wanted to confirm (since I wasn't sure in the first e-mail) --- the XMLPipeDB Utilities source code is in trunk/xmlpipedbutils in SourceForge's Subversion repo. > > John David N. Dionisio, PhD > Associate Professor, Computer Science > Loyola Marymount University > > > > On Feb 5, 2011, at 10:02 AM, Richard Brous wrote: > > > Hi Dondi, > > > > So I'm at the point in working with M tuberculosis that I was able to exactly reproduce Dr. Dahlquist's problematic TallyEngine results. > > > > gmb2b60 Results > > > > > > > > Now the proverbial question - What next to solve the Ordered Locus import/count issue? > > > > ********************************************** > > Here is my thought process: > > > > Step 1: How does the import process work at the high level? (obviously correct me if I'm wrong) > > > > I believe that basically as each XML tag is read, it is placed in the proper Postgres table(s) based on some criteria. There is also likely some sort of check that each individual tag is in valid XML format unless we don't care at this stage (care at export) or maybe the parser just skips over and goes on to the next . > > > > Step 2: What could be the problem? > > > > Either - > > a. XML tags are being parsed incorrectly (ignored/skipped)? > > b. Decision criteria of which table they should be added to? > > > > ********************************************** > > > > I read on the sourceforge wiki: > > > > XMLPipeDB has a modular architecture with three components that may be used separately or together. XSD-to-DB reads an XSD (XML Schema Definition) and automatically generates an SQL schema, Java classes, and Hibernate mappings. XMLPipeDB Utilities provides functionality for configuring the database, importing data, and performing queries. GenMAPP Builder is based on the XMLPipeDB Utilities and exports GenMAPP-compatible Gene Databases based on data from UniProt and Gene Ontology (GO). > > > > So I should probably start with the XMLPipeDB Utilities which are where? I don't see any in the basic distribution or are they not standalone and called from the command line? > > > > Thanks! > > > > Richard > > > <ATT00001..txt><ATT00002..txt> |
|
From: Richard B. <rbr...@gm...> - 2011-02-08 01:48:19
|
OK, so based on your approach: 1. I'll start with reviewing the queries for xmlpipedb-match and sql queries needed for the respective results as you requested. I was also thinking I may need to review the schema from xml into postgres but the issue isn't likely a schema error. The error most likely lies in how xmlpipedbutils queries the data from xml source and writes to the tables what it returns? 2. I'll review the code: trace the entrance of tally engine in the gmbuilder code then follow it through the xmlpipedbutils. Richard On Sat, Feb 5, 2011 at 10:28 AM, John David N. Dionisio <do...@lm...>wrote: > Just wanted to confirm (since I wasn't sure in the first e-mail) --- the > XMLPipeDB Utilities source code is in trunk/xmlpipedbutils in SourceForge's > Subversion repo. > > John David N. Dionisio, PhD > Associate Professor, Computer Science > Loyola Marymount University > > > > On Feb 5, 2011, at 10:02 AM, Richard Brous wrote: > > > Hi Dondi, > > > > So I'm at the point in working with M tuberculosis that I was able to > exactly reproduce Dr. Dahlquist's problematic TallyEngine results. > > > > gmb2b60 Results > > > > > > > > Now the proverbial question - What next to solve the Ordered Locus > import/count issue? > > > > ********************************************** > > Here is my thought process: > > > > Step 1: How does the import process work at the high level? (obviously > correct me if I'm wrong) > > > > I believe that basically as each XML tag is read, it is placed in the > proper Postgres table(s) based on some criteria. There is also likely some > sort of check that each individual tag is in valid XML format unless we > don't care at this stage (care at export) or maybe the parser just skips > over and goes on to the next . > > > > Step 2: What could be the problem? > > > > Either - > > a. XML tags are being parsed incorrectly (ignored/skipped)? > > b. Decision criteria of which table they should be added to? > > > > ********************************************** > > > > I read on the sourceforge wiki: > > > > XMLPipeDB has a modular architecture with three components that may be > used separately or together. XSD-to-DB reads an XSD (XML Schema Definition) > and automatically generates an SQL schema, Java classes, and Hibernate > mappings. XMLPipeDB Utilities provides functionality for configuring the > database, importing data, and performing queries. GenMAPP Builder is based > on the XMLPipeDB Utilities and exports GenMAPP-compatible Gene Databases > based on data from UniProt and Gene Ontology (GO). > > > > So I should probably start with the XMLPipeDB Utilities which are where? > I don't see any in the basic distribution or are they not standalone and > called from the command line? > > > > Thanks! > > > > Richard > > |
|
From: John D. N. D. <do...@lm...> - 2011-02-08 01:36:27
|
Hi Rich, Not sure what you mean by "rev on the release" --- you successfully uploaded, so that should be enough. Note though that the readme that you uploaded is called "gmbuilder-2_0b61-readme.txt" and not "gmbuilder-2.0b61-readme.txt" (period instead of underscore). AFAIK you can upload and reupload the file as much as needed, until the contents of that folder are correct. John David N. Dionisio, PhD Associate Professor, Computer Science Loyola Marymount University On Feb 7, 2011, at 5:29 PM, Richard Brous wrote: > OK, b61 has had its version updated (string constant), was zipped up and added to sourceforge along with a corresponding readme.txt. > > Should be go to go now for rev on the release. Do I have rights to do that if so where on Sourceforge should I go? (I checked project admin tab but no items display) > > Richard > > On Mon, Feb 7, 2011 at 4:41 PM, John David N. Dionisio <do...@lm...> wrote: > OK, got it; just holler if you run into issues. BTW Rich, since your "From:" address has changed, the mailing list did not recognize your last message as coming from a member. If you plan to keep using your Gmail address, be sure to subscribe to the mailing list with that address. > > John David N. Dionisio, PhD > Associate Professor, Computer Science > Loyola Marymount University > > > > On Feb 7, 2011, at 4:18 PM, Richard Brous wrote: > > > I'll give it a try. I already committed the link change up to the repository so there will be no committment issues (this time!). > > > > If I run into the same issue as before I'll let you know and you will probably have to handle it. > > > > rb > > > > On Mon, Feb 7, 2011 at 3:39 PM, John David N. Dionisio <do...@lm...> wrote: > > Hi Kam, > > > > Sure, no problem --- Rich, do you want to do the release yourself, or would you rather that I do it? Let me know. Either way, make sure you're all committed up before releasing. Especially if I'll be the one doing it :) > > > > John David N. Dionisio, PhD > > Associate Professor, Computer Science > > Loyola Marymount University > > > > > > > > On Feb 7, 2011, at 2:43 PM, Kam Dahlquist wrote: > > > > > Hi Dondi, Rich, > > > > > > I'm looking at the M smegmatis gdb that Rich created and the new link > > > out works and the gdb works with the major functions of > > > GenMAPP/MAPPFinder. Would it be possible to release a version of > > > GenMAPP Builder with this change so that I can try an export myself? > > > > > > I'm going to try to finish going over the gdb itself so that we can > > > also release it, but that will take me a little more time. > > > > > > Thanks, > > > Dr. D > > > > > > > > > ------------------------------------------------------------------------------ > > > The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: > > > Pinpoint memory and threading errors before they happen. > > > Find and fix more than 250 security defects in the development cycle. > > > Locate bottlenecks in serial and parallel code that limit performance. > > > http://p.sf.net/sfu/intel-dev2devfeb > > > _______________________________________________ > > > xmlpipedb-developer mailing list > > > xml...@li... > > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > > > > > > ------------------------------------------------------------------------------ > > The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: > > Pinpoint memory and threading errors before they happen. > > Find and fix more than 250 security defects in the development cycle. > > Locate bottlenecks in serial and parallel code that limit performance. > > http://p.sf.net/sfu/intel-dev2devfeb > > _______________________________________________ > > xmlpipedb-developer mailing list > > xml...@li... > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > > > > <ATT00001..txt><ATT00002..txt> > > > ------------------------------------------------------------------------------ > The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: > Pinpoint memory and threading errors before they happen. > Find and fix more than 250 security defects in the development cycle. > Locate bottlenecks in serial and parallel code that limit performance. > http://p.sf.net/sfu/intel-dev2devfeb > _______________________________________________ > xmlpipedb-developer mailing list > xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > > <ATT00001..txt><ATT00002..txt> |
|
From: Richard B. <rbr...@gm...> - 2011-02-08 01:29:35
|
OK, b61 has had its version updated (string constant), was zipped up and added to sourceforge along with a corresponding readme.txt. Should be go to go now for rev on the release. Do I have rights to do that if so where on Sourceforge should I go? (I checked project admin tab but no items display) Richard On Mon, Feb 7, 2011 at 4:41 PM, John David N. Dionisio <do...@lm...>wrote: > OK, got it; just holler if you run into issues. BTW Rich, since your > "From:" address has changed, the mailing list did not recognize your last > message as coming from a member. If you plan to keep using your Gmail > address, be sure to subscribe to the mailing list with that address. > > John David N. Dionisio, PhD > Associate Professor, Computer Science > Loyola Marymount University > > > > On Feb 7, 2011, at 4:18 PM, Richard Brous wrote: > > > I'll give it a try. I already committed the link change up to the > repository so there will be no committment issues (this time!). > > > > If I run into the same issue as before I'll let you know and you will > probably have to handle it. > > > > rb > > > > On Mon, Feb 7, 2011 at 3:39 PM, John David N. Dionisio <do...@lm...> > wrote: > > Hi Kam, > > > > Sure, no problem --- Rich, do you want to do the release yourself, or > would you rather that I do it? Let me know. Either way, make sure you're > all committed up before releasing. Especially if I'll be the one doing it > :) > > > > John David N. Dionisio, PhD > > Associate Professor, Computer Science > > Loyola Marymount University > > > > > > > > On Feb 7, 2011, at 2:43 PM, Kam Dahlquist wrote: > > > > > Hi Dondi, Rich, > > > > > > I'm looking at the M smegmatis gdb that Rich created and the new link > > > out works and the gdb works with the major functions of > > > GenMAPP/MAPPFinder. Would it be possible to release a version of > > > GenMAPP Builder with this change so that I can try an export myself? > > > > > > I'm going to try to finish going over the gdb itself so that we can > > > also release it, but that will take me a little more time. > > > > > > Thanks, > > > Dr. D > > > > > > > > > > ------------------------------------------------------------------------------ > > > The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio > XE: > > > Pinpoint memory and threading errors before they happen. > > > Find and fix more than 250 security defects in the development cycle. > > > Locate bottlenecks in serial and parallel code that limit performance. > > > http://p.sf.net/sfu/intel-dev2devfeb > > > _______________________________________________ > > > xmlpipedb-developer mailing list > > > xml...@li... > > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > > > > > > > ------------------------------------------------------------------------------ > > The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: > > Pinpoint memory and threading errors before they happen. > > Find and fix more than 250 security defects in the development cycle. > > Locate bottlenecks in serial and parallel code that limit performance. > > http://p.sf.net/sfu/intel-dev2devfeb > > _______________________________________________ > > xmlpipedb-developer mailing list > > xml...@li... > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > > > > <ATT00001..txt><ATT00002..txt> > > > > ------------------------------------------------------------------------------ > The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: > Pinpoint memory and threading errors before they happen. > Find and fix more than 250 security defects in the development cycle. > Locate bottlenecks in serial and parallel code that limit performance. > http://p.sf.net/sfu/intel-dev2devfeb > _______________________________________________ > xmlpipedb-developer mailing list > xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > |
|
From: John D. N. D. <do...@lm...> - 2011-02-08 00:41:48
|
OK, got it; just holler if you run into issues. BTW Rich, since your "From:" address has changed, the mailing list did not recognize your last message as coming from a member. If you plan to keep using your Gmail address, be sure to subscribe to the mailing list with that address. John David N. Dionisio, PhD Associate Professor, Computer Science Loyola Marymount University On Feb 7, 2011, at 4:18 PM, Richard Brous wrote: > I'll give it a try. I already committed the link change up to the repository so there will be no committment issues (this time!). > > If I run into the same issue as before I'll let you know and you will probably have to handle it. > > rb > > On Mon, Feb 7, 2011 at 3:39 PM, John David N. Dionisio <do...@lm...> wrote: > Hi Kam, > > Sure, no problem --- Rich, do you want to do the release yourself, or would you rather that I do it? Let me know. Either way, make sure you're all committed up before releasing. Especially if I'll be the one doing it :) > > John David N. Dionisio, PhD > Associate Professor, Computer Science > Loyola Marymount University > > > > On Feb 7, 2011, at 2:43 PM, Kam Dahlquist wrote: > > > Hi Dondi, Rich, > > > > I'm looking at the M smegmatis gdb that Rich created and the new link > > out works and the gdb works with the major functions of > > GenMAPP/MAPPFinder. Would it be possible to release a version of > > GenMAPP Builder with this change so that I can try an export myself? > > > > I'm going to try to finish going over the gdb itself so that we can > > also release it, but that will take me a little more time. > > > > Thanks, > > Dr. D > > > > > > ------------------------------------------------------------------------------ > > The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: > > Pinpoint memory and threading errors before they happen. > > Find and fix more than 250 security defects in the development cycle. > > Locate bottlenecks in serial and parallel code that limit performance. > > http://p.sf.net/sfu/intel-dev2devfeb > > _______________________________________________ > > xmlpipedb-developer mailing list > > xml...@li... > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > > > ------------------------------------------------------------------------------ > The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: > Pinpoint memory and threading errors before they happen. > Find and fix more than 250 security defects in the development cycle. > Locate bottlenecks in serial and parallel code that limit performance. > http://p.sf.net/sfu/intel-dev2devfeb > _______________________________________________ > xmlpipedb-developer mailing list > xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > > <ATT00001..txt><ATT00002..txt> |
|
From: Richard B. <rbr...@gm...> - 2011-02-08 00:18:40
|
I'll give it a try. I already committed the link change up to the repository so there will be no committment issues (this time!). If I run into the same issue as before I'll let you know and you will probably have to handle it. rb On Mon, Feb 7, 2011 at 3:39 PM, John David N. Dionisio <do...@lm...>wrote: > Hi Kam, > > Sure, no problem --- Rich, do you want to do the release yourself, or would > you rather that I do it? Let me know. Either way, make sure you're all > committed up before releasing. Especially if I'll be the one doing it :) > > John David N. Dionisio, PhD > Associate Professor, Computer Science > Loyola Marymount University > > > > On Feb 7, 2011, at 2:43 PM, Kam Dahlquist wrote: > > > Hi Dondi, Rich, > > > > I'm looking at the M smegmatis gdb that Rich created and the new link > > out works and the gdb works with the major functions of > > GenMAPP/MAPPFinder. Would it be possible to release a version of > > GenMAPP Builder with this change so that I can try an export myself? > > > > I'm going to try to finish going over the gdb itself so that we can > > also release it, but that will take me a little more time. > > > > Thanks, > > Dr. D > > > > > > > ------------------------------------------------------------------------------ > > The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: > > Pinpoint memory and threading errors before they happen. > > Find and fix more than 250 security defects in the development cycle. > > Locate bottlenecks in serial and parallel code that limit performance. > > http://p.sf.net/sfu/intel-dev2devfeb > > _______________________________________________ > > xmlpipedb-developer mailing list > > xml...@li... > > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > > > > ------------------------------------------------------------------------------ > The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: > Pinpoint memory and threading errors before they happen. > Find and fix more than 250 security defects in the development cycle. > Locate bottlenecks in serial and parallel code that limit performance. > http://p.sf.net/sfu/intel-dev2devfeb > _______________________________________________ > xmlpipedb-developer mailing list > xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer > |
|
From: John D. N. D. <do...@lm...> - 2011-02-07 23:39:16
|
Hi Kam, Sure, no problem --- Rich, do you want to do the release yourself, or would you rather that I do it? Let me know. Either way, make sure you're all committed up before releasing. Especially if I'll be the one doing it :) John David N. Dionisio, PhD Associate Professor, Computer Science Loyola Marymount University On Feb 7, 2011, at 2:43 PM, Kam Dahlquist wrote: > Hi Dondi, Rich, > > I'm looking at the M smegmatis gdb that Rich created and the new link > out works and the gdb works with the major functions of > GenMAPP/MAPPFinder. Would it be possible to release a version of > GenMAPP Builder with this change so that I can try an export myself? > > I'm going to try to finish going over the gdb itself so that we can > also release it, but that will take me a little more time. > > Thanks, > Dr. D > > > ------------------------------------------------------------------------------ > The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: > Pinpoint memory and threading errors before they happen. > Find and fix more than 250 security defects in the development cycle. > Locate bottlenecks in serial and parallel code that limit performance. > http://p.sf.net/sfu/intel-dev2devfeb > _______________________________________________ > xmlpipedb-developer mailing list > xml...@li... > https://lists.sourceforge.net/lists/listinfo/xmlpipedb-developer |
|
From: Kam D. <kda...@lm...> - 2011-02-07 22:55:34
|
Hi Dondi, Rich, I'm looking at the M smegmatis gdb that Rich created and the new link out works and the gdb works with the major functions of GenMAPP/MAPPFinder. Would it be possible to release a version of GenMAPP Builder with this change so that I can try an export myself? I'm going to try to finish going over the gdb itself so that we can also release it, but that will take me a little more time. Thanks, Dr. D |
|
From: Kam D. <kda...@lm...> - 2010-10-20 23:24:17
|
Hi, Kevin discovered an issue with importing the GO OBO-XML today. It appears that the XML format might have changed. A GO XML file that I used last week imported with no problem, but one downloaded today (attached) gives the error shown in the screenshot (also attached). Thanks, Kam |
|
From: Don M. <djs...@gm...> - 2010-10-04 23:37:17
|
GenMAPP Builder 2.0 Beta 59 has been released. This release of GenMAPP Builder adds a descriptive error message when attempting to export to a GenMAPP database when running GenMAPP Builder in 64-bit Java. |
|
From: Don M. <djs...@gm...> - 2010-09-29 17:48:03
|
GenMAPP Builder 2.0 Beta 58 has been released. This release of GenMAPP Builder fixes an issue encountered with 64-bit Java runtime environments, previously identified as a Windows 7 issue. To avoid the error, when exporting to a GenMAPP database, have a 32-bit Java runtime environment installed and start GenMAPP Builder using gmbuilder-32bit.bat. |
|
From: Kam D. <kda...@lm...> - 2010-09-23 18:01:11
|
Hi, see error attached. Dr. D At 09:45 AM 9/23/2010, you wrote: >GenMAPP Builder 2.0 Beta 57 has been released. This release of >GenMAPP Builder fixes a regular expression syntax error encountered >with the previous Windows 7 fix attempt. > > |
16 messages has been excluded from this view by a project administrator.
