Re: [PyWrapper-devel] Managing the concept server

[<wi...@go...>]

just quick:

- just ot avoid confusion. the root"table" is a root Alias!

- not available I am working on. it means the following: for a given model
pywrapper analyses the mappings and comes up with a list of aliases used fo=
r
the IndexingRecord. If this list includes the preferred alias, it uses this
to build the tablealias tree from, otherwise just picks one randomly. So no=
t
available means your alias is not usable for a record definition. otherwise
a single datasource would only be allowed to represent one kind of object. =
a
specimen with identifications could also serve taxon data in theory for
example - with a different root alias. we maybe should have avoided this in
TAPIR, but its like this now.

- pywrapper tries to reduce the list of available aliases. This gives bette=
r
performance. For example if a metadata table and a specimen table both are
found to be used as root aliases, but the specimen table contains a FK to
metadata, pywrapper knows it can ignore the metadata table, cause its ID
will depend on the specimen ID. I gotta check this routine again to be sure
it makes sense.


And: in the configtools mapping pane (overview) there is now a section that
lists all shared models if there are any configured. Together with the
number of dontrepeats, the preferred and required root alias.

Markus



On 11/14/06, Skofic A. Milko (IPGRI) <M.S...@cg...> wrote:
>
> OK I added the preferredRootTable to the preferences and it sees it
> alright, except that it says it cannot use it:
> <diagnostic time=3D"2006-11-14T09:44:05.92" level=3D"warn">
> Preferred root alias 'ACC' not available. Use alias 'ACC_STORAGE' out
> of [u'ACC_STORAGE']
> </diagnostic>
>
> What does "not available" mean? If you want to try for yourself the
> address is:
> http://tapir.grinfo.net:8080/pywrapper
> the passwords are the usual ACDC, the data source is EURISCO and the
> query I use is:
> <?xml version=3D'1.0' encoding=3D'UTF-8'?>
> <request>
>         <header>
>                 <source accesspoint=3D"
> http://tapir.grinfo.net:8080/pywrapper/
> pywrapper?dsa=3DEURISCO" sendtime=3D"2006-11-13T12:54:29.43"/>
>         </header>
>         <search count=3D'true' start=3D'0' limit=3D'50'>
>                 <externalOutputModel location=3D"
> http://eurisco.ecpgr.org/schemas/
> MCPD_EURISCO_1.00.xml"/>
>                 <filter>
>                         <like>
>                                 <concept id=3D'GENUS@mcpd_eurisco_100' />
>                                 <literal value=3D'B*' />
>                         </like>
>                 </filter>
>                 <orderBy>
>                         <concept id=3D'INSTCODE@mcpd_eurisco_100' />
>                         <concept id=3D'GENUS@mcpd_eurisco_100' />
>                         <concept id=3D'ACCENUMB@mcpd_eurisco_100' />
>                 </orderBy>
>         </search>
> </request>
>
> Hope it helps!
>
> Milko
>
> P.S. Shouldn't the preferredRootTable be set in the structure pane,
> like it was with BioCASE?
>
>
> On Nov 13, 2006, at 23:57 , Markus D=F6ring wrote:
>
> > alright.
> > looks as if it works. can you try milko?
> > m
> >
> >
> > On 13.11.2006, at 23:43, Markus D=F6ring wrote:
> >
> > ahh man.
> > its close to work but not yet. but I removed the serious error with
> > models. you can update now, but the preferred root alias selection
> > doesnt yet work fully.
> > the idea is that you can assign a preferred root alias in the
> > dsa_prefs for a shared model. but it will only be used if the alias
> > is one of the potential ones. Otherwise you can also assign a
> > "requiredRootAlias" which is always added to the root alias list no
> > matter what. but that still doesnt work...
> >
> > heres what it look likes in the prefs:
> >
> > <model location=3D"http://rs.tdwg.org/tapir/cs/abcd2.06/model/
> > abcd206.xml" preferredRootAlias=3D"herbariumsheets">
> >
> > good night gringos,
> > markus
> >
> >
> >
> > On 13.11.2006, at 21:41, Javier de la Torre wrote:
> >
> > Thats the biocase way by the way.
> >
> > We will not be able to solve this problem at the TAPIR level
> > without a more complete mapping... the idea of being able to map
> > tables, or kind of objects, to the XML tree will solve that... but
> > thats TAPIR 1.5
> >
> > Cheers.
> >
> > On 13/11/2006, at 18:54, Markus D=F6ring wrote:
> >
> >> for fixed models we could surely cheat that way. assign a root
> >> table that HAS to be used first.
> >> should be the easiest (still requires some hours Id suspect).
> >> markus
> >>
> >>
> >> On 13.11.2006, at 18:17, Skofic A. Milko (IPGRI) wrote:
> >>
> >> I finally fixed the alias file: the strange thing is that the
> >> TAPIR schema parser gave something like:
> >>
> >> NICODE                                                      http://
> >> www.ipgri.org/schemas/mcpd_eurisco/1.00::NICODE
> >> INSTCODE                                                    http://
> >> www.ipgri.org/schemas/mcpd_eurisco/1.00::INSTCODE
> >> ACCENUMB                                                    http://
> >> www.ipgri.org/schemas/mcpd_eurisco/1.00::ACCENUMB
> >>
> >> but it's OK, once one gets the feel for the alias file you can do
> >> it by hand... except if you have a schema like ABCD... -:)
> >>
> >>> comment #1: try not to use :: in the full concepts but make them
> >>> look like a simple URL. for example http://www.ipgri.org/schemas/
> >>> mcpd_eurisco/1.00/NICODE
> >> I now have this form:
> >>  http://www.ipgri.org/schemas/mcpd_eurisco/1.00/Accessions/
> >> Accession/NICODE
> >>
> >>> commen #3: the idea is that the server CANNOT start up with
> >>> conflicting or broken alias files. this shoulkd be fixed before.
> >>> dont you agree with that? otherwise you could get unforseen weird
> >>> results.
> >> I agree, alias files are usually not going to be managed by users,
> >> so this error is not too problematic.
> >>
> >> Now, regarding the structure, I did a bunch of tests, I managed to
> >> create an output model from ABCD to EURISCO and it works. The only
> >> problem is trying to use directly the EURISCO model. EURISCO has
> >> ACC as the main table, but ACC is linked to other tables with
> >> repeating values, although no required field is in these tables.
> >>
> >> When I try to query and look at the SQL it produces, it uses the
> >> repeating tables as the origin of the joins, which is correct if
> >> you translate the database graph into a tree, but there must be
> >> some way to indicate that the root of the joins is ACC, not the
> >> other tables.
> >>
> >> After some testing I realised that when you clear the cache files
> >> from the ConfigTool, it doesn't cleat the http cache files, so I
> >> could make any change in the schema and it would not be reflected
> >> in TAPIR.
> >>
> >> So, let me know if you have some ideas about the structure problem
> >> we have, there should be some way to indicate or force the main
> >> table (Table A, in this case) to be the root of the tree.
> >>
> >> Thanks!
> >>
> >> Milko
> >>
> >>
> >> On Nov 13, 2006, at 16:25 , Markus D=F6ring wrote:
> >>
> >>> Milko,
> >>>
> >>> comment #1: try not to use :: in the full concepts but make them
> >>> look like a simple URL. for example http://www.ipgri.org/schemas/
> >>> mcpd_eurisco/1.00/NICODE
> >>>
> >>> comment #2: the error says: "biocase.CNS.NonUniqueConceptError".
> >>> so it looks as if the aliases or qualified names are not unique.
> >>> check. another problem could be that the alias.txt should have
> >>> "=3D" between an alias and the qualified concept. But I think its
> >>> not required.
> >>>
> >>> commen #3: the idea is that the server CANNOT start up with
> >>> conflicting or broken alias files. this shoulkd be fixed before.
> >>> dont you agree with that? otherwise you could get unforseen weird
> >>> results.
> >>>
> >>> markus
> >>>
> >>>
> >>> On 11/13/06, Skofic A. Milko (IPGRI) <M.S...@cg...> wrote:
> >>> Hi guys, I am trying to create an alias file to use other schemas
> >>> and test mapping and output models. I have created a schema, run
> >>> it through TAPIR's schema parser and tried to set up another
> >>> concept server.
> >>>
> >>> I get an error whenever I startup TAPIR, and I am sure it has to
> >>> do with the alias file and its other related files.
> >>> The error is:
> >>>
> >>> Traceback (most recent call last):
> >>>   File "../webapp/start_server.py", line 56, in ?
> >>>     import root, page
> >>>   File "/Library/WebServer/Services/pywrapper/webapp/root.py",
> >>> line 17, in ?
> >>>     import page, biocase.configuration
> >>>   File "/Library/WebServer/Services/pywrapper/webapp/page.py",
> >>> line 24, in ?
> >>>     class Page(object):
> >>>   File "/Library/WebServer/Services/pywrapper/webapp/page.py",
> >>> line 26, in Page
> >>>     _dsaObjDict =3D biocase.pywrapper.datasource_factory.getDsaDict()
> >>>   File "/Library/WebServer/Services/pywrapper/lib/biocase/
> >>> pywrapper/datasource_factory.py", line 49, in getDsaDict
> >>>     dsaObj.readPrefsFile ()
> >>>   File "/Library/WebServer/Services/pywrapper/lib/biocase/
> >>> pywrapper/datasource.py", line 218, in readPrefsFile
> >>>     parse(filename, handler)
> >>>   File "/Library/WebServer/Services/pywrapper/tools/python/lib/
> >>> python2.4/site-packages/_xmlplus/sax/__init__.py", line 31, in parse
> >>>     parser.parse(filename_or_stream)
> >>>   File "/Library/WebServer/Services/pywrapper/tools/python/lib/
> >>> python2.4/site-packages/_xmlplus/sax/expatreader.py", line 109,
> >>> in parse
> >>>     xmlreader.IncrementalParser.parse(self, source)
> >>>   File "/Library/WebServer/Services/pywrapper/tools/python/lib/
> >>> python2.4/site-packages/_xmlplus/sax/xmlreader.py", line 123, in
> >>> parse
> >>>     self.feed(buffer)
> >>>   File "/Library/WebServer/Services/pywrapper/tools/python/lib/
> >>> python2.4/site-packages/_xmlplus/sax/expatreader.py", line 216,
> >>> in feed
> >>>     self._parser.Parse(data, isFinal)
> >>>   File "/Library/WebServer/Services/pywrapper/tools/python/lib/
> >>> python2.4/site-packages/_xmlplus/sax/expatreader.py", line 312,
> >>> in start_element
> >>>     self._cont_handler.startElement(name, AttributesImpl(attrs))
> >>>   File "/Library/WebServer/Services/pywrapper/lib/biocase/
> >>> pywrapper/datasource_handler.py", line 73, in startElement
> >>>     self._lastConceptObj =3D buildConceptFromAttrs(attrs,
> >>> ns=3Dself._lastSchemaNS)
> >>>   File "/Library/WebServer/Services/pywrapper/lib/biocase/
> >>> pywrapper/filter/concept.py", line 215, in buildConceptFromAttrs
> >>>     conceptObj =3D buildConcept(qname, ns)
> >>>   File "/Library/WebServer/Services/pywrapper/lib/biocase/
> >>> pywrapper/filter/concept.py", line 239, in buildConcept
> >>>     CNServer =3D createCNServer
> >>> (aliasList=3DcfgObj.ConceptNameServer.AliasList,
> >>> cachedir=3DcfgObj.cacheLocator)
> >>>   File "/Library/WebServer/Services/pywrapper/lib/biocase/
> >>> CNS.py", line 433, in createCNServer
> >>>     CNS.addAliasLocation(alias)
> >>>   File "/Library/WebServer/Services/pywrapper/lib/biocase/
> >>> CNS.py", line 168, in addAliasLocation
> >>>     self._read(fh)
> >>>   File "/Library/WebServer/Services/pywrapper/lib/biocase/
> >>> CNS.py", line 241, in _read
> >>>     self.addConcept( c, additionalAliases=3D[] )
> >>>   File "/Library/WebServer/Services/pywrapper/lib/biocase/
> >>> CNS.py", line 346, in addConcept
> >>>     raise NonUniqueConcept(conceptObj)
> >>> biocase.CNS.NonUniqueConceptError in atexit._run_exitfuncs:
> >>> Traceback (most recent call last):
> >>>   File "/Library/WebServer/Services/pywrapper/tools/python/lib/
> >>> python2.4/atexit.py", line 24, in _run_exitfuncs
> >>>     func(*targs, **kargs)
> >>>   File "/Library/WebServer/Services/pywrapper/tools/python/lib/
> >>> python2.4/logging/__init__.py", line 1333, in shutdown
> >>>     h.close()
> >>>   File "/Library/WebServer/Services/pywrapper/tools/python/lib/
> >>> python2.4/logging/__init__.py", line 674, in close
> >>>     del _handlers[self]
> >>> KeyError: <logging.StreamHandler instance at 0x1385418>
> >>> Error in sys.exitfunc:
> >>> Traceback (most recent call last):
> >>>   File "/Library/WebServer/Services/pywrapper/tools/python/lib/
> >>> python2.4/atexit.py", line 24, in _run_exitfuncs
> >>>     func(*targs, **kargs)
> >>>   File "/Library/WebServer/Services/pywrapper/tools/python/lib/
> >>> python2.4/logging/__init__.py", line 1333, in shutdown
> >>>     h.close()
> >>>   File "/Library/WebServer/Services/pywrapper/tools/python/lib/
> >>> python2.4/logging/__init__.py", line 674, in close
> >>>     del _handlers[self]
> >>> KeyError: <logging.StreamHandler instance at 0x1385418>
> >>>
> >>> This listing comes from the terminal. I am sure I did something
> >>> wrong with the files, however it should not crash like this.
> >>> Anyways here are the files:
> >>>
> >>> The alias file is at http://eurisco.ecpgr.org/schemas/alias.txt
> >>> The schema file is at: http://eurisco.ecpgr.org/schemas/
> >>> MCPD_EURISCO_1.00.xsd
> >>> And the model file is at: http://eurisco.ecpgr.org/schemas/
> >>> MCPD_EURISCO_1.00.xml
> >>>
> >>> I have now removed the reference to that alias file so the system
> >>> can work.
> >>> Looking at the alias file I see a strange syntax: is that
> >>> correct? I only changed the label, alias and added the models
> >>> section.
> >>>
> >>> Thanks!
> >>>
> >>> Milko
> >>>
> >>
> >>
> >
> >
> >
>
>