pybliographer-devel — Development of pybliographer

[pyblio-devel] [ pybliographer-Bugs-1936923 ] focus problems (fwd)

[Zoltan K. <z....@gm...>]

---------- Forwarded message ----------
Date: Mon, 07 Apr 2008 10:20:33 -0700
From: SourceForge.net <no...@so...>
To: no...@so...
Subject: [ pybliographer-Bugs-1936923 ] focus problems

Bugs item #1936923, was opened at 2008-04-07 19:20
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=104825&aid=1936923&group_id=4825

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: interface
Group: Cosmetic bug
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Emmanuel Beffara (xbiff)
Assigned to: Nobody/Anonymous (nobody)
Summary: focus problems

Initial Comment:
I find the focus behaviour of windows in Pybliographic unpleasant. That might depend on the window manager and user settings, I am using Gnome with Metacity and without sloppy focus, and Pybliographer 1.2.9 as packaged in Ubuntu. When the entry editor is open, it always stays on top of the main window, however it does not get focus when cycling through windows using Alt-Tab. This is especially annoying when filling an entry from information on a web page: you cannot switch between the browser and the editor using the keyboard, moreover the main window covers the browser window when editing. I think that using the standard focus behaviour for the editor window would be better (surely providing an option for this in the settings would be possible).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=104825&aid=1936923&group_id=4825

[pyblio-devel] [ pybliographer-Bugs-1885316 ] Support for UTF-8 (fwd)

[Zoltan K. <z....@gm...>]

---------- Forwarded message ----------
Date: Fri, 22 Feb 2008 14:18:33 -0800
From: SourceForge.net <no...@so...>
To: no...@so...
Subject: [ pybliographer-Bugs-1885316 ] Support for UTF-8

Bugs item #1885316, was opened at 2008-02-02 19:06
Message generated for change (Comment added) made by jonaskahn
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=104825&aid=1885316&group_id=4825

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Aanjhan (aanjhan)
Assigned to: Nobody/Anonymous (nobody)
Summary: Support for UTF-8

Initial Comment:
Reported in Ubuntu LP: #38708
Link: https://bugs.edge.launchpad.net/ubuntu/+source/pybliographer/+bug/38708

Original report
------------------------------
I am using UTF-8 on my machine. In my environment, there is

LANG=en_GB.UTF-8
LANGUAGE=en_GB:en

Default language in Gnome is English/United Kingdom.

I have edited bibtex files with kile (also on UTF-8). I have books in German, French and Portuguese in addition to English. In those languages there are characters such as Ä or á that are not being used in English. While tetex, gedit or a terminal window are showing them properly, the are shown correctly in pybliographer. é as an example is shown as é.

Regards

Erich
--------------------------------

----------------------------------------------------------------------

Comment By: Jonas Kahn (jonaskahn)
Date: 2008-02-22 23:18

Message:
Logged In: YES 
user_id=2017058
Originator: NO

Well, it would seem it does not understand that pretty standard LateX
either:

@ARTICLE{Guta&Kahn,
   AUTHOR = {Gu\c{t}\u{a}, M. and Kahn, J.},
   TITLE = {Local asymptotic normality for qubit states},
   JOURNAL = {Phys. Rev. A},
   VOLUME = {73},
   PAGES = {052108} ,
   YEAR = 2006,
   URL = {arXiv:quant-ph/0512075},
}

Here, the Romanian letters \c{t} and \u{a} are not understood.

I run the ubuntu gutsy packages, and usually edit my bibtex files
directly.

----------------------------------------------------------------------

Comment By: Frédéric Gobry (gobry)
Date: 2008-02-13 22:25

Message:
Logged In: YES 
user_id=27260
Originator: NO

pybliographer expects standard LaTeX accented characters, like \'e for é.
Out of curiosity, what tool do you use to format your bibtex file?


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=104825&aid=1885316&group_id=4825

[pyblio-devel] [ pybliographer-Bugs-1885316 ] Support for UTF-8

[Zoltan K. <z....@gm...>]

---------- Forwarded message ----------
Date: Sat, 02 Feb 2008 10:06:31 -0800
From: SourceForge.net <no...@so...>
To: no...@so...
Subject: [ pybliographer-Bugs-1885316 ] Support for UTF-8

Bugs item #1885316, was opened at 2008-02-02 21:06
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=104825&aid=1885316&group_id=4825

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Aanjhan (aanjhan)
Assigned to: Nobody/Anonymous (nobody)
Summary: Support for UTF-8

Initial Comment:
Reported in Ubuntu LP: #38708
Link: https://bugs.edge.launchpad.net/ubuntu/+source/pybliographer/+bug/38708

Original report
------------------------------
I am using UTF-8 on my machine. In my environment, there is

LANG=en_GB.UTF-8
LANGUAGE=en_GB:en

Default language in Gnome is English/United Kingdom.

I have edited bibtex files with kile (also on UTF-8). I have books in German, French and Portuguese in addition to English. In those languages there are characters such as Ä or á that are not being used in English. While tetex, gedit or a terminal window are showing them properly, the are shown correctly in pybliographer. é as an example is shown as é.

Regards

Erich
--------------------------------

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=104825&aid=1885316&group_id=4825

Re: [pyblio-devel] openoffice-pyuno and external python programs

[Zoltan K. <z....@gm...>]

On Wed, 12 Dec 2007, Caolan McNamara wrote:

> On Tue, 2007-10-16 at 13:06 +0200, Zoltan Kota wrote:
> > The openoffice.org-pyuno package installs uno.py and other files 
> > under /usr/lib/openoffice.org/program/, so with the default installations 
> > and settings importing uno.py from external python programs gives an 
> > import error.
> 
> And how about in rawhide now, does that work for you as you wanted...
> 
> [caolan@Jehannum ~]$ python -c "import uno"
> [caolan@Jehannum ~]$ 


I checked it in my rawhide box. "import uno" is ok now. I will play a bit 
with the devel version of pyblio to test the OOo connection through uno.

Zoltan

[pyblio-devel] [ pybliographer-Bugs-1808858 ] pybliographic can not open file from command line with space (fwd)

[Zoltan K. <z....@gm...>]

---------- Forwarded message ----------
Date: Sat, 06 Oct 2007 22:49:10 -0700
From: SourceForge.net <no...@so...>
To: no...@so...
Subject: [ pybliographer-Bugs-1808858 ] pybliographic can not open file
    from command line with space

Bugs item #1808858, was opened at 2007-10-06 22:49
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=104825&aid=1808858&group_id=4825

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Luke (lcampagn)
Assigned to: Nobody/Anonymous (nobody)
Summary: pybliographic can not open file from command line with space

Initial Comment:
If I try to open a file from the command line and the file name has spaces in it (for example: $ pybliographic some\ file.bib), I get an error message because the file name is interpreted as two separate files.

This can be fixed easily--change both lines that look like:
    exec ${pyblio} --quiet ${script} $*
to:
    exec ${pyblio} --quiet ${script} "$@"


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=104825&aid=1808858&group_id=4825

Re: [pyblio-devel] format entries

[Zoltan K. <z....@gm...>]

On Sun, 23 Sep 2007, [ISO-8859-1] Fr=E9d=E9ric Gobry wrote:

> > I have just wanted to export/format entries as txt file with pyblio-1.3=
=2E
> > Format menu item (see pyblio-1.2) is removed from the Cite menu? I miss=
ed
> > something, or there is an alternative function somewhere? :-)
>=20
> I haven't yet reintroduced this properly. Basically, formatting would
> be citing to a text file, wouldn't it? Alternatively, I thought about

yes.

> citing to a small window, so that you can copy/paste to you heart's
> content.

Hm. Maybe I would prefer the text file. And what about html? It should be=
=20
also reintroduced I suppose.=20

Zoltan

Re: [pyblio-devel] format entries

[<go...@pu...>]

> I have just wanted to export/format entries as txt file with pyblio-1.3.
> Format menu item (see pyblio-1.2) is removed from the Cite menu? I missed
> something, or there is an alternative function somewhere? :-)

I haven't yet reintroduced this properly. Basically, formatting would
be citing to a text file, wouldn't it? Alternatively, I thought about
citing to a small window, so that you can copy/paste to you heart's
content.

Would that be ok?

--=20
Fr=E9d=E9ric

[pyblio-devel] citeseer support is done

[<go...@pu...>]

Hi,

I've added support for citeseer in pyblio 1.3. It is still a bit flaky
(I haven't written all the tests, and the site itself is a bit flaky,
returning weird results from time to time), but it should be usable.

The code, once refactored, should be a good start for the IEEE parser.
S=E9bastien, I don't know exactly when I'll be able to refactor it
cleanly, so if you have the opportunity please go ahead and put in
some common module what you think you might be able to reuse for the
IEEE parser.

--=20
Fr=E9d=E9ric

Re: [pyblio-devel] [Pybliographer-devel] Load bibtex and then save bibtex - problem

[Samuel J. <ma...@Sa...>]

Hi Doug,




> Many people use, say, bibtex as a sort of XML: they make up fields to store
> values. This doesn't work to well with a paradigm based on schemas. If there
> was an easy to use API for the above, that would be great.

My usecase is indeed very simple - in theory.

Multiple people can open a bibtex file that is under version control
(subversion).
Different gui clients introduce a different mark-up and sorting of the
entries (think of line breaks and so on). These changes do not affect
the content itself but will lead to a lot of changes in the subversion
versioning system, potentionally leading to conflicts.
My idea is: Before the svn commit I run a script that will normalize
the whole bibtex file so that only changes in the content are
reflected. If everyone uses that script before checkin, there should
be no such superflous changes due to markup stuff.

Consider this
@ARTICLE{tom2007, ... }
and this
@article {tom2007,
              ... }

both are the same entry and are valid bibtex, but subversion makes a
difference here.


>
> In the end, I only used Pyblio.Parsers.Syntax.BibTeX.Parser
> and then store common fields in a general database. The results (still in
> alpha) can be seen here:

I thought about that, too ...

>
>  http://myro.roboteducation.org/~dblank/reference/

I do find a lot of things there but where exactly is the bibtex stuff?

cheers
 Samuel

Re: [pyblio-devel] [Pybliographer-devel] Load bibtex and then save bibtex - problem

[Doug B. <dou...@gm...>]

Samuel's original code and confusion was almost identical to mine from a
month or so ago.

I think the main problem is a general usecase that is quite different from
the way pybliographer was originally designed. Basically, the pattern is
very simple:

1) open a bib file of some type (regardless of schema) in memory
2) do something to it
3) write it out in same or different format

Many people use, say, bibtex as a sort of XML: they make up fields to store
values. This doesn't work to well with a paradigm based on schemas. If ther=
e
was an easy to use API for the above, that would be great.

In the end, I only used Pyblio.Parsers.Syntax.BibTeX.Parser and then store
common fields in a general database. The results (still in alpha) can be
seen here:

http://myro.roboteducation.org/~dblank/reference/

-Doug

On 8/14/07, Fr=E9d=E9ric Gobry <go...@pu...> wrote:
>
> > # Get the default parser, though I do not know why this is needed and
> what
> > # are the other options...
> > Registry.parse_default()
>
> This is needed to bootstrap the registry with the default set of
> schemas, adapters, formatters,... In an application, you could have
> specific directories holding these definitions, which you would load
> explicitely. The parse_default() method loads from predefined
> locations (Pyblio/RIP/ mainly). I don't like the idea of doing this
> unconditionally when Pyblio is imported (it's an application-level
> behavior, not a library-level)
>
> > # Get the schema associated with the default bibtex parser
> > bibtex_schema =3D Registry.getSchema("org.pybliographer/bibtex/0.1")
> >
> > # We need to ensure the file does not exist yet.
> > try: os.unlink(out_f)
> > except OSError: pass
> >
> > # Create a new db using this schema. I'd like to use an in-memory
> storage but
> > # somehow then the parsing does not work.
> > db =3D Store.get('file').dbcreate(temp_f, bibtex_schema)
>
> The memory store should work fine with the fix below
>
> > # Import the content of the bibtex file into it
> > reader =3D BibTeX.Reader()
> > reader.parse( open(in_f), db )
> >
> >
> > # To save it as BibTeX, we get an
> > # adapter from PubMed to BibTeX
> > bibtex =3D Adapter.adapt_schema(db, 'org.pybliographer/bibtex/0.1')
>
> You only need to adapt from one schema to another, which is not the
> case here. I'll fix the code so that it's a no-op to call
> adapt_schema() with the same schema.
>
> > # Now we have a "virtual" bibtex database, that we can actually
> > # save as a BibTeX file
> > from Pyblio.Parsers.Semantic.BibTeX import Writer
> >
> > w =3D Writer()
> > w.write(open(out_f, 'w'), bibtex.entries, bibtex)
>
> This code works for me if I get rid of the adapter (and then I can use
> the memory store too).
>
> > Is there an easier way to accomplish this?
>
> There was already a short discussion about providing a simpler api for
> simple tasks. I'm completely ok with this, it's just that I don't have
> the time to work on it right now. Patches welcome :) Doug (cc'ed), did
> you have an opportunity to work on this? Typical tasks should be
> defined first, but this comes with usage. Opening a file of a specific
> format looks like a good candidate, and this layer could arguably run
> parse_default() when it is imported.
>
> >     w.write(open(out_f, 'w'), bibtex.entries, bibtex)
> > AttributeError: 'NoneType' object has no attribute 'entries'
>
> That was caused by adapt_schema returning None (I've fixed that,
> returning db if no conversion is needed, and raising an exception in
> case of Error)
>
> > - Are new fields in the bibtex file supported? For example there is a
> > key called "doi"
> >   that stands for "digital object identifier" in some of my test
> > bibtex files that cannot be loaded due to an unknown "doi" element.
>
> This requires a multi-layered answer :) Yes, you can simply add "doi"
> to the bibtex schema, and support it in the reader and writer classes.
> This is the correct thing to do for "regular" bibtex fields, ie fields
> that are generic enough for everybody (doi probably deserves this
> status). For less generic tags, the correct idea is to have a more
> specific schema, and derive your own parser that handles your specific
> tags.
>
> BibTeX has the particularity (when compared with other formats like
> RIS,...) that everybody can add his own tags. This is very convenient
> locally, but can be annoying when exchanging data. So far, pyblio-1.3
> does not support arbitrary tags very well, because I did not come up
> with a nice way to support them without loosing the advantages of
> having a schema.
>
> --
> Fr=E9d=E9ric
>

Re: [pyblio-devel] [Pybliographer-devel] Load bibtex and then save bibtex - problem

[<go...@pu...>]

> Though this reflects what is done, I (as a user of pyblio) still
> cannot guess why to load any directory first. load_settings(fromDir)
> and load_default_settings() could be an option...
> Then it would be intuitively clear to load the settings first and then
> perform other actions...
> just my 2 cents.

Adopted :)

> > I haven't checked the user documentation for a while, I probably need
> > to revisit it with these new classes.
>
> I guess that would be great!

Feel free to comment / update if you notice problems, as a newcomer
you're in a good position to detect issues there

> > The trace is probably cut, I don't see the actual
> > exception being raised.
>
> That trace was all I get on the console, no further
> exception is raised.

What I mean is that there is usually the name of the actual exception
that happened (AttributeError, KeyError,...)... It's weird that you
don't have it

> That seems ok besides the fact that I don't have (and don't need)
> twisted and bsdbd. My code just uses plain filestore.

Fair enough

> Tomorrow I'll try that arch programm to get the lates dev branch.

Yes please, that will make things more comparable.

> > So your workflow implies managing bibtex entries with completely
> > arbitrary fields that you need to "properly" handle? Is it good enough
> > to assume these fields are all of type Text? Do you need to actually
> > touch these fields or is it enough not to drop them?
>
> Text as field type is fine. It would be ok if these fields
>  a) do not produce an error on readind a bibtex file and
>  b) show up again, when saving to bibtex.
>
> I do _not_ need to access (and modify) them from within pyblio.

Ok. There is a Blob type I planned to introduce, which might be a
better match for this (there will be no attempt to index & search
them).

As this is a common request, I'll create a derived BibTeX Reader and
Writer, with a different schema that is basically the standard schema
+ a blob field, and the Reader & Writers will use the content of the
blob to store the extra fields. I'll keep you posted.

--=20
Fr=E9d=E9ric

Re: [pyblio-devel] [Pybliographer-devel] Load bibtex and then save bibtex - problem

[Samuel J. <ma...@Sa...>]

On 8/15/07, Fr=E9d=E9ric Gobry <go...@pu...> wrote:
>   load(directory)
>   load_default_directories()

Though this reflects what is done, I (as a user of pyblio) still
cannot guess why to load any directory first. load_settings(fromDir)
and load_default_settings() could be an option...
Then it would be intuitively clear to load the settings first and then
perform other actions...
just my 2 cents.


> I haven't checked the user documentation for a while, I probably need
> to revisit it with these new classes.

I guess that would be great!


> The trace is probably cut, I don't see the actual
> exception being raised.

That trace was all I get on the console, no further
exception is raised.


> Do you have problems when you run pyblio's
> testsuite?

./testsuite.sh
'ut_crossref.py': missing dependency No module named
twisted.webwarning: only te    sting the file store: bsddb is too old
((4, 3, 0, 0, 0) instead of (4, 3, 3, 0,     0))
'ut_pubmed.py': missing dependency No module named
twisted.trialwarning: only te    sting the file store: store 'bsddb'
is not available
'ut_wok.py': missing dependency No module named twisted.webunittest:
...........    ............................................................=
....................
   ........................................................................
unittest: [163 tests in 1.517s]
unittest: OK

That seems ok besides the fact that I don't have (and don't need)
twisted and bsdbd. My code just uses plain filestore.
Tomorrow I'll try that arch programm to get the lates dev branch.



> So your workflow implies managing bibtex entries with completely
> arbitrary fields that you need to "properly" handle? Is it good enough
> to assume these fields are all of type Text? Do you need to actually
> touch these fields or is it enough not to drop them?

Text as field type is fine. It would be ok if these fields
 a) do not produce an error on readind a bibtex file and
 b) show up again, when saving to bibtex.

I do _not_ need to access (and modify) them from within pyblio.



best regards
 Samuel


--=20
 Dipl. Inf. Samuel John
 Ph.D. student at
   Faculty of Technology, Neuroinformatics Group,
   Bielefeld University, 33501 Bielefeld, Germany
 in cooperation with
   HONDA Research Institute Europe GmbH
   Carl-Legien-Stra=DFe 30, D-63073 Offenbach/Main, Germany

Re: [Pybliographer-devel] Load bibtex and then save bibtex - problem

[<go...@pu...>]

> Ok the need for Registry.parse_default() is understandable and ok, I
> just wish there were some help/docu that tells me what and why. Now as
> you explained it, I fully agree. It's just that the name
> "init_default" or "load_defaults" may perhaps be better suited for the
> developer.

I suck at naming :) (which is a problem when writing APIs...) I've
renamed it, as it is  best done now than later:

  load(directory)
  load_default_directories()

> Yep, alright. That was just because I did not knew better and was not
> able to figure it out on my own.

I haven't checked the user documentation for a while, I probably need
to revisit it with these new classes.

>
>
> > > w =3D Writer()
> > > w.write(open(out_f, 'w'), bibtex.entries, bibtex)
> >
> > This code works for me if I get rid of the adapter (and then I can use
> > the memory store too).
>
> Well, sadly I have still an error, but another one (see my code at the
> bottom of this mail as reference. I use the sample.bib you provided
> with the distribution):
>   File "bibtexnorm.py", line 25, in ?
>     w.write(open(out_f, 'w'), db.entries, db)
>   File "/home/sjohn/lib/python/pybliographer-1.3.3-py2.4.egg/Pyblio/Parse=
rs/Syntax/BibTeX/__init__.py",
> line 610, in write
>     self.record_begin ()
>   File "/home/sjohn/lib/python/pybliographer-1.3.3-py2.4.egg/Pyblio/Parse=
rs/Syntax/BibTeX/__init__.py",
> line 569, in record_begin
>     self.key =3D str (self.record ['id'] [0])
>
> I use python 2.4.1 here on my system (I don't have admin rights and
> have to wait until 2.5 is installed, but I think 2.4.x should do it).

Yes, I try to be a bit conservative wrt python versions, and even 2.3
should do it. The trace is probably cut, I don't see the actual
exception being raised. In any case, the code you attached works fine
with sample.bib here. Do you have problems when you run pyblio's
testsuite?

> > > - Are new fields in the bibtex file supported? For example there is a
> > > key called "doi"
> > >   that stands for "digital object identifier" in some of my test
> > > bibtex files that cannot be loaded due to an unknown "doi" element.
> >
> > This requires a multi-layered answer :) Yes, you can simply add "doi"
> > to the bibtex schema, and support it in the reader and writer classes.
>
> I will definetly need this, so I need to find out how to do it.
>
>
> > This is the correct thing to do for "regular" bibtex fields, ie fields
> > that are generic enough for everybody (doi probably deserves this
> > status). For less generic tags, the correct idea is to have a more
> > specific schema, and derive your own parser that handles your specific
> > tags.
>
> Hmmm ...
>
> >
> > BibTeX has the particularity (when compared with other formats like
> > RIS,...) that everybody can add his own tags.
>
> Yes, I know, and we use this for some additions.
>
> [...]
> > can be annoying when exchanging data. So far, pyblio-1.3
> > does not support arbitrary tags very well, because I did not come up
> > with a nice way to support them without loosing the advantages of
> > having a schema.
>
> I agree that conversion to and from other formats will be
> hard/impossible(?) with arbitrary fields. But I have to find a
> solution for this, otherwise I cannot use pybliographer for our
> needs... :-(

So your workflow implies managing bibtex entries with completely
arbitrary fields that you need to "properly" handle? Is it good enough
to assume these fields are all of type Text? Do you need to actually
touch these fields or is it enough not to drop them?

--=20
Fr=E9d=E9ric

Re: [Pybliographer-devel] Load bibtex and then save bibtex - problem

[Samuel J. <ma...@Sa...>]

Hello!

Ok the need for Registry.parse_default() is understandable and ok, I
just wish there were some help/docu that tells me what and why. Now as
you explained it, I fully agree. It's just that the name
"init_default" or "load_defaults" may perhaps be better suited for the
developer.


> > # Get the schema associated with the default bibtex parser
> The memory store should work fine with the fix below

Ok, the moromy store works for me too, now.


> You only need to adapt from one schema to another, which is not the
> case here. I'll fix the code so that it's a no-op to call
> adapt_schema() with the same schema.

Yep, alright. That was just because I did not knew better and was not
able to figure it out on my own.


> > w = Writer()
> > w.write(open(out_f, 'w'), bibtex.entries, bibtex)
>
> This code works for me if I get rid of the adapter (and then I can use
> the memory store too).

Well, sadly I have still an error, but another one (see my code at the
bottom of this mail as reference. I use the sample.bib you provided
with the distribution):
  File "bibtexnorm.py", line 25, in ?
    w.write(open(out_f, 'w'), db.entries, db)
  File "/home/sjohn/lib/python/pybliographer-1.3.3-py2.4.egg/Pyblio/Parsers/Syntax/BibTeX/__init__.py",
line 610, in write
    self.record_begin ()
  File "/home/sjohn/lib/python/pybliographer-1.3.3-py2.4.egg/Pyblio/Parsers/Syntax/BibTeX/__init__.py",
line 569, in record_begin
    self.key = str (self.record ['id'] [0])

I use python 2.4.1 here on my system (I don't have admin rights and
have to wait until 2.5 is installed, but I think 2.4.x should do it).
I don't know how to fix this. The dependecies (cElementTree, NumPy)
are working fine and I their self-tests are ok.




> > Is there an easier way to accomplish this?
>
> There was already a short discussion about providing a simpler api for
> simple tasks. I'm completely ok with this, it's just that I don't have
> the time to work on it right now. Patches welcome :) Doug (cc'ed), did
> you have an opportunity to work on this? Typical tasks should be
> defined first, but this comes with usage. Opening a file of a specific
> format looks like a good candidate, and this layer could arguably run
> parse_default() when it is imported.

That would be a nice addition but I am fine with the way it is now
since I don't think it is too complicated, once you know what you have
to write. The steps are clear: 1. choose a schema, 2. setup the
database, 3. parse some content 4. use the writer to write to a file.


>
> >     w.write(open(out_f, 'w'), bibtex.entries, bibtex)
> > AttributeError: 'NoneType' object has no attribute 'entries'
>
> That was caused by adapt_schema returning None (I've fixed that,
> returning db if no conversion is needed, and raising an exception in
> case of Error)

Yes, you are right. This one is fixed now, when I use the db instead
of the superflous adapter. But (see above) another error shows up.


>
> > - Are new fields in the bibtex file supported? For example there is a
> > key called "doi"
> >   that stands for "digital object identifier" in some of my test
> > bibtex files that cannot be loaded due to an unknown "doi" element.
>
> This requires a multi-layered answer :) Yes, you can simply add "doi"
> to the bibtex schema, and support it in the reader and writer classes.

I will definetly need this, so I need to find out how to do it.


> This is the correct thing to do for "regular" bibtex fields, ie fields
> that are generic enough for everybody (doi probably deserves this
> status). For less generic tags, the correct idea is to have a more
> specific schema, and derive your own parser that handles your specific
> tags.

Hmmm ...

>
> BibTeX has the particularity (when compared with other formats like
> RIS,...) that everybody can add his own tags.

Yes, I know, and we use this for some additions.

[...]
> can be annoying when exchanging data. So far, pyblio-1.3
> does not support arbitrary tags very well, because I did not come up
> with a nice way to support them without loosing the advantages of
> having a schema.

I agree that conversion to and from other formats will be
hard/impossible(?) with arbitrary fields. But I have to find a
solution for this, otherwise I cannot use pybliographer for our
needs... :-(



Any ideas on this and the bug from above?





Here follows the code as it is right now. I load the sample.bib as
in_f and just want to write it as bibtex into an output file.

----------------------------------
import sys, os

in_f, out_f = sys.argv[1:3]

from Pyblio.Parsers.Semantic import BibTeX
from Pyblio import Store, Registry
from Pyblio.Parsers.Semantic.BibTeX import Writer

Registry.parse_default()
bibtex_schema = Registry.getSchema("org.pybliographer/bibtex/0.1")
db = Store.get('memory').dbcreate(None, bibtex_schema)

# We need to ensure the file does not exist yet.
try: os.unlink(out_f)
except OSError: pass


# Import the content of the bibtex file (in_f) into it
reader = BibTeX.Reader()
reader.parse( open(in_f), db )


w = Writer()
w.write(open(out_f, 'w'), db.entries, db)
----------------------------------



Thanks & cheers
 Samuel

Re: [Pybliographer-devel] Load bibtex and then save bibtex - problem

[<go...@pu...>]

> # Get the default parser, though I do not know why this is needed and wha=
t
> # are the other options...
> Registry.parse_default()

This is needed to bootstrap the registry with the default set of
schemas, adapters, formatters,... In an application, you could have
specific directories holding these definitions, which you would load
explicitely. The parse_default() method loads from predefined
locations (Pyblio/RIP/ mainly). I don't like the idea of doing this
unconditionally when Pyblio is imported (it's an application-level
behavior, not a library-level)

> # Get the schema associated with the default bibtex parser
> bibtex_schema =3D Registry.getSchema("org.pybliographer/bibtex/0.1")
>
> # We need to ensure the file does not exist yet.
> try: os.unlink(out_f)
> except OSError: pass
>
> # Create a new db using this schema. I'd like to use an in-memory storage=
 but
> # somehow then the parsing does not work.
> db =3D Store.get('file').dbcreate(temp_f, bibtex_schema)

The memory store should work fine with the fix below

> # Import the content of the bibtex file into it
> reader =3D BibTeX.Reader()
> reader.parse( open(in_f), db )
>
>
> # To save it as BibTeX, we get an
> # adapter from PubMed to BibTeX
> bibtex =3D Adapter.adapt_schema(db, 'org.pybliographer/bibtex/0.1')

You only need to adapt from one schema to another, which is not the
case here. I'll fix the code so that it's a no-op to call
adapt_schema() with the same schema.

> # Now we have a "virtual" bibtex database, that we can actually
> # save as a BibTeX file
> from Pyblio.Parsers.Semantic.BibTeX import Writer
>
> w =3D Writer()
> w.write(open(out_f, 'w'), bibtex.entries, bibtex)

This code works for me if I get rid of the adapter (and then I can use
the memory store too).

> Is there an easier way to accomplish this?

There was already a short discussion about providing a simpler api for
simple tasks. I'm completely ok with this, it's just that I don't have
the time to work on it right now. Patches welcome :) Doug (cc'ed), did
you have an opportunity to work on this? Typical tasks should be
defined first, but this comes with usage. Opening a file of a specific
format looks like a good candidate, and this layer could arguably run
parse_default() when it is imported.

>     w.write(open(out_f, 'w'), bibtex.entries, bibtex)
> AttributeError: 'NoneType' object has no attribute 'entries'

That was caused by adapt_schema returning None (I've fixed that,
returning db if no conversion is needed, and raising an exception in
case of Error)

> - Are new fields in the bibtex file supported? For example there is a
> key called "doi"
>   that stands for "digital object identifier" in some of my test
> bibtex files that cannot be loaded due to an unknown "doi" element.

This requires a multi-layered answer :) Yes, you can simply add "doi"
to the bibtex schema, and support it in the reader and writer classes.
This is the correct thing to do for "regular" bibtex fields, ie fields
that are generic enough for everybody (doi probably deserves this
status). For less generic tags, the correct idea is to have a more
specific schema, and derive your own parser that handles your specific
tags.

BibTeX has the particularity (when compared with other formats like
RIS,...) that everybody can add his own tags. This is very convenient
locally, but can be annoying when exchanging data. So far, pyblio-1.3
does not support arbitrary tags very well, because I did not come up
with a nice way to support them without loosing the advantages of
having a schema.

--=20
Fr=E9d=E9ric

[Pybliographer-devel] Load bibtex and then save bibtex - problem

[Samuel J. <ma...@Sa...>]

Hi there!

I use pybliographer 1.3.3 as it can be downloaded from sourceforge and
just want to try to load a bibtex file and save it again. But I am not
able to get it working. Perhaps someone here can help. My code:


------------------------------------

import sys, os

in_f, temp_f, out_f =3D sys.argv[1:4]

from Pyblio.Parsers.Semantic import BibTeX
from Pyblio import Store, Registry, Adapter

# Get the default parser, though I do not know why this is needed and what
# are the other options...
Registry.parse_default()

# Get the schema associated with the default bibtex parser
bibtex_schema =3D Registry.getSchema("org.pybliographer/bibtex/0.1")

# We need to ensure the file does not exist yet.
try: os.unlink(out_f)
except OSError: pass

# Create a new db using this schema. I'd like to use an in-memory storage b=
ut
# somehow then the parsing does not work.
db =3D Store.get('file').dbcreate(temp_f, bibtex_schema)

# Import the content of the bibtex file into it
reader =3D BibTeX.Reader()
reader.parse( open(in_f), db )


# To save it as BibTeX, we get an
# adapter from PubMed to BibTeX
bibtex =3D Adapter.adapt_schema(db, 'org.pybliographer/bibtex/0.1')

# Now we have a "virtual" bibtex database, that we can actually
# save as a BibTeX file
from Pyblio.Parsers.Semantic.BibTeX import Writer

w =3D Writer()
w.write(open(out_f, 'w'), bibtex.entries, bibtex)

------------------------------

Is there an easier way to accomplish this?
And more important has anyone a clue why I get the following error:


    w.write(open(out_f, 'w'), bibtex.entries, bibtex)
AttributeError: 'NoneType' object has no attribute 'entries'



Some questions arise:
- Why do I need to call parse_default() ?
- Why does the parsing of bibtex fail when using a 'memory' database?
(that is the reason why I use the temp_f file.)
- Are new fields in the bibtex file supported? For example there is a
key called "doi"
  that stands for "digital object identifier" in some of my test
bibtex files that cannot be loaded due to an unknown "doi" element.


best regards
 Samuel

--=20
 Dipl. Inf. Samuel John
 Ph.D. student at
   Faculty of Technology, Neuroinformatics Group,
   Bielefeld University, 33501 Bielefeld, Germany
 in cooperation with
   HONDA Research Institute Europe GmbH
   Carl-Legien-Stra=DFe 30, D-63073 Offenbach/Main, Germany

Re: [Pybliographer-devel] time to get more feedback?

[Zoltan K. <z....@gm...>]

On Fri, 10 Aug 2007, [ISO-8859-1] Fr=E9d=E9ric Gobry wrote:

> > Sorry, I have been on holiday. I will be 'online' after next week.
> > It would be nice to add the compat elementtree module, and then to
> > package.
>=20
> Do you have something in the work for the compat module?

Not yet. Are you available on IRC in one of the evenings this week? Maybe=
=20
we could discuss a bit.

Zoltan

Re: [Pybliographer-devel] Some confusions... (concerning python-bibtex, pybliographer1.3, pyblio)

[Samuel J. <ma...@Sa...>]

Hello Fr=E9d=E9ric and thanks for the quick answer!

On 8/10/07, Fr=E9d=E9ric Gobry <go...@pu...> wrote:
> We are slowly trying to get rid of the current confusing state. In short:
>
>  - for scripts, you're interested in pyblio-core-1.3. This contains a
> python-only bibtex parser and does not depend on python-bibtex.
Wow that sounds nice and explains why I was not able to find any
import python-bibtex like stuff...

>  - the GUI of the development version is pyblio-1.3, and depends on
> pyblio-core-1.3. The stable version is pyblio-1.2 and depends on
> python-bibtex-1.2.
So, I will remove python-bibtex-1.2 and pyblio-1.2 and just work with
the scripts of 1.3.


> These names match the current Arch branches (our versioning system),
@arch: Interesting, I've never heard of that system. I'll give it a try.


> Yes, definitely. It still has rough edges, and some simple tasks would
> benefit from a simpler API to help people get started, but what is
> there should be decently tested. The API is not yet written in stone,
> so your best bet is to stay close to the arch branch, and send your
> comments/patches here.
Ok, I go for it and post comments and API related sutff here...

best regards,
 Samuel

--=20
 Dipl. Inf. Samuel John
 Ph.D. student at
   Faculty of Technology, Neuroinformatics Group,
   Bielefeld University, 33501 Bielefeld, Germany
 in cooperation with
   HONDA Research Institute Europe GmbH
   Carl-Legien-Stra=DFe 30, D-63073 Offenbach/Main, Germany

Re: [Pybliographer-devel] time to get more feedback?

[<go...@pu...>]

> Sorry, I have been on holiday. I will be 'online' after next week.
> It would be nice to add the compat elementtree module, and then to
> package.

Do you have something in the work for the compat module?

> I'll make a note when I'm back.

Sure, enjoy :)

--=20
Fr=E9d=E9ric

Re: [Pybliographer-devel] Some confusions... (concerning python-bibtex, pybliographer1.3, pyblio)

[<go...@pu...>]

Hi,

> I am evaluating the pybliographer python scripts (not the GUI version)
> to read a bibtex file and write it out again with the goal to keep a
> stable bibtex file which would be more suited to use in a subversion
> system by multiple users and their bibtex clients. By stable I mean
> that whitespaces and ordering of the entries etc. is normalized so
> that changes only occur when the content is changed. So I hope to
> reduce the conflicts when using the bibtex file in a SVN.
>
> The pybliographer seems a good choise for that (I hope so). The
> concept of parsers (syntax and semantic) and schemes is great. But I
> am a bit confused about where to start. I installed python-bibtex with
> all dependencies. Then I installed pybliographer1.3.
>
> Does pybliographer 1.3 rely on python-bibtex or have you rewritten the
> bibtex parser in pure python? Somehow pybliographer 1.3 only installs
> pyblio and not any script called pybliographer.py. The 1.2 version of
> pybliographer indeed has a pybliographer.py. But do I need that
> pybliographer.py when I just want to wirte my own python script that
> loads a bibtex, does some normalitzing and writes it out again to a
> plain bibtex file?

We are slowly trying to get rid of the current confusing state. In short:

 - for scripts, you're interested in pyblio-core-1.3. This contains a
python-only bibtex parser and does not depend on python-bibtex.

 - the GUI of the development version is pyblio-1.3, and depends on
pyblio-core-1.3. The stable version is pyblio-1.2 and depends on
python-bibtex-1.2.

These names match the current Arch branches (our versioning system),
which I suggest you use for development, as I haven't released lots of
changes that have been done on 1.3. We should release new versions in
the 1.3 series soon, with package names that will match these names.
In the meantime, be aware that pybliographer-1.3.3 is really
pyblio-core-1.3, which explains why it does not install the GUI.

> I read the manual.pdf that comes with the 1.3 version and I read the
> html help that is available online.
>
> Is the pyblio (that is installed by pybliographer1.3) ready to use?

Yes, definitely. It still has rough edges, and some simple tasks would
benefit from a simpler API to help people get started, but what is
there should be decently tested. The API is not yet written in stone,
so your best bet is to stay close to the arch branch, and send your
comments/patches here.

Cheers,

--=20
Fr=E9d=E9ric

[Pybliographer-devel] Some confusions... (concerning python-bibtex, pybliographer1.3, pyblio)

[Samuel J. <ma...@Sa...>]

Hi there!

I am evaluating the pybliographer python scripts (not the GUI version)
to read a bibtex file and write it out again with the goal to keep a
stable bibtex file which would be more suited to use in a subversion
system by multiple users and their bibtex clients. By stable I mean
that whitespaces and ordering of the entries etc. is normalized so
that changes only occur when the content is changed. So I hope to
reduce the conflicts when using the bibtex file in a SVN.

The pybliographer seems a good choise for that (I hope so). The
concept of parsers (syntax and semantic) and schemes is great. But I
am a bit confused about where to start. I installed python-bibtex with
all dependencies. Then I installed pybliographer1.3.

Does pybliographer 1.3 rely on python-bibtex or have you rewritten the
bibtex parser in pure python? Somehow pybliographer 1.3 only installs
pyblio and not any script called pybliographer.py. The 1.2 version of
pybliographer indeed has a pybliographer.py. But do I need that
pybliographer.py when I just want to wirte my own python script that
loads a bibtex, does some normalitzing and writes it out again to a
plain bibtex file?

I read the manual.pdf that comes with the 1.3 version and I read the
html help that is available online.

Is the pyblio (that is installed by pybliographer1.3) ready to use?

Thanks in advance and best regards,

  Samuel John

--=20
 Dipl. Inf. Samuel John
 Ph.D. student at
   Faculty of Technology, Neuroinformatics Group,
   Bielefeld University, 33501 Bielefeld, Germany
 in cooperation with
   HONDA Research Institute Europe GmbH
   Carl-Legien-Stra=DFe 30, D-63073 Offenbach/Main, Germany

Re: [Pybliographer-devel] time to get more feedback?

[Zoltan K. <z....@gm...>]

On Sun, 22 Jul 2007, [ISO-8859-1] Fr=E9d=E9ric Gobry wrote:

> I'd like to widen the audience of the current 1.3 branch. Do you think
> you would have time to work on packaging it? Say, to have something by
> the end of this month?

Sorry, I have been on holiday. I will be 'online' after next week.=20
It would be nice to add the compat elementtree module, and then to=20
package.=20
I'll make a note when I'm back.

Zoltan

[Pybliographer-devel] time to get more feedback?

[<go...@pu...>]

Hi,

I'd like to widen the audience of the current 1.3 branch. Do you think
you would have time to work on packaging it? Say, to have something by
the end of this month?

--=20
Fr=E9d=E9ric

Re: [Pybliographer-devel] python 2.5 and ElementTree

[<go...@pu...>]

2007/7/20, Zoltan Kota <z....@gm...>:
> On Thu, 19 Jul 2007, [ISO-8859-1] Fr=E9d=E9ric Gobry wrote:
>
> > > try:
> > >    from xml.etree.cElementTree import ElementTree, TreeBuilder
> > > except ImportError:
> > >    from cElementTree import ElementTree, TreeBuilder
>
> >
> > Good catch. Yes, it's definitely a good idea. Maybe making this in a
> > compatibility module,
> > so that we don't need to put this in each and every file that uses
> > elementtree? I had the
> > same issue with berkeley db, though its usage is limited to a single fi=
le.
>
>
> A grep run on Pyblio dir gives the following matches:
>
> ./Store.py:55:from cElementTree import ElementTree, iterparse, tostring
> ./Cite/Citator.py:33:from cElementTree import ElementTree
> ./External/WOK.py:10:from cElementTree import ElementTree, XML, dump
> ./External/PubMed.py:34:from cElementTree import ElementTree, XML, dump
> ./Schema.py:38:from cElementTree import ElementTree
> ./Parsers/Syntax/XMLEndNote.py:28:import cElementTree as ElementTree
> ./Parsers/Syntax/XMLMARC.py:29:import cElementTree as ElementTree
>
>
> These imports are a bit 'stochastic'. Do you have any suggestion how to
> make a compat module?

A simple way would be to make a smbol ElementTree available from the
compat module, and update all the references so that they use a
qualified name (ElementTree.iterparse, ElementTree.ElementTree,...)

--=20
Fr=E9d=E9ric

Re: [Pybliographer-devel] support for web of knowledge, new ui for external queries

[<go...@pu...>]

> It does except one. Check the 1.2 ui:
> The first filter left from '[]Only items with abstracts' (containing
> options like 'All Fields', 'Affiliation', 'Author Name',...) is missing
> from the new UI.

I've added this missing field.

--=20
Fr=E9d=E9ric