refdb-users — List for RefDB users

Re: [Refdb-users] About IDs and citation keys

[Markus H. <mar...@mh...>]

Quoting Torsten Bronger <br...@ph...>:

> Hallöchen!
>
> I need something that *never* changes for a reference entry, so I
> plan to use the ID.  My problem is that I don't easily get it after
> addition of the reference.  I only get the generated citation key.
> But the citation key can change.
>

The citation key is not supposed to change. In fact, it takes quite  
some effort to change the citation key. I'm not aware of a simple way  
using RIS, and risx prefers the citation key over the ID if both are  
specified. Also, if you have to re-create a database from a RIS or  
risx dump, the ID is more likely to change than the citation key.

Do you have some scenario in mind which is likely to alter the citation key?

regards,
Markus



-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-users] About IDs and citation keys

[Torsten B. <br...@ph...>]

Hallöchen!

I need something that *never* changes for a reference entry, so I
plan to use the ID.  My problem is that I don't easily get it after
addition of the reference.  I only get the generated citation key.
But the citation key can change.

So after having added the references, I must search for them with
the citation keys in order to get the IDs.

Is all this correct?

Tschö,
Torsten.

-- 
Torsten Bronger, aquisgrana, europa vetus
                   Jabber ID: tor...@ja...

[Refdb-users] Error creating docbook bibliography

[Markus H. <mar...@mh...>]

Hi,

 > <citationlist>
 > 	<citation>date83</citation>
 > 	</citationlist>

The problem here is that the tutorial apparently did not tell you
explicitly that you need to edit mydoc.short.xml if you intend to use
the short notation of references. If you edit mydoc.xml, you need to
use the full notation. In any case, the id file should look like this:

<citationlist>
   <citation>
      <xref>IDOkuda1993</xref>
   </citation>
</citationlist>

 > 
 > The only problem here is the inexistence of 
 > http://refdb.sourceforge.net/dtd/citationlistx.dtd.
 > 

The DTDs were reorganized on the web page quite a while
ago. Unfortunately the tutorial is even older, so the links are dead
indeed. The correct URL is:

http://refdb.sourceforge.net/dtd/citationlistx/citationlistx.dtd

 > t_refdb.refdb_citekey IN () ORDER BY t_refdb.refdb_id

This is a consequence of the fact that refdbd was not able to extract
valid IDs or citekeys from your id file. This should work ok as soon
as you end up with a valid id file.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-users] Error creating docbook bibliography

[Dr. M. G. <go...@Hd...>]

I've tried to follow the instructions given in 
http://refdb.sourceforge.net/tutorial/x1442.html to generate a docbook 
bibliograpy for the following docbook5 document test instance mydoc.xml:

<?xml version="1.0" encoding="UTF-8"?>
<book version="5.0" xmlns="http://docbook.org/ns/docbook"
       xmlns:xlink="http://www.w3.org/1999/xlink"
       xmlns:xi="http://www.w3.org/2001/XInclude"
       xmlns:svg="http://www.w3.org/2000/svg"
       xmlns:m="http://www.w3.org/1998/Math/MathML"
       xmlns:html="http://www.w3.org/1999/xhtml"
       xmlns:db="http://docbook.org/ns/docbook">
   <chapter>
     <title>Test</title>

     <section>
       <title>A citation</title>
       <para>We only need <citation role="REFDB">date83</citation>.</para>
     </section>
   </chapter>
        <xi:include href="mydoc.bib.xml">
   <xi:fallback>mydoc.bib.xml appears to be missing</xi:fallback>
   </xi:include>
</book>

Now the runbib execution on behalf of the generated Makefile fails. The 
command actually reads:

 >runbib -u goik -w XXX -d mi -G raw.css -r -t db50x -E utf-8 mydoc.xml
 >select failed

The List of citations gets extracted correctly:

 > cat mydoc.id.xml
<?xml version="1.0"?>
<!DOCTYPE citationlist PUBLIC "-//Markus Hoenicka//DTD CitationList//EN" 
"http://refdb.sourceforge.net/dtd/citationlistx.dtd">

<citationlist>
	<citation>date83</citation>
	</citationlist>

The only problem here is the inexistence of 
http://refdb.sourceforge.net/dtd/citationlistx.dtd.

On the refdb server side I find the following incorrect SQL statement:

...
SELECT meta_app,meta_type,meta_dbversion FROM t_meta WHERE meta_type='risx'
connected to database server using database:
mi
no character encoding conversion required
output encoding is:
UTF-8
chunk added successfully
finished transferring data
chunk added successfully
SELECT DISTINCT t_refdb.refdb_id, t_refdb.refdb_type, 
t_refdb.refdb_pubyear, t_refdb.refdb_startpage, t_refdb.refdb_endpage, 
t_refdb.refdb_abstract, t_refdb.refdb_title, t_refdb.refdb_volume, 
t_refdb.refdb_issue, t_refdb.refdb_booktitle, t_refdb.refdb_city, 
t_refdb.refdb_publisher, t_refdb.refdb_title_series, 
t_refdb.refdb_address, t_refdb.refdb_issn, t_refdb.refdb_periodical_id, 
t_refdb.refdb_pyother_info, t_refdb.refdb_secyear, 
t_refdb.refdb_secother_info, t_refdb.refdb_user1, t_refdb.refdb_user2, 
t_refdb.refdb_user3, t_refdb.refdb_user4, t_refdb.refdb_user5, 
t_refdb.refdb_typeofwork, t_refdb.refdb_area, t_refdb.refdb_ostype, 
t_refdb.refdb_degree, t_refdb.refdb_runningtime, 
t_refdb.refdb_classcodeintl, t_refdb.refdb_classcodeus, 
t_refdb.refdb_senderemail, t_refdb.refdb_recipientemail, 
t_refdb.refdb_mediatype, t_refdb.refdb_numvolumes, 
t_refdb.refdb_edition, t_refdb.refdb_computer, 
t_refdb.refdb_conferencelocation, t_refdb.refdb_registrynum, 
t_refdb.refdb_classification, t_refdb.refdb_section, 
t_refdb.refdb_pamphletnum, t_refdb.refdb_chapternum, 
t_refdb.refdb_citekey FROM t_refdb WHERE refdb_type!='DUMMY' AND 
t_refdb.refdb_citekey IN () ORDER BY t_refdb.refdb_id
1064: You have an error in your SQL syntax; check the manual that 
corresponds to your MySQL server version for the right syntax to use 
near ') ORDER BY t_refdb.refdb_id' at line 1
select failed
...

Of courcse this statement is ill-formed since it contains "... AND 
t_refdb.refdb_citekey IN () ...". I suspect the key "date83" (which is 
present in the database in question) is missing here e.g. "... AND 
t_refdb.refdb_citekey IN ('date83') ..."?

My System: Linux, Fedora core 10, refdb 0.9.9-1, built according to the 
installation instructions

I've also tried the latest SVN version with exactly the same result.

-- 
Martin Goik  Tel.  +49-711-8923-2164  http://www.HdM-Stuttgart.de/~goik
GnuPG public key: https://www.hdm-stuttgart.de/~goik/goik.asc

Der Wohlstand ist das Durchgangsstadium von der Armut
zur Unzufriedenheit                      (Paul Spree)

Re: [Refdb-users] Retrieving reference lists

[Markus H. <mar...@mh...>]

Quoting Torsten Bronger <br...@ph...>:

> Hallöchen!
>
> What's the best method to get all reference lists of a particular
> user?
>
>     :NCK:~^username-   or    :NTI:~^username-
>
> or something completely different?
>

Both should work just fine. There is no particular command for  
retrieving reference lists by username.

regards,
Markus


-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-users] Retrieving reference lists

[Torsten B. <br...@ph...>]

Hallöchen!

What's the best method to get all reference lists of a particular
user?

    :NCK:~^username-   or    :NTI:~^username-

or something completely different?

Tschö,
Torsten.

-- 
Torsten Bronger, aquisgrana, europa vetus
                   Jabber ID: tor...@ja...

Re: [Refdb-users] About the AV and L1 fields

[Torsten B. <br...@ph...>]

Hallöchen!

Markus Hoenicka writes:

> Torsten Bronger writes:
>
>> But is L2 a personal or a global field?  According to the manual,
>> they "hold information which is stored for each user separately".
>> 
>> The RISX DTD has <link> tags in both the pubinfo (global) and the
>> libinfo (personal) elements.
>
> The sources say that L1 through L4 elements from risx pubinfo
> elements are imported without being attached to a particular
> user. In contrast, L1 through L4 elements from libinfo elements
> are owned by the corresponding user. When you retrieve risx data,
> you'll get back both types, i.e. the "global" pubinfo links and
> the personal libinfo links. A global paper copy cannot be defined
> at this time, except by abusing L2.

Sounds close to perfect for my plan.  Okay, then:

L1 (pubinfo): path to the public PDF
L2 (pubinfo): locations of offprint
L1 (libinfo): path to personal PDF
AV: location of personal offprint
RP: whether there is a personal offprint
U1: relevance
U2: global notes
U3: group ID list
U4: whether it is an institute's publication

It's slightly strange to distinguish between "personal" and "global"
offprint, but maybe there are people who don't want to share their
copies.  ;-)  Be that as it may, I'm closer to the original
semantics now, which is certainly good.

> [...]
>
>> Currently, I use the extension fields like this:
>> 
>> U1: globally set relevance, from "*" to "*****"
>> U2: global notes, in Markdown syntax
>> U3: relative path to the PDF on the central server
>> U4: locations of printed copies
>> M1: comma-separated list of Django group IDs to which this reference
>>     belongs
>
> What are these group IDs used for? Is that similar to Unix groups?

Yes, very similar.  A user can be in zero or more groups.

Actually, the Django framework defines groups to organise users,
however, you can connect everything to them.  For example, if there
is a "polymer technology" user group in an institute, there is also
a bunch of "polymer technology" papers.

Tschö,
Torsten.

-- 
Torsten Bronger, aquisgrana, europa vetus
                   Jabber ID: tor...@ja...

Re: [Refdb-users] About the AV and L1 fields

[Markus H. <mar...@mh...>]

Torsten Bronger writes:
 > 
 > But is L2 a personal or a global field?  According to the manual,
 > they "hold information which is stored for each user separately".
 > 
 > The RISX DTD has <link> tags in both the pubinfo (global) and the
 > libinfo (personal) elements.
 > 

The sources say that L1 through L4 elements from risx pubinfo elements
are imported without being attached to a particular user. In contrast,
L1 through L4 elements from libinfo elements are owned by the
corresponding user. When you retrieve risx data, you'll get back both
types, i.e. the "global" pubinfo links and the personal libinfo
links. A global paper copy cannot be defined at this time, except by
abusing L2.

 > This sounds neat, however, it would not help me because I need three
 > things: a global link to a file, a local link to a file, and a
 > real-world place of an offprint.  It is so complicated because some
 > collegues need personal copies of PDFs in order to write notes in
 > the PDF file.  (Weird.)
 > 

If I understand correctly, all this information could be stored using
risx. It is just that AV is not a global field per se, but if you
employ some mechanism to set this only once you'd get
close. Otherwise, as I've mentioned previously, you'd get the first
available AV entry which may or may not be useful.

 > Currently, I use the extension fields like this:
 > 
 > U1: globally set relevance, from "*" to "*****"
 > U2: global notes, in Markdown syntax
 > U3: relative path to the PDF on the central server
 > U4: locations of printed copies
 > M1: comma-separated list of Django group IDs to which this reference
 >     belongs

What are these group IDs used for? Is that similar to Unix groups?

 > M2: "1" for "institute publication", "0" for "not".
 > 

In general, it is not recommended to store user-defined information in
RIS M1 through M3 fields. RIS uses an arcane mechanism to map
information in these fields to reference type specific data (e.g. area
for maps, ostype for computer programs, doi for various types). You'll
lose information if you write your own stuff to these fields. If you
use global and personal links as intended, you'd free U3 and U4 for
the information you'd like to store in M1 and M2.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-users] About the AV and L1 fields

[Torsten B. <br...@ph...>]

Hallöchen!

Markus Hoenicka writes:

> Quoting Torsten Bronger <br...@ph...>:
>
>> I'd like to store non-URL location information in the
>> bibliographic datasets, e.g. "a printed copy is available in the
>> office of Dr. Meier".  This should be *global* information,
>> i.e. not only available to a single user.
>>
>> The AV tag is only available to one user.  While the RefDB
>> tutorial says that only N1, AV, and RP are personal, the manual
>> suggests that this is also true for L2.  Besides, L2 seem to
>> store only URLs.  However, I don't really know, as RIS itself is
>> very poorly specified.
>>
>> I don't want to waste a valuable U1..U5 field for this.  So can I
>> use L2?
>
> Basically yes. To the best of my knowledge refdbd does not check
> whether the contents of L2 look like an URL. [...]

But is L2 a personal or a global field?  According to the manual,
they "hold information which is stored for each user separately".

The RISX DTD has <link> tags in both the pubinfo (global) and the
libinfo (personal) elements.

> However, there used to be a more ingenous way to handle this. The
> get_reprint() backend function takes a username as an argument
> (among others). [...] This would give each user a chance to locate
> at least one copy, regardless of who provided the
> information. This mechanism has the advantage that each user can
> still request his own availability if needed.

This sounds neat, however, it would not help me because I need three
things: a global link to a file, a local link to a file, and a
real-world place of an offprint.  It is so complicated because some
collegues need personal copies of PDFs in order to write notes in
the PDF file.  (Weird.)

Currently, I use the extension fields like this:

U1: globally set relevance, from "*" to "*****"
U2: global notes, in Markdown syntax
U3: relative path to the PDF on the central server
U4: locations of printed copies
M1: comma-separated list of Django group IDs to which this reference
    belongs
M2: "1" for "institute publication", "0" for "not".


The distinction between "U" and "M" is that I may use extended notes
for "M" someday when I have too much time.  ;-)  Furthermore:


RP: can never be "on request"
AV: link to the personal file for the user on the central server
N1: personal notes, in Markdown syntax

> [...]
>
> - use yet another switch for the getref command that consolidates  
> personal information if the current user does not have any.

This sounds more explicit so I would prefer this.  But again, do it
only if you see a real use case.  I am not such a case because I
need a clearly defined fallback to the *central* PDF if the user's
AV is empty.  Besides, I need a PDF link and an offprint location.

Tschö,
Torsten.

-- 
Torsten Bronger, aquisgrana, europa vetus
                   Jabber ID: tor...@ja...

Re: [Refdb-users] About the AV and L1 fields

[Markus H. <mar...@mh...>]

Quoting Torsten Bronger <br...@ph...>:

> I'd like to store non-URL location information in the bibliographic
> datasets, e.g. "a printed copy is available in the office of
> Dr. Meier".  This should be *global* information, i.e. not only
> available to a single user.
>
> The AV tag is only available to one user.  While the RefDB tutorial
> says that only N1, AV, and RP are personal, the manual suggests that
> this is also true for L2.  Besides, L2 seem to store only URLs.
> However, I don't really know, as RIS itself is very poorly
> specified.
>
> I don't want to waste a valuable U1..U5 field for this.  So can I
> use L2?
>

Basically yes. To the best of my knowledge refdbd does not check  
whether the contents of L2 look like an URL. Any string should be ok.  
It may just be a bit surprising if the html backend tries to render  
random strings as links, but this may not be a problem in your custom  
interface.

However, there used to be a more ingenous way to handle this. The  
get_reprint() backend function takes a username as an argument (among  
others). If this is not NULL, only the AV info of that particular user  
is returned. If you pass NULL instead, a sequential search tries to  
locate AV fields among all users that either have it in file or on  
request, and returns the first match. This would give each user a  
chance to locate at least one copy, regardless of who provided the  
information. This mechanism has the advantage that each user can still  
request his own availability if needed.

The thing is that this mechanism is simply not used by any of the  
backends at this time. All backends pass usernames to retrieve  
personal information. This is especially true for the risx backend as  
this is designed to preserve the personal information for individual  
users. In your case this is undesired. I can imagine two ways out:

- use yet another switch for the getref command that consolidates  
personal information if the current user does not have any.

- define a special username ("refdb" comes to mind) that retrieves  
consolidated information instead of that of an existing user.

Both changes seem reasonably simple to implement. Let me know if that  
would meet your needs.

regards,
Markus



-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-users] About the AV and L1 fields

[Torsten B. <br...@ph...>]

Hallöchen!

I'd like to store non-URL location information in the bibliographic
datasets, e.g. "a printed copy is available in the office of
Dr. Meier".  This should be *global* information, i.e. not only
available to a single user.

The AV tag is only available to one user.  While the RefDB tutorial
says that only N1, AV, and RP are personal, the manual suggests that
this is also true for L2.  Besides, L2 seem to store only URLs.
However, I don't really know, as RIS itself is very poorly
specified.

I don't want to waste a valuable U1..U5 field for this.  So can I
use L2?

Tschö,
Torsten.

-- 
Torsten Bronger, aquisgrana, europa vetus
                   Jabber ID: tor...@ja...

[Refdb-users] Question about <entry> in RISX

[Markus H. <mar...@mh...>]

Torsten Bronger writes:
 > According to
 > <http://refdb.sourceforge.net/risx/elements/entry.html>, the
 > contents models for <entry> is
 > 
 >     (part?,publication,set?,libinfo*,contents?)+
 > 
 > Why the "+"?  How is this "+" realised in RIS?
 > 

Frankly, I can't remember why. It actually looks pretty useless. The
"+" would allow to have several "instances" of an entry, like when a
work was republished by a different publisher. MODS can do this
too. However, I can't imagine RefDB can handle this information in a
useful way, so I wouldn't rely on the presence of the "+" in the next
release. Thanks a lot for pointing this out.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-users] Question about <entry> in RISX

[Torsten B. <br...@ph...>]

Hallöchen!

According to
<http://refdb.sourceforge.net/risx/elements/entry.html>, the
contents models for <entry> is

    (part?,publication,set?,libinfo*,contents?)+

Why the "+"?  How is this "+" realised in RIS?

Tschö,
Torsten.

-- 
Torsten Bronger, aquisgrana, europa vetus
                   Jabber ID: tor...@ja...

Re: [Refdb-users] Django client

[Torsten B. <br...@ph...>]

Hallöchen!

Bruce D'Arcus writes:

> On Tue, Feb 24, 2009 at 10:49 AM, Torsten Bronger
> <br...@ph...> wrote:
>
> [...]
>
>>> Might there be a way to design them in such a way that someone
>>> could later use a Django-native model instead (for those that
>>> don't want to install RefDB)?
>>
>> No.  I cannot -- and don't want to -- re-engineer the 1,000,000
>> LOC of RefDB.  On the contrary, the current plan is to have no
>> database models in the Django app at all.
>
> No, that's not what I'm saying (though reimplementing a good chunk
> of what RefDB does in Python/Django would require hundreds of
> lines; not a million).

Well, the million was a miscounting of Ohloh, in fact it's 150,000
lines.  Let the pure bibliography management be 20,000 lines in
Python.  This is pretty realistic.  And these are not "easy" 20,000
lines but nasty things like string-parsing, manipulating trees, and
implementing format converters.

> I was more just wondering if it'd be possible to leave room for
> others to do that later.

One can leave the interface to RefDB slim, then it could be
substituted by Django models.

But the better approach is to make more packages of RefDB, e.g. an
Ubuntu package.  The main shortcoming of RefDB on the long run are
import and export formats, which can be easily solved in the
frontend (Django in my case).  The rest just works, and there is no
need to reinvent it.

Tschö,
Torsten.

-- 
Torsten Bronger, aquisgrana, europa vetus
                   Jabber ID: tor...@ja...

Re: [Refdb-users] Django client

[Bruce D'A. <bda...@gm...>]

On Tue, Feb 24, 2009 at 10:49 AM, Torsten Bronger
<br...@ph...> wrote:
> Hallöchen!
>
> Bruce D'Arcus writes:
>
>> [...]
>>
>> So what, then, are relevant class models you are thinking of?
>
> The low-level Python module will implement the classes Reference and
> ReferenceList.  Reference itself contains the classes Part,
> Publication, Set, PubInfo, LibInfo, and Author.  The other elements
> and attributes of RISX are realised through class attributes.

Sounds good. I'd change Author to something more generic though
(Contributor) and maybe subclass it if necessary.

>> Might there be a way to design them in such a way that someone
>> could later use a Django-native model instead (for those that
>> don't want to install RefDB)?
>
> No.  I cannot -- and don't want to -- re-engineer the 1,000,000 LOC
> of RefDB.  On the contrary, the current plan is to have no database
> models in the Django app at all.

No, that's not what I'm saying (though reimplementing a good chunk of
what RefDB does in Python/Django would require hundreds of lines; not
a million). I was more just wondering if it'd be possible to leave
room for others to do that later.

But that might be a bad idea.

>> BTW, awhile back I started working on a collection of little Django
>> apps to manage my own academic content. The "document" app model code
>> is here:
>>
>> <http://github.com/bdarcus/django-scholar/blob/7c06b6b2fd8cda77299e25406d6dc163a8e6ec71/documents/models.py>
>>
>> I went with a much lighter-weight alternative though, so gave up working on it.
>>
>> <http://github.com/bdarcus/mysite/tree/master>
>
> Nice, but this looks very different, more like a scholar version of
> a Personal Information Manager.  ;)

Correct.

Bruce

Re: [Refdb-users] Django client

[Torsten B. <br...@ph...>]

Hallöchen!

Bruce D'Arcus writes:

> [...]
>
> So what, then, are relevant class models you are thinking of?

The low-level Python module will implement the classes Reference and
ReferenceList.  Reference itself contains the classes Part,
Publication, Set, PubInfo, LibInfo, and Author.  The other elements
and attributes of RISX are realised through class attributes.

> Might there be a way to design them in such a way that someone
> could later use a Django-native model instead (for those that
> don't want to install RefDB)?

No.  I cannot -- and don't want to -- re-engineer the 1,000,000 LOC
of RefDB.  On the contrary, the current plan is to have no database
models in the Django app at all.

> BTW, awhile back I started working on a collection of little Django
> apps to manage my own academic content. The "document" app model code
> is here:
>
> <http://github.com/bdarcus/django-scholar/blob/7c06b6b2fd8cda77299e25406d6dc163a8e6ec71/documents/models.py>
>
> I went with a much lighter-weight alternative though, so gave up working on it.
>
> <http://github.com/bdarcus/mysite/tree/master>

Nice, but this looks very different, more like a scholar version of
a Personal Information Manager.  ;)

Tschö,
Torsten.

-- 
Torsten Bronger, aquisgrana, europa vetus
                   Jabber ID: tor...@ja...

Re: [Refdb-users] Django client

[Bruce D'A. <bda...@gm...>]

On Tue, Feb 24, 2009 at 7:31 AM, Torsten Bronger
<br...@ph...> wrote:
> Hallöchen!
>
> I've started
>
> https://www.ohloh.net/p/pyrefdb and
> https://www.ohloh.net/p/django-refdb
>
> however, I need at least basic functionality as soon as possible, so
> I ask here whether someone is willing to help.

I don't have time (and not that much skill either), but can blog your
request if you like.

I've just got a couple of questions though ...

> The goal is to have
> something like http://www.refbase.net but with Django integration,
> simpler interface and RefDB citation lists.
>
> I want to have adding/editing/searching references, PDF upload,
> citation lists, and Endnote/BibTeX export for the first milestone.
> Then it would work for me.  But on the long run, things like mass
> import, different ways to browse through notes, and edit permissions
> are desirable.
>
> RefDB handles all the data.  As long as there is no compelling need
> for it, the Django app contains no database models.
>
> The communication is realised though risx which is converted to
> full-fledged Python objects.

So what, then, are relevant class models you are thinking of?

Might there be a way to design them in such a way that someone could
later use a Django-native model instead (for those that don't want to
install RefDB)?

BTW, awhile back I started working on a collection of little Django
apps to manage my own academic content. The "document" app model code
is here:

<http://github.com/bdarcus/django-scholar/blob/7c06b6b2fd8cda77299e25406d6dc163a8e6ec71/documents/models.py>

I went with a much lighter-weight alternative though, so gave up working on it.

<http://github.com/bdarcus/mysite/tree/master>

Bruce

[Refdb-users] Django client

[Torsten B. <br...@ph...>]

Hallöchen!

I've started

https://www.ohloh.net/p/pyrefdb and
https://www.ohloh.net/p/django-refdb

however, I need at least basic functionality as soon as possible, so
I ask here whether someone is willing to help.  The goal is to have
something like http://www.refbase.net but with Django integration,
simpler interface and RefDB citation lists.

I want to have adding/editing/searching references, PDF upload,
citation lists, and Endnote/BibTeX export for the first milestone.
Then it would work for me.  But on the long run, things like mass
import, different ways to browse through notes, and edit permissions
are desirable.

RefDB handles all the data.  As long as there is no compelling need
for it, the Django app contains no database models.

The communication is realised though risx which is converted to
full-fledged Python objects.

So far, basic communication with the RefDB server works well, but it
returns everything as RIS strings instead of convenient objects.
This is the next construction site.

The frame for the Django application is already running, it can
create RefDB users from Django users automatically, and the
connection to RefDB is working.

Anybody interested?

Tschö,
Torsten.

-- 
Torsten Bronger, aquisgrana, europa vetus
                   Jabber ID: tor...@ja...

Re: [Refdb-users] MySQL v. PostgreSQL

[Tom H. <to...@fu...>]

A number of things to be aware of.

MySQL supports setting the character set and collation at the column  
level. If it is not set it inherits the defauls, latin1 swedish case  
insensitive collation. Or the default for the DB or the server if set.

So the columns need to be set to UTF, explicitly or the default for  
the refdb DB needs to be set.

That controls how the data is stored..

There is more to the story..

Character set and collation for the connection.

You need to set those appropriately for the client.

Set names UTF8 perhaps. Or in a config file.
See this page..http://dev.mysql.com/doc/refman/5.1/en/charset- 
literal.html

I also see the bug in regex matches but I did not see the same issue  
in " Like " comparisons.

Regex is a bytewise comparison.

Like.. I am not sure.

--
Tom

On 20 Feb 2009, at 06:16, Markus Hoenicka wrote:

> Quoting Torsten Bronger <br...@ph...>:
>
>> No, but I'm sure this would solve the problem then.  It sounds
>> highly plausible.
>>
>> Okay, so the only issue that remains is that you can't (sanely)
>> search for patterns like M.ller.  You need it e.g. if you don't know
>> whether Müller was inserted into the DB as Muller or Müller.
>> Admittetly, this use case is rare but it may be confusing sometimes.
>>
>
> Another issue is that the data stored in your MySQL database are not
> valid UTF-8 data. You just don't notice (except when you search for
> umlauts) because MySQL converts them back to correct UTF-8 whenever
> you retrieve data via RefDB. This is going to cause trouble if you
> ever upgrade libdbi-drivers.
>
>> Given that RefDB isn't a resource hog so speed is not an issue, I
>> still think though that PG should be prefered over MySQL in case you
>> have the choice.
>>
>
> If you do that, upgrading libdbi-drivers is certainly not required.
> And this is why RefDB uses libdbi after all: to leave the choice to
> the users.
>
> regards,
> Markus
>
> -- 
> Markus Hoenicka
> mar...@ca...
> (Spam-protected email: replace the quadrupeds with "mhoenicka")
> http://www.mhoenicka.de
>
>
>
> ---------------------------------------------------------------------- 
> --------
> Open Source Business Conference (OSBC), March 24-25, 2009, San  
> Francisco, CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the  
> Enterprise
> -Strategies to boost innovation and cut costs with open source  
> participation
> -Receive a $600 discount off the registration fee with the source  
> code: SFAD
> http://p.sf.net/sfu/XcvMzF8H
> _______________________________________________
> Refdb-users mailing list
> Ref...@li...
> https://lists.sourceforge.net/lists/listinfo/refdb-users

Re: [Refdb-users] MySQL v. PostgreSQL

[Markus H. <mar...@mh...>]

Quoting Torsten Bronger <br...@ph...>:

> No, but I'm sure this would solve the problem then.  It sounds
> highly plausible.
>
> Okay, so the only issue that remains is that you can't (sanely)
> search for patterns like M.ller.  You need it e.g. if you don't know
> whether Müller was inserted into the DB as Muller or Müller.
> Admittetly, this use case is rare but it may be confusing sometimes.
>

Another issue is that the data stored in your MySQL database are not  
valid UTF-8 data. You just don't notice (except when you search for  
umlauts) because MySQL converts them back to correct UTF-8 whenever  
you retrieve data via RefDB. This is going to cause trouble if you  
ever upgrade libdbi-drivers.

> Given that RefDB isn't a resource hog so speed is not an issue, I
> still think though that PG should be prefered over MySQL in case you
> have the choice.
>

If you do that, upgrading libdbi-drivers is certainly not required.  
And this is why RefDB uses libdbi after all: to leave the choice to  
the users.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-users] MySQL v. PostgreSQL

[Torsten B. <br...@ph...>]

Hallöchen!

Markus Hoenicka writes:

> Quoting Torsten Bronger <br...@ph...>:
>
>> 0.8.2 for both.
>
> I think that explains it. The bug in dbd_mysql.c I was talking
> about was fixed in revision 1.93 on Dec 11, 2007 (see
> http://libdbi-drivers.cvs.sourceforge.net/viewvc/libdbi-drivers/libdbi-drivers/drivers/mysql/dbd_mysql.c?view=log). 0.8.2
> was released in Feb 2007. Is there a way to upgrade libdbi and
> libdbi-drivers to 0.8.3?

No, but I'm sure this would solve the problem then.  It sounds
highly plausible.

Okay, so the only issue that remains is that you can't (sanely)
search for patterns like M.ller.  You need it e.g. if you don't know
whether Müller was inserted into the DB as Muller or Müller.
Admittetly, this use case is rare but it may be confusing sometimes.

Given that RefDB isn't a resource hog so speed is not an issue, I
still think though that PG should be prefered over MySQL in case you
have the choice.

Thank you for your clarifications!

Tschö,
Torsten.

-- 
Torsten Bronger, aquisgrana, europa vetus
                   Jabber ID: tor...@ja...

Re: [Refdb-users] MySQL v. PostgreSQL

[Markus H. <mar...@mh...>]

Quoting Torsten Bronger <br...@ph...>:

> 0.8.2 for both.
>

I think that explains it. The bug in dbd_mysql.c I was talking about  
was fixed in revision 1.93 on Dec 11, 2007 (see  
http://libdbi-drivers.cvs.sourceforge.net/viewvc/libdbi-drivers/libdbi-drivers/drivers/mysql/dbd_mysql.c?view=log). 0.8.2 was released in Feb 2007. Is there a way to upgrade libdbi and libdbi-drivers to 0.8.3? If you can't get hold of packages, both should compile out of the box on not too exotic Linux/Unix  
systems.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-users] MySQL v. PostgreSQL

[Torsten B. <br...@ph...>]

Hallöchen!

Markus Hoenicka writes:

> [...]
>
> I recall having a bug in the mysql driver which might cause just
> this (the connection encoding was not set properly, so
> libmysqlclient and the server used different encodings which MySQL
> transforms for us, helpful as it is). This bug was fixed a while
> ago. Which version of libdbi and libdbi-drivers are you using?

0.8.2 for both.

Tschö,
Torsten.

-- 
Torsten Bronger, aquisgrana, europa vetus
                   Jabber ID: tor...@ja...

Re: [Refdb-users] MySQL v. PostgreSQL

[Markus H. <mar...@mh...>]

Torsten Bronger writes:
 > bronger@wilson:~$ iconv -l|grep -i utf8
 > ISO-10646/UTF8/
 > UTF8//
 > 

That explains why refdbd does not bail out. Instead it has iconv
convert the data from utf-8 to utf-8...

 > -  RefDB encodes the UTF-8 data that it gets from me again, but it
 >    does so after it prints it to the log output, or
 > 

No, it doesn't. Most of the code looks like this chunk:

--8<----
      sprintf(sql_command, "INSERT INTO t_user (user_name) VALUES (%s)", quoted_user);

      LOG_PRINT(LOG_DEBUG, sql_command);

      dbires1 = dbi_conn_query(conn, sql_command);
--8<----

i.e. the query string is assembled, and the very same string is first
written to the log and then sent to the database engine.

 > -  RefDB passes UTF-8 data to the database and tells MySQL that it's
 >    Latin-1.
 > 

I recall having a bug in the mysql driver which might cause just this
(the connection encoding was not set properly, so libmysqlclient and
the server used different encodings which MySQL transforms for us,
helpful as it is). This bug was fixed a while ago. Which version of
libdbi and libdbi-drivers are you using?

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-users] MySQL v. PostgreSQL

[Torsten B. <br...@ph...>]

Hallöchen!

Markus Hoenicka writes:

> [...]
>
> [...] The weird thing is that this should cause an iconv error, as
> "UTF8" is not a valid encoding name supported by iconv. What does
>
> iconv -l|grep -i utf8
>
> return on your system?

bronger@wilson:~$ iconv -l|grep -i utf8
ISO-10646/UTF8/
UTF8//

> [...]
>
> Did you check those datasets which you inserted when creating the
> above logs? Is it possible that you changed your setup at some
> point, and only datasets fail which you added before that point?

No, it's very well reproducible here.  The Umlauts are
double-encoded in the MySQL database (I can see it in the MySQL
admin tool).  On the other hand, they are correct in RefDB's log
output.  So there are two possibilities:

-  RefDB encodes the UTF-8 data that it gets from me again, but it
   does so after it prints it to the log output, or

-  RefDB passes UTF-8 data to the database and tells MySQL that it's
   Latin-1.

If I correct the data in the MySQL table, RefDB sends Latin-1 data
to me.

Tschö,
Torsten.

-- 
Torsten Bronger, aquisgrana, europa vetus
                   Jabber ID: tor...@ja...