refdb-devel — List for RefDB developers

[Refdb-devel] refdb-backup portability and design

[Markus H. <mar...@mh...>]

Hi David,

I've tested the refdb-backup script. After some minor adjustments it
appears to do the right thing. There were two portability issues:

- cut does not understand the long options on many platforms. It is
  better to use something like cut -d ' ' -f 1.

- select is a bash feature. Although you use #!/bin/sh at the top,
  you'll actually run bash on Linux (try ls -al /bin/sh). The BSDs
  have a real sh which is different from bash. Therefore I have to
  ask for bash explicitly in the shebang on FreeBSD.

If you want to do this, we'll have to use the configure output to fill
in the location of bash (/bin/bash on Linux and Cygwin,
/usr/local/bin/bash on most BSDs). I do this in a few other scripts so
all we need to do this is already in place. The other option is to
restrict the script to real sh features.

Another, more general problem is that I see a great potential of this
script to run backups automatically. I'd like to define a cron job
that runs this script once a week or so. For this to work we'd need a
way to pass parameters on the command line instead of an interactive
interface.

Therefore I'd suggest to use getopt or getopts to read the required
user input from the command line. getopt is a standalone POSIX
utility, could be used from sh, and should be available on all
platforms. On the other hand, getopts is a bash builtin and simpler to
use.

I can handle the getopts/bash stuff if you agree. Let me know what you
think.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-devel] makestyle (refdb-ms) added to cvs

[David N. <dav...@sw...>]

The refdb-ms (makestyle) utility, recently discussed on refdb-users, has 
been checked in to refdb cvs.  See the top-level directory 'makestyle'.

Many thanks to Markus for his invitation to add refdb-ms to the sf 
repository.

Regards,
David.

[Refdb-devel] Php interface using RefDB

[Markus H. <mar...@mh...>]

Hi,

please browse the refdb-users list archives. There were a couple of
efforts to develop a web-based interface.

regards,
Markus

Olivier M=E9doc writes:
 > Hi. I'm currently in a training which consist in developing an intra=
net
 > in an optic laboratory. A part of the Intranet is a Bibliography sha=
ring
 > application. So i use RefDB. I use refdbc with the -C option in orde=
r to
 > access to the database because i haven't understand the cgi function=
nality.
 >=20
 > If someone is interrested in, or work on the same subject, he can
 > contact me. I am particulary interested in a php client, like the pe=
rl one.
 >=20

--=20
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-devel] Php interface using RefDB

[<oli...@la...>]

Hi. I'm currently in a training which consist in developing an intranet
in an optic laboratory. A part of the Intranet is a Bibliography sharing
application. So i use RefDB. I use refdbc with the -C option in order to
access to the database because i haven't understand the cgi functionnality.

If someone is interrested in, or work on the same subject, he can
contact me. I am particulary interested in a php client, like the perl one.

[Refdb-devel] Buglet in risx handling of journal custom abbreviations

[Markus H. <mar...@mh...>]

Hi Luc,

Luc Pardon writes:
 > 	1) match DTD and source. If you change the DTD, it might be a good idea
 > to test for "customabbrev1 OR "cust1", for compatibility with the
 > current DTD.
 > 

I've fixed the code in risxhandler.c to check for "user1" and "user2".

 > 	2) change the catch-all "else" to read "else if type = "gen"" (in
 > pseusdo-code <g>)
 > 

This actually has to handle the case when there is no type attribute
and when the type attribute is set to "gen". I've changed it this way.

 > 	3) add a new catch-all to yield an error saying "invalid title type"
 >    

Done.

 >    I can provide a test case if desired.
 > 

Please do so. I've checked in the fixed version of risxhandler.c, but
before I close the bug on SF I'd like to run a test with your data.

regards,
Markus


-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-devel] Buglet in risx handling of journal custom abbreviations

[Luc P. <re...@sk...>]

Hello,

   The DTD (and the docs) lists the allowed values for the title type
attribute as : abbrev, full, gen, user1, user2.

   However, at lines 744 and 757 of risxhandler.c (distro of 0.9.4 and
in CVS as rev. 1.28), it tests for custabbrev1 and custabbrev2 instead
of user1 and user2.

   Consequences:

	* when using custabbrev1/2 (as per the source) the XML doesn't validate
against the risx DTD
	* when using cust1/2 (as per the DTD), the contents of these fields are
used as the title of the article, overwriting the real title from the
"part" element in the database. Hence the title is lost.

   The latter is particularly bad, and it occurs because on line 770
there is a catch-all "else" clause that kicks in if the type attribute
is not full, abbrev, custabbrev1 or custabbrev2.

   Suggested changes:

	1) match DTD and source. If you change the DTD, it might be a good idea
to test for "customabbrev1 OR "cust1", for compatibility with the
current DTD.

	2) change the catch-all "else" to read "else if type = "gen"" (in
pseusdo-code <g>)

	3) add a new catch-all to yield an error saying "invalid title type"
   

   Note: the overwriting of the title seems to be dependent on a) the
relative order of the part and publication elements and b) the presence
or not of author elements inside the publication element. 

   I can provide a test case if desired.

   I have created bug report 949697 for this as well.

   Best regards,

   Luc Pardon

Re: [Refdb-devel] parentheses in keywords

[rob c. <ro...@st...>]

>> Ah, this is a different story. Bash eats the double quotes so
>> refdbc will never see them. This one works, though:
>> 
>> refdbc -C getref "\":KW:='52-67-5 \(Penicillamine\)'\""

argh.....sorry, i'm dense.....thanks again....

rc

Re: [Refdb-devel] parentheses in keywords

[Markus H. <mar...@mh...>]

Markus Hoenicka writes:
 > In any case, I feel the missing closing paren is a bug in the query
 > parser. It should work with less quoting. I'll investigate this.
 > 

Ok, here goes. I've changed the query language a little in order to
let the user decide whether or not to use regular expressions. So far
RefDB used regexps in basically all string matches. I've now changed
the meaning of "=" in the query language to "literal match", whereas
the new operator "~" means "regexp match".

Second, I've fiddled a little with the quoting stuff. refdbc now makes
sure to keep the quotes of quoted arguments like :KW:='what
ever'. This makes quoting on the command line a little easier.

As a result, all of the following should now work as expected:

refdbc: getref :KW:='52-67-5 (Penicillamine)'

refdbc: getref :KW:~'52-67-5 \(Penicillamine\)'

refdbc -d alltypes -C  getref ":KW:='52-67-5 (Penicillamine)'"

refdbc -d alltypes -C  getref ":KW:~'52-67-5 \(Penicillamine\)'"

If you don't want to use the CVS version, hang on for a day or
two. I'm planning to release the next prerelease fairly soon.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-devel] parentheses in keywords

[Markus H. <mar...@mh...>]

rob caSSon writes:
 > okay...i think i'm hitting something different now, and may have tracked
 > down what is the real problem; when i execute the above query from the
 > refdbc shell, i get the expected results, but when i do this from bash:
 > 
 >     refdbc -C getref ":KW:='52-67-5 \(Penicillamine\)'"
 > 

Ah, this is a different story. Bash eats the double quotes so
refdbc will never see them. This one works, though:

refdbc -C getref "\":KW:='52-67-5 \(Penicillamine\)'\""

In any case, I feel the missing closing paren is a bug in the query
parser. It should work with less quoting. I'll investigate this.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-devel] parentheses in keywords

[rob c. <ro...@st...>]

> I had a second look at this issue. The difference between sqlite and
> the other database servers is that the former does not use Unix
> regexps but the simpler SQL regexps instead. Parentheses do not have a
> special meaning in SQL regexps, this is why it worked for me.

yup...certainly makes sense...

> MySQL and PostgreSQL use Unix regexps in queries like getref
> :KW:=whatever. This means that you have to escape any characters with
> a special meaning if you want them matched literally. The following
> query works for me using MySQL or PostgreSQL as a backend:
> 
> getref ":KW:='52-67-5 \(Penicillamine\)'"
> 
> Now the question is whether RefDB should offer the choice to do a
> literal match instead of a regexp match.

okay...i think i'm hitting something different now, and may have tracked
down what is the real problem; when i execute the above query from the
refdbc shell, i get the expected results, but when i do this from bash:

    refdbc -C getref ":KW:='52-67-5 \(Penicillamine\)'"

i get the "subselect failed" error, and the log shows the cut off
parentheses again...

7:pid=26684:Thu Jan 29 22:14:35 2004:SELECT DISTINCT t_xkeyword.xref_id FROM t_xkeyword, t_keyword WHERE t_xkeyword.xkeyword_type='REFERENCE' AND t_keyword.keyword_id=t_xkeyword.keyword_id AND t_keyword.keyword_name RLIKE '52-67-5 \\(Penicillamine\\'

if you want more logs, i'd be happy to oblige....

thanks again for all your work,
rc

btw: just added a primitive upload facility to the web front end....must
upload RIS data, but again, as a proof-of-concept, it seems to
work...feel free to upload stuff:

    http://junk.lib.muohio.edu/refdb/upload.php

Re: [Refdb-devel] parentheses in keywords

[Markus H. <mar...@mh...>]

Hi Rob,

I had a second look at this issue. The difference between sqlite and
the other database servers is that the former does not use Unix
regexps but the simpler SQL regexps instead. Parentheses do not have a
special meaning in SQL regexps, this is why it worked for me.

MySQL and PostgreSQL use Unix regexps in queries like getref
:KW:=whatever. This means that you have to escape any characters with
a special meaning if you want them matched literally. The following
query works for me using MySQL or PostgreSQL as a backend:

getref ":KW:='52-67-5 \(Penicillamine\)'"

Now the question is whether RefDB should offer the choice to do a
literal match instead of a regexp match.

regards,
Markus

rob caSSon writes:
 > > getref ":KW:='10102-43-9 (Nitric Oxide)'"
 > > 
 > > works like a charm. The trick is to employ the double quotes to hide
 > > the single quotes from the query parser. This way they'll be left
 > > alone until the SQL tokenizer works on the actual query.
 > 
 > i bet this works with sqlite...i'll have to give it a try, but the RLIKE
 > still wants the parentheses quoted in mysql according to the logs...
 > 
 > i'll try to test postgresql later tonight, if i get the chance...
 > 
 > staying db-agnostic is often a pain, but i'm glad you're doing it....
 > 
 > 

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-devel] parentheses in keywords

[rob c. <ro...@st...>]

> getref ":KW:='10102-43-9 (Nitric Oxide)'"
> 
> works like a charm. The trick is to employ the double quotes to hide
> the single quotes from the query parser. This way they'll be left
> alone until the SQL tokenizer works on the actual query.

i bet this works with sqlite...i'll have to give it a try, but the RLIKE
still wants the parentheses quoted in mysql according to the logs...

i'll try to test postgresql later tonight, if i get the chance...

staying db-agnostic is often a pain, but i'm glad you're doing it....

Re: [Refdb-devel] parentheses in keywords

[rob c. <ro...@st...>]

> Looks like it is a lot easier:
> 
> getref ":KW:='10102-43-9 (Nitric Oxide)'"

ahhh....i should've tried that...thanks again...

rob

[Refdb-devel] parentheses in keywords

[Markus H. <mar...@mh...>]

Markus Hoenicka writes:
 > I'd rather suspect it's my fault. I can reproduce it over here, but I
 > think that the problem is unrelated to SQL parser problems. I guess it
 > is my query parser that screws up things. As a temporary workaround,
 > something like (using SQLite):
 > 
 >   getref -t risx :KW:="10102-43-9 (Nitric Oxide%"
 > 
 > should do the trick. I'll try to get back with a fix asap.
 > 

Looks like it is a lot easier:

getref ":KW:='10102-43-9 (Nitric Oxide)'"

works like a charm. The trick is to employ the double quotes to hide
the single quotes from the query parser. This way they'll be left
alone until the SQL tokenizer works on the actual query.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-devel] parentheses in keywords

[Markus H. <mar...@mh...>]

rob caSSon writes:
 > i'm playing with a web front end to refdb-0.9.4-pre5, and may have
 > stumbled on a small bug...
 > 
 > using the test data, searching on something like "10102-43-9 (Nitric
 > Oxide)" fails with "subselect failed"
 > 
 > i first tried escaping them with slashes for the RLIKE, as this works
 > for [], but to no avail..
 > 
 > from the looks of the log, the final parentheses is getting cut off:
 > 
 > getref -t risx :KW:="10102-43-9 (Nitric Oxide)"
 > 
 > from the log:
 > 
 > SELECT DISTINCT t_xkeyword.xref_id FROM t_xkeyword, t_keyword
 >     WHERE t_xkeyword.xkeyword_type='REFERENCE'
 >         AND t_keyword.keyword_id=t_xkeyword.keyword_id
 >         AND t_keyword.keyword_name RLIKE '10102-43-9 (Nitric Oxide'
 > 
 > aside from the missing parentheses, it just needs the necessary escaping
 > for the RLIKE:
 > 
 > SELECT DISTINCT t_xkeyword.xref_id FROM t_xkeyword, t_keyword
 >     WHERE t_xkeyword.xkeyword_type='REFERENCE'
 >         AND t_keyword.keyword_id=t_xkeyword.keyword_id
 >         AND t_keyword.keyword_name RLIKE '10102-43-9 \\(Nitric Oxide\\)'
 > 
 > hope this isn't a dupe, or just plain stupidity on my part ;)
 > 

I'd rather suspect it's my fault. I can reproduce it over here, but I
think that the problem is unrelated to SQL parser problems. I guess it
is my query parser that screws up things. As a temporary workaround,
something like (using SQLite):

  getref -t risx :KW:="10102-43-9 (Nitric Oxide%"

should do the trick. I'll try to get back with a fix asap.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-devel] parentheses in keywords

[rob c. <ro...@st...>]

i'm playing with a web front end to refdb-0.9.4-pre5, and may have
stumbled on a small bug...

using the test data, searching on something like "10102-43-9 (Nitric
Oxide)" fails with "subselect failed"

i first tried escaping them with slashes for the RLIKE, as this works
for [], but to no avail..

from the looks of the log, the final parentheses is getting cut off:

getref -t risx :KW:="10102-43-9 (Nitric Oxide)"

from the log:

SELECT DISTINCT t_xkeyword.xref_id FROM t_xkeyword, t_keyword
    WHERE t_xkeyword.xkeyword_type='REFERENCE'
        AND t_keyword.keyword_id=t_xkeyword.keyword_id
        AND t_keyword.keyword_name RLIKE '10102-43-9 (Nitric Oxide'

aside from the missing parentheses, it just needs the necessary escaping
for the RLIKE:

SELECT DISTINCT t_xkeyword.xref_id FROM t_xkeyword, t_keyword
    WHERE t_xkeyword.xkeyword_type='REFERENCE'
        AND t_keyword.keyword_id=t_xkeyword.keyword_id
        AND t_keyword.keyword_name RLIKE '10102-43-9 \\(Nitric Oxide\\)'

hope this isn't a dupe, or just plain stupidity on my part ;)

rob

Re: [Refdb-devel] Anonymous CVS checkout

[Markus H. <mar...@mh...>]

Bruce D'Arcus writes:
 > Hmm, here's what I get:
 > 
 > -rw-------  1 root  staff  173 14 Nov 14:19 /Users/darcusb/.cvspass
 > 
 > Isn't that wrong?
 > 

Oh how stupid. I should have thought a second about why you use sudo
to set the permissions in the first place. You don't own that file so
cvs rightfully refuses to use it for authentication.

Do a

sudo chown darcusb:darcusb /Users/darcusb/.cvspass

assuming that your Unix username is darcusb and your primary group is
darcusb as well. You should then see something like:

-rw-------  1 darcusb  darcusb  173 14 Nov 14:19 /Users/darcusb/.cvspass

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-devel] Anonymous CVS checkout

[Bruce D'A. <bd...@fa...>]

On Dec 25, 2003, at 6:22 AM, Michael Smith wrote:

>> I still get the permission error.  Any other ideas?
>
> Can you do an "ls -l =E2=80=BE/.cvspass" and double-check to make sure =
the=20
> perms
> are actually getting set?

Hmm, here's what I get:

-rw-------  1 root  staff  173 14 Nov 14:19 /Users/darcusb/.cvspass

Isn't that wrong?

Bruce

Re: [Refdb-devel] refdb-mode release?

[Markus H. <mar...@mh...>]

Hi Mike,

Michael Smith writes:
 > But I think it can be released initially even without those features.

Sounds good.

 > 
 > > You could add a chapter to the manual if you wish.
 > 
 > OK. I'd like to do that, but not sure I can get it done within the next
 > couple of days.
 > 

That's fine. The documentation in the elisp file should be sufficient
for the time being.

 > OK. I think I can do that. I've not actually ever written an
 > autoconf-based installer -- just done it using manually created
 > makefiles. But I think I can probably find some examples of other Emacs
 > packages that have autoconf-based installers, and just use those as
 > examples.
 > 

Look at calc or w3. These projects are way more sophisticated but
still may give a clue. Basically we just need to find out which Emacs
versions the stuff should be compiled for and into which directory it
should be installed. I also see this as a nice-to-have feature. For
the time being we could just ship the elisp files as they are.

regards,
Markus


-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-devel] refdb-mode release?

[Michael S. <sm...@xm...>]

Markus,

You wrote:

> Mike,
> 
> I'm planning to release a new version of RefDB in the next couple of
> days, mainly to fix the bug with disappearing datasets after updating
> the personal info. I was wondering whether you think refdb-mode is
> stable and feature-complete enough to release it as well.

It's definitely stable enough, and though I wouldn't describe it as
feature-complete, I think it has enough features to be be useful.
Currently, it lets you:

  - add references
  - delete references
  - get references
  - select output type (screen, RIS, etc.)
  - select output format (IDs only, all fields, etc.)
  - select among multiple RefDB databases

In the short term, I'd like to add capability to:

  - update references (via updateref and pickref)
  - delete personal references via pickref

But I think it can be released initially even without those features.

> You could add a chapter to the manual if you wish.

OK. I'd like to do that, but not sure I can get it done within the next
couple of days.

> I'd like to move ris.el over to the elisp directory as well, to keep
> all Emacs-related code in one package.

OK. I'll move it in CVS.

> We should also write a small autoconf-based installer that copies the
> files into site-lisp, byte-compiles them, and maybe appends the
> necessary code to load the modes to site-start.el or to 

OK. I think I can do that. I've not actually ever written an
autoconf-based installer -- just done it using manually created
makefiles. But I think I can probably find some examples of other Emacs
packages that have autoconf-based installers, and just use those as
examples.

  --Mike

[Refdb-devel] refdb-mode release?

[Markus H. <mar...@mh...>]

Mike,

I'm planning to release a new version of RefDB in the next couple of
days, mainly to fix the bug with disappearing datasets after updating
the personal info. I was wondering whether you think refdb-mode is
stable and feature-complete enough to release it as well. You could
add a chapter to the manual if you wish.

I'd like to move ris.el over to the elisp directory as well, to keep
all Emacs-related code in one package. We should also write a small
autoconf-based installer that copies the files into site-lisp,
byte-compiles them, and maybe appends the necessary code to load the
modes to site-start.el or to ~/.emacs. There's quite a few elisp
projects using autoconf-based installers that we could steal the code
from.

Let me know what you think.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-devel] xlink?

[Michael S. <sm...@xm...>]

Hi Bruce,

You wrote:

> Mike,
> 
> Do you know much about xlink?

Not much, actually

> The following example was posted to the MODS list, which strikes me as
> quite interesting. This suggests it ought to be possible to link chapters
> to books and articles to journals as well...
> 
> I'm unclear on how an XSLT processor handles these links though, as my
> initial experiments (with an external file containing just journal names)
> didn't work. 

I don't think XSLT processors are designed to do anything special with
xlink attributes. I think they see them as being just like any other
attribute. To do anything with document instances that contain xlinks, I
think you need to use stylesheets that are set up to process them.

I'm not sure what exactly you mean about linking chapters
to books and articles to journals, but if you're talking about
cross-document links, I don't think xlink is going to do any magic as
far as that goes. Even with your stylesheets set up to process xlinks,
if you want to link to something that's in a different physical file,
you'll either need to just use an absolute URI to it, or use something
like the DocBook olink mechanism that Bob Stayton added support for to
the DocBook XSLT stylesheets.

  --Mike

Re: [Refdb-devel] refdb-mode updated (v1.5)

[Michael S. <sm...@xm...>]

Markus Hoenicka <mar...@mh...> writes:

> Michael Smith writes:
>  > Yeah, I'll set it up to automatically put the refdb-output buffer into
>  > refdb-mode. The only downside to that is, the way I currently have
>  > things set up, I think that will mean that every time you run a query
>  > and it returns output, Emacs will re-initialize the database list and
>  > rebuild the RefDB menu, which (on my machine at least) takes a couple of
>  > seconds to do.
>  > 
> 
> Can't refdb-mode set a global the first time it initializes the
> database list and skip rebuilding the menu in subsequent calls?

Yeah, it can't. I thought about this last night and realized it'd be
simple to do. I include the change next time I check in.

  --Mike

Re: [Refdb-devel] refdb-mode updated (v1.5)

[Markus H. <mar...@mh...>]

Michael Smith writes:
 > Yeah, I'll set it up to automatically put the refdb-output buffer into
 > refdb-mode. The only downside to that is, the way I currently have
 > things set up, I think that will mean that every time you run a query
 > and it returns output, Emacs will re-initialize the database list and
 > rebuild the RefDB menu, which (on my machine at least) takes a couple of
 > seconds to do.
 > 

Can't refdb-mode set a global the first time it initializes the
database list and skip rebuilding the menu in subsequent calls?

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-devel] xlink?

[Bruce D'A. <bd...@fa...>]

Mike,

Do you know much about xlink?  

The following example was posted to the MODS list, which strikes me as
quite interesting. This suggests it ought to be possible to link chapters
to books and articles to journals as well...

I'm unclear on how an XSLT processor handles these links though, as my
initial experiments (with an external file containing just journal names)
didn't work. 

Bruce

<mods>
   <titleInfo>
      <title>Names are a pain in the butt</title>
   </titleInfo>
   <name xlink:href="#name.Robert.Smith">
      <role><roleTerm type="code">aut</roleTerm></role>
      <displayForm>Bob Smith</displayForm>
   </name>
</mods>
<!-- Somewhere else in the file: -->
<mods ID="name.Robert.Smith">
   <name type="personal">
      <namePart type="family">Smith</namePart>
      <namePart type="given">Robert</namePart>
   </name>
   <note type="biographical">...</note>
   <!-- and so forth -->
</mods>