refdb-users — List for RefDB users

[Refdb-users] TEI bibl, style data

[Markus H. <mar...@mh...>]

Hi,

Ralf Stephan writes:
 > Hello,
 > could you please answer two questions, first: can RefDB read loose
 > TEI bibliographies, i.e., those in text using <bibl>...</bibl>?
 > The runbib man page is unclear on it.
 > 

You'll find a better description of the requirements for processing
documents with RefDB in the manual
(http://refdb.sourceforge.net/doc.html), see especially chapter
10. RefDB will not process any <bibl> elements in the text.

 > Second, what could be the cause of the command 
 > 'runbib -d mybib file.tei' failing with
 >    ... lots of xml errors ...
 > failed to access style data
 > mv: cannot stat `.dsl': No such file or directory
 > The tei file is verified PGTEI, an extension to P4.
 > 
 > I have given the path to the P4/P5 style sheets to configure,
 > and they were found. refdb-init went smoothly except 
 > Loading /usr/local/share/refdb/styles/schemas.xml
 > server error. Stop.
 > 
 > (That file exists and is readable)

Thanks for reporting this. refdb-init should no attempt to load
schemas.xml which is just a helper for nxml-mode. I'll see to it to
avoid this problem in the future.

As for your document processing error, this is due to some missing
runbib options. Your error messages indicate that runbib erroneously
attempted to process file.tei as a SGML file. You'll have to use at
least the -S option to specify a bibliography style and the -t option
to tell runbib which type of bibliography you want.

Instead of doing all this manually, I'd suggest to have a look at
refdbnd. Run this script in the directory which contains
file.tei. When the script asks for the basename, you can pass the full
name of your file. If all goes well, refdbnd will use appropriate
values to process this document correctly. You can then just type
"make pdf" or "make html" to process your document.

Please do not hesitate to ask for further assistance if this doesn't
pan out as expected.

regards,
Markus


-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-users] TEI bibl, style data

[Ralf S. <ra...@ar...>]

Hello,
could you please answer two questions, first: can RefDB read loose
TEI bibliographies, i.e., those in text using <bibl>...</bibl>?
The runbib man page is unclear on it.

Second, what could be the cause of the command 
'runbib -d mybib file.tei' failing with
   ... lots of xml errors ...
failed to access style data
mv: cannot stat `.dsl': No such file or directory
The tei file is verified PGTEI, an extension to P4.

I have given the path to the P4/P5 style sheets to configure,
and they were found. refdb-init went smoothly except 
Loading /usr/local/share/refdb/styles/schemas.xml
server error. Stop.

(That file exists and is readable)
I'm using sqlite3 if it matters.

Regards,
ralf

[Refdb-users] ANN: refdb-publist-1.1 available

[Markus H. <mar...@mh...>]

Hi,

refdb-publist is essentially a brushed-up refdbnd project which allows
you to generate publication lists dynamically from the data stored in
your RefDB database. Be it the publication list in your CV, or an
annual department report, maintenance is simplified to running
e.g. "make pdf" whenever you need an updated version. A real-world
example can be viewed here:

http://www.mhoenicka.de/mhoenicka-publist.pdf

I've updated the refdb-publist package to work with RefDB 0.9.9. In
addition to the changes mandated by the new RefDB version, I've
rearranged the package a bit to make things easier to understand.

The package is available on the SF download page, see

http://sourceforge.net/projects/refdb

The documentation was updated and is available online now, see

http://refdb.sourceforge.net/doc.html#refdb-publist

Feel free to send comments, suggestions, and bug reports to this list.

regards,
Markus


-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-users] Possible data corruption in MySQL

[Markus H. <mar...@mh...>]

Hi all,

I'd like to make you aware of a problem with the libdbi MySQL driver =20
which you are using if you run MySQL as your database engine for =20
RefDB. There is a bug which may screw up your data under the following =20
circumstances (other scenarios are possible though):

1) Your database uses UTF-8 encoding

2) You store multibyte characters (like UTF-8 umlauts)

Now what happens is that the MySQL driver incorrectly causes the data =20
to be transformed to "UTF-8" again, making umlauts take up 4 instead =20
of 2 bytes. If you retrieve your data with RefDB again, the problem =20
may be silent as the data are transformed back again. Assume your =20
dataset with the ID 1 has an author called "B=E4uml". Using the current =20
MySQL driver the author name will be stored with the umlaut "=E4" =20
encoded as 4 byte.

1) if you retrieve the dataset using "getref :ID:=3D1", the query will =20
succeed, and the author will be returned as "B=E4uml" with the "=E4" =20
taking up 2 byte.

2) if you attempt to retrieve the dataset using "getref :AX:~B=E4uml", =20
the query will fail as the "=E4" in your query string uses 2 bytes =20
whereas the author name in the database uses 4 bytes for the same =20
character.

A fixed version of the MySQL driver will be available shortly. It is =20
important to understand that you have to export your data using the =20
current version of the MySQL driver and reimport the data using the =20
new driver if you upgrade your libdbi drivers.

Please do not hesitate to seek assistance if you attempt to upgrade =20
your installation.

regards,
Markus


--=20
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-users] ANN: updated PHP web interface for 0.9.9

[Markus H. <mar...@mh...>]

Hi,

in an attempt to make the progress in the PHP web interface available
to users independently of major releases, I've compiled a small
archive with the current svn versions of the files. The archive (37kB)
is available for download here:

http://refdb.sourceforge.net/pre/refdb-phpweb-0.9.9-2.tar.gz

The tarball contains instructions how to "install" the files properly.

The new features of this prerelease are:

- the output of reference and index searches is now chunked. You can
  page through the output with "previous" and "next" buttons as
  appropriate. By default, 10 references and 50 index entries are
  displayed

- the number of reference entries and of index entries per page can
  now be set in the preferences tab

Your continued feedback is more than welcome.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-users] style gallery

[Bruce D'A. <bda...@gm...>]

On Nov 26, 2007 3:55 AM, Markus Hoenicka <mar...@mh...> wrote:

...

> Don't worry about stepping on toes. If someone else does something
> better, it is always good to know. However, this one-click
> installation thing may be appropriate for a GUI program or for a
> browser-add-on like Zotero. I haven't seen that many Unix command line
> applications that support one-click installations of add-ons. I guess
> it is somehow contrary to the philosophy of these tools. But that
> doesn't mean we couldn't add some convenience to the style
> installation. Right now you just need to run "refdba -C addstyle
> <file>" after downloading the style which isn't that hard either. But
> we could add a little script which detects new styles in the directory
> and adds them automatically. This way, you'd just have to drop them in
> the appropriate directory.

FWIW, the way Zotero is doing it is, when you click on the link, it
results in the server serving up the CSL file with a "text/x-csl"
content type. E.g.:

$  curl -I http://www.zotero.org/styles/mhra/dev/install
HTTP/1.1 200 OK
Date: Sun, 25 Nov 2007 17:35:47 GMT
Server: Apache/2.2.3 (CentOS)
X-Powered-By: PHP/5.2.4
Connection: close
Content-Type: text/x-csl

So Zotero then sets itself to handle that type.

I liike this approach, since it's both simple and general (since other
tools could use it).

Not sure how easy that is to configure outside of the web browser, but
I presume it's doable.

Bruce

Re: [Refdb-users] style gallery

[Markus H. <mar...@mh...>]

Quoting Bruce D'Arcus <bda...@gm...>:

> On a related note, the Zotero guys have put up a CSL style repository,
> which allows one-click installation of the styles (with latest dev
> builds).
>
> <http://www.zotero.org/styles>
>
> Don't mean to step on toes; just thought it might be interesting for =20
>  people ...

Don't worry about stepping on toes. If someone else does something =20
better, it is always good to know. However, this one-click =20
installation thing may be appropriate for a GUI program or for a =20
browser-add-on like Zotero. I haven't seen that many Unix command line =20
applications that support one-click installations of add-ons. I guess =20
it is somehow contrary to the philosophy of these tools. But that =20
doesn't mean we couldn't add some convenience to the style =20
installation. Right now you just need to run "refdba -C addstyle =20
<file>" after downloading the style which isn't that hard either. But =20
we could add a little script which detects new styles in the directory =20
and adds them automatically. This way, you'd just have to drop them in =20
the appropriate directory.

regards,
Markus






--=20
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-users] style gallery

[Bruce D'A. <bda...@gm...>]

On Nov 21, 2007 8:06 PM, Markus Hoenicka <mar...@mh...> wrote:

...

> The available styles are now listed on the homepage, along with the
> html outputs of a test document using these styles:
>
> http://refdb.sourceforge.net/styles.html

On a related note, the Zotero guys have put up a CSL style repository,
which allows one-click installation of the styles (with latest dev
builds).

<http://www.zotero.org/styles>

Don't mean to step on toes; just thought it might be interesting for people ...

Bruce

[Refdb-users] style gallery

[Markus H. <mar...@mh...>]

Hi all,

I've finally found some time to implement a few of the standard
bibliography styles like Chicago and Harvard. These styles are not yet
complete, but they cover the most common reference types. I've noticed
a few minor bugs in refdbd while optimizing these styles. All of these
are fixed in the current svn revision. I've also noticed a few
shortcomings of the citestylex.dtd. These will be fixed in a future
release to get rid of the nasty hacks I had to resort to. Anyway, you
can get pretty far with the current svn version which I intend to
release fairly soon as a prerelease.

The available styles are now listed on the homepage, along with the
html outputs of a test document using these styles:

http://refdb.sourceforge.net/styles.html

Feel free to play with these styles and report any errors or
shortcomings. Also, if you have customized any styles or written any
from scratch, please do not hesitate to send them in to get them
listed on the web page as well.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-users] ANN: refdb-elisp-1.5 available

[Markus H. <mar...@mh...>]

Hi all,

I've uploaded a new version of the refdb-elisp package, which turns
Emacs into a RefDB frontend and then some. refdb-mode is a minor mode
which works with PSGML, nxml-mode, AucTeX, and Muse. You can look up
entries in your database, add and edit entries, retrieve them in a
variety of formats, cite references in your DocBook, TEI, LaTeX, or
Muse documents, and do reverse lookups with a mouseclick. You can even
create, transform, and view documents using RefDB bibliographies.

The new version 1.5 was updated to work with the latest RefDB release
0.9.9 and is available on the RefDB project page:

http://sourceforge.net/project/showfiles.php?group_id=26091&package_id=169552

Please send bugs reports and suggestions to this list.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-users] file:// and refdcrc

[Daniel O'D. <dan...@ul...>]

If the directory is a subdirectory of the one apache is pointing at for
refdb, you seem to be able to get away with relative paths in L1. If
apache is pointing at /usr/share/refdb/www for example, and you store
your pdfs in /usr/share/refdb/www/bibliography, you can enter
bibliography/AUTHOR.PDF in L1 and how it resolve fine in the php
interface.

-d

On Thu, 2007-11-08 at 00:28 +0100, Markus Hoenicka wrote:
> Dan O'Donnell writes:
>  > Hi have a question about the PDF root file business. If I want to put my
>  > local offprints in /var/www/research/bibliography/ and if my refdb setup
>  > is at http://refdb.example.com/ (and on the machine
>  > in /user/share/refdb/) I presumably change my refdbrc to
>  > have /var/www/research/bibliography/ as pdfroot.
>  > 
>  > 1) Does the substitution of file:// with PDF root take place during
>  > output (I'm assuming it does)?
>  > 
> 
> That's right. The URLs are stored as provided, but they are expanded
> when they're exported to a display format. They're not expanded when
> exported to an export format (RIS or risx).
> 
>  > 2) if I am using the PHP interface, how will the rleative path work?
>  > I.e. if the PDF root is not a subdirectory of the path I put in my
>  > apache configuration file, will it still work? or do I need to make sure
>  > that the path is a child of the main refdb directory?
>  > 
> 
> I'm afraid your repository would have to be in a subdirectory that
> Apache is allowed to access. This is not necessarily the folder of the
> RefDB web interface. You could allow Apache to read pretty much any
> directory. Remember that by default each user's public_html folder is
> readable by Apache too. I have to admit that I've never explored the
> pdfroot stuff in conjunction with the PHP interface, so there may be
> some dark corners left.
>  
>  > 3) Once I change the PDF root, is restarting refdbdctl enough for the
>  > change to take effect?
>  > 
> 
> Actually, changing the pdfroot setting itself should suffice because
> it is a client-side configuration. The clients are started in batch
> mode anyway by the PHP interface, so they will pick up the new
> configuration the next time they are run. If you use refdbc in
> interactive mode from the command line, you'd have to restart that in
> order to let the new setting take effect (or use the "set pdfroot
> <newval>" command).
> 
> regards,
> Markus
> 
-- 
Daniel Paul O'Donnell, PhD
Associate Professor and Chair, Department of English
Director, Digital Medievalist Project http://www.digitalmedievalist.org/
Chair and CEO, Text Encoding Initiative http://www.tei-c.org/
University of Lethbridge
Lethbridge AB T1K 3M4
Canada
Vox: +1 403 329-2378
Fax: +1 403 382-7191

[Refdb-users] file:// and refdcrc

[Markus H. <mar...@mh...>]

Dan O'Donnell writes:
 > Hi have a question about the PDF root file business. If I want to put my
 > local offprints in /var/www/research/bibliography/ and if my refdb setup
 > is at http://refdb.example.com/ (and on the machine
 > in /user/share/refdb/) I presumably change my refdbrc to
 > have /var/www/research/bibliography/ as pdfroot.
 > 
 > 1) Does the substitution of file:// with PDF root take place during
 > output (I'm assuming it does)?
 > 

That's right. The URLs are stored as provided, but they are expanded
when they're exported to a display format. They're not expanded when
exported to an export format (RIS or risx).

 > 2) if I am using the PHP interface, how will the rleative path work?
 > I.e. if the PDF root is not a subdirectory of the path I put in my
 > apache configuration file, will it still work? or do I need to make sure
 > that the path is a child of the main refdb directory?
 > 

I'm afraid your repository would have to be in a subdirectory that
Apache is allowed to access. This is not necessarily the folder of the
RefDB web interface. You could allow Apache to read pretty much any
directory. Remember that by default each user's public_html folder is
readable by Apache too. I have to admit that I've never explored the
pdfroot stuff in conjunction with the PHP interface, so there may be
some dark corners left.
 
 > 3) Once I change the PDF root, is restarting refdbdctl enough for the
 > change to take effect?
 > 

Actually, changing the pdfroot setting itself should suffice because
it is a client-side configuration. The clients are started in batch
mode anyway by the PHP interface, so they will pick up the new
configuration the next time they are run. If you use refdbc in
interactive mode from the command line, you'd have to restart that in
order to let the new setting take effect (or use the "set pdfroot
<newval>" command).

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-users] file:// and refdcrc

[Dan O'D. <dan...@ul...>]

Hi have a question about the PDF root file business. If I want to put my
local offprints in /var/www/research/bibliography/ and if my refdb setup
is at http://refdb.example.com/ (and on the machine
in /user/share/refdb/) I presumably change my refdbrc to
have /var/www/research/bibliography/ as pdfroot.

1) Does the substitution of file:// with PDF root take place during
output (I'm assuming it does)?

2) if I am using the PHP interface, how will the rleative path work?
I.e. if the PDF root is not a subdirectory of the path I put in my
apache configuration file, will it still work? or do I need to make sure
that the path is a child of the main refdb directory?

3) Once I change the PDF root, is restarting refdbdctl enough for the
change to take effect?

-dan
-- 
Daniel Paul O'Donnell, PhD
Chair, Text Encoding Initiative <http://www.tei-c.org/>
Director, Digital Medievalist Project <http://www.digitalmedievalist.org/>
Associate Professor and Chair of English
University of Lethbridge
Lethbridge AB T1K 3M4
Vox: +1 403 329 2378
Fax: +1 403 382-7191
Homepage: http://people.uleth.ca/~daniel.odonnell/

[Refdb-users] ANN: refdb-0.9.9-1 available

[Markus H. <mar...@mh...>]

Hi,

I had to fix two bugs which render the PHP web interface
useless. First, one of the essential PHP files was not installed
during make install. Second, the manual used incorrect directories in
its instructions for modifying httpd.conf. I've uploaded a new release
0.9.9-1 to the sourceforge repository at:

https://sourceforge.net/project/showfiles.php?group_id=26091

If you have already downloaded 0.9.9, it is reasonably simple to fix
these problems without downloading and building RefDB again. These two
steps are all it takes:

- copy phpweb/includes/refdblib.inc.php to
  /usr/local/share/refdb/www/includes/ (use /usr/share... if you used
  "/usr" as a prefix during configure)

- in the instructions how to modify httpd.conf in order to run the php
  web interface, read "www" instead of "phpweb" in the directories. A
  fixed version of the instructions is also available online, see
  http://refdb.sourceforge.net/manual/ch04s11.html

I apologize for any inconveniences.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-users] Build problems

[Dan O'D. <dan...@ul...>]

On Wed, 2007-07-11 at 09:17 +0100, Markus Hoenicka wrote:
> Hi Dan,
> 
> Quoting Dan O'Donnell <dan...@ul...>:

> > 2) phpweb doesn't seem to be building: when I complete the build and run
> > refdb-init, I end up with a directory /www. This has includes. but not
> > refdblib.inc.php which is called by index.php. So I get a php error.
> >
> 
> This is pretty embarrassing. I forgot to add refdblib.inc.php to the  
> relevant Makefile.am. Now everyone out there trying to test the web  
> interface thinks it's broken anyway. I'll upload a fixed archive later  
> today.

The instructions need to be updated to, since the instructions on
installing the phpo tell you to point at */phpweb not */www

-d

> 
> regards,
> Markus
> 
-- 
Daniel Paul O'Donnell, PhD
Chair, Text Encoding Initiative <http://www.tei-c.org/>
Director, Digital Medievalist Project <http://www.digitalmedievalist.org/>
Associate Professor and Chair of English
University of Lethbridge
Lethbridge AB T1K 3M4
Vox: +1 403 329 2378
Fax: +1 403 382-7191
Homepage: http://people.uleth.ca/~daniel.odonnell/

Re: [Refdb-users] Build problems

[Markus H. <mar...@mh...>]

Hi Dan,

Quoting Dan O'Donnell <dan...@ul...>:

> 1) config --prefix=3D doesn't seem to be working exactly the way I thought
> it would: if I set it to /usr/share I seem to end up
> with /usr/share/share/refdb; setting it to /usr gets me /usr/share/refdb
>

The autotools handle all of the etc, share, lib, and other stuff =20
themselves. You just indicate the top-level directory where all this =20
stuff goes. This is usually one of /usr or /usr/local, rarely /opt or =20
a folder under your home directory. I don't know whether ubuntu has =20
strict rules for software not installed from packages, but /usr/local =20
(the default) may be the better choice. But then I'd use =20
--sysconfdir=3D/etc. Debian (and ubuntu probably too) allows to back up =20
the configuration of the whole system by tarring /etc.

> 2) phpweb doesn't seem to be building: when I complete the build and run
> refdb-init, I end up with a directory /www. This has includes. but not
> refdblib.inc.php which is called by index.php. So I get a php error.
>

This is pretty embarrassing. I forgot to add refdblib.inc.php to the =20
relevant Makefile.am. Now everyone out there trying to test the web =20
interface thinks it's broken anyway. I'll upload a fixed archive later =20
today.

regards,
Markus

--=20
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-users] RefDB-0.9.9: AUTHORS file out of date

[Markus H. <mar...@mh...>]

Markus Hoenicka writes:
 > Hi all,
 > 
 > I knew that with such an ambitious update something just had to go  
 > wrong. Much to my regret I found out today that the AUTHORS file does  
 > not yet honor the work of Paul O'Donnell and Dominic Reusser on the  
 > web interface. This will be fixed in upcoming releases. Paul, Dominic,  
 > please accept my apologies for this unintentional omission.
 > 

time to get some sleep - Paul O'Donnell is of course Dan O'Donnell,
not to be confused with Paul Hoadley who is also listed in the AUTHORS
file. Sorry about that. I'll shut up now.

Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-users] RefDB-0.9.9: AUTHORS file out of date

[Markus H. <mar...@mh...>]

Hi all,

I knew that with such an ambitious update something just had to go  
wrong. Much to my regret I found out today that the AUTHORS file does  
not yet honor the work of Paul O'Donnell and Dominic Reusser on the  
web interface. This will be fixed in upcoming releases. Paul, Dominic,  
please accept my apologies for this unintentional omission.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-users] ANN: refdb-0.9.9 released

[Markus H. <mar...@mh...>]

Hi all,

today I've uploaded the RefDB version 0.9.9 sources, along with
updated or new Perl modules required by this release. The
documentation and DTD files available on the web page were updated as
well, and the web page itself also received some polish. Please visit:

http://refdb.sourceforge.net

for general information, and 

http://sourceforge.net/project/showfiles.php?group_id=26091

to download the files.

The changes compared to the previous release are quite substantial and
are probably the hugest between any releases so far. Besides a
boatload of internal changes that simplify future developments, there
are a couple of new or improved features. There is a much nicer and
easier-to-use PHP web interface written from scratch, MODS support,
SRU support, namespace support in all XML formats, raw bibliographies,
and support for DocBook V5.0 and TEI P5, to name a few. Please see the
detailed list of changes below for further information.

I recommend to consult the UPGRADING file before installing the new
release. Both your reference data and custom bibliography styles may
require some manual fixes to work best with the new version.

As there were *so* many changes and new features, I'd be surprised if
there was not a single bug. Please report problems, suggestions and
rants to this list.

regards,
Markus


Bugfixes:

- raw bibliographies were fixed again

- a bug in the tokenizers used by refdbd and by the clients to parse
  command line arguments has been fixed. Now escaped characters should
  be treated as such in all cases.

- fixed the missing author type in part authors imported from risx

- fixed the style export on OS X (PPC)

- fixed the publication year output in RIS export if the otherinfo
  field is empty

- crash during bibliography creation on OS X was fixed

- unknown reference data types are now changed to "GEN" when importing
  risx data as well

- abbreviated journal names which use periods without spaces are now
  imported properly

- config.rpath was added to the tarball to avoid configure errors on
  particular OSes

- upgrading the main database no longer creates duplicate entries in
  t_meta when using PostgreSQL as the database engine

- fixed missing permissions after upgrading the main PostgreSQL database

- the preconfigured locations of log and pid files now honor the
  prefix (usually /usr or /usr/local)

- fixed a table locking error if a risx dataset is skipped during
  import

- special characters are now properly converted to XML entities even
  in long texts with lots of these characters

- now purges "(", ")", and "?" from citation keys

- now checks the reference types before adding them to the
  database. Unknown types are changed to "GEN".

- a sanity check in the RIS import code prevents segfaults if risx
  data are accidentally fed to it.

- risx datasets which are accidentally added as RIS datasets no longer
  crash refdbd.

- a possible client/server hiccup caused by checkref reports of a
  particular size was fixed.

- the handling of the (entirely broken by design) issue, edition, and
  volume fields in a variety of reference types has been improved

- when checking risx datasets the tables are now properly unlocked if
  a dataset is skipped


Improvements:

- the en2ris script now uses UTF-8 as the default output encoding

- boatload of improvements of the PHP web interface, among these is
  the tag cloud display of authors, keywords, and periodical names in
  the search results.

- the handling of the AV field has been cleaned up. As of this
  version, the AV field should only contain a physical location,
  i.e. either a description of the location, or a call number. The
  formerly applied PATH: kludge to manage links to local copies of PDF
  files is no longer supported. To store user-specific locations of
  PDF files, use the L1-L4 fields and start the URL with the "file://"
  protocol prefix. These links are assumed to point to local copies
  owned by the user. If the URL uses a relative path, i.e.
    "file://offprints/Miller2002"
  instead of
    "file:///home/markus/offprints/Miller2002"
  (note the additional root slash in the latter URL), the same pdfroot
  mangling is applied as was previously to the AV  - PATH: paths.
  Please see the UPGRADING file for some hints how to efficiently
  migrate existing AV field contents.

- in order to reflect the above change in risx datasets, the risx dtd
  was adapted accordingly. The <link> element is now also allowed in
  <libinfo>. Although the DTD can't enforce this, the pubinfo/link
  elements are supposed to hold publicly accessible URLs of offprints,
  images and such, whereas the libinfo/link elements should receive
  only user-specific local paths to offprints.

- if a character encoding conversion is necessary during data import
  or data export, refdbd now uses the transliteration facility of iconv
  for characters which cannot be mapped to the target encoding, a
  problem which often arises when exporting UTF-8 data to European
  encodings.

- the new PHP web interface was thoroughly overhauled to cope with
  whitespace, apostrophes, and regular expression characters in
  keywords, author names, and journal names. Adding references now works
  from risx data too, and the feedback about the success of adding a
  file has been improved. Adding data from the web form was improved too
  in that empty fields are left out, author name and keyword lists are
  appropriately tokenized, and addref or updateref is used whenever
  appropriate. The code now makes sure to use a unique temporary filename
  which is deleted again afterwards, avoiding clutter in your /tmp. The
  field descriptions of the Add reference form was overhauled too to better
  reflect the contents depending on which reference type is used. The month
  selector now creates a numeric month to comply with the RIS format's
  requirements.

- added a getax command to retrieve authors of any level

- added namespace handling for DocBook and TEI stylesheet driver
  files. TEI P4 and DocBook V4.x files are now always processed with the
  non-namespaced stylesheets, whereas TEI P5 and DocBook V5 files are
  handled by the stylesheets with namespace support. In contrast to the
  stock DocBook stylesheets there is no auto-detection whether or not
  the document uses namespaces.

- TEI P5 cooked bibliographies are now directly generated by refdbd
  instead of transforming them from DocBook bibliographies

- refdbib now optionally retrieves raw bibliographies based on a
  citationlistx input document. This allows using RefDB as a data
  source for SGML/XML documents even if RefDB's citation and
  bibliography formatting capabilities are not desired or applicable

- the PHP web interface has been replaced with a new implementation
  from scratch (thanks to Dan O'Donnell and Dominik Reusser).

- added config files for runbib and refdbjade to pass additional
  options to (open)jade, especially to increase or disable the error
  limit

- added a standalone SRU server for personal use or for testing purposes

- bibliography styles now have elements to record the style author, a
  description, and an URL that points to the source of the
  bibliographic rules.

- added a new legal bibliography style contributed by Jeremy Malcolm

- DocBook and TEI XML output is now indented and can optionally be namespaced

- DocBook XML V5.0 support

- the internal representation of authors and titles is now based on
  the three-level library system (part, publication, series) instead of
  on the once-and-for-all-screwed RIS definition. This makes both the
  programmer's life and the style author's life quite a bit easier

- the internal representation of the M1-M3 fields was improved to
  simplify data access. The contents of these RIS fields is now stored
  in separate fields which reflect the purpose in context of the
  respective reference type

- citation styles now support both authors and editors in monographic data

- citation styles now support the replacements of the M1-M3 fields,
  i.e. stuff like conference location, sender and recipient email
  addresses, class codes, running time and so on

- the output of adduser/deleteuser was cleaned up if no hostname is
  available

- the format specifier NX of the getref command no longer retrieves
  all extended notes associated with a particular reference, as these
  would always include the users reference lists. If you positively want
  those lists, use ALL or NXNL instead.

- the citestylex.dtd now supports different formatting options for
  authors (TEXT) and editors (TEXTED). The styles shipped with RefDB
  were updated accordingly.

- all backends were updated and now export the contents of the RIS
  fields M1-M3 as far as possible

- two new citation styles were added (Tissue_Eng. and Biomaterials)

- the bibliography sorting order is now case-insensitive and takes
  into account missing authors, using the title for sorting in this
  case

- the MySQL reference database schema now uses TEXT instead of BLOB to
  make use of the improved character encoding and collation features of
  newer MySQL versions

- the new refdbc:countref and refdbc:countnote commands are equivalent
  to the getref and getnote commands, respectively, except that they
  do not actually retrieve the reference or note data. Instead, they
  return the number of matching references and notes, respectively.

- the query language supports additional relational operators: <=
  (less than or equal), >= (larger than or equal), <> (not like).

- there is a new backend for MODS output, an XML format maintained by
  the Library of Congress.

- the XML output (risx, mods, xnote) now supports namespaces.

- the -n switch of getbib is no longer supported (it was hardly ever
  used anyway). refdbd now sends a stylespec no matter what.

- the getnote/countnote commands now also support query strings of
  unlimited length, just like the getref/countref commands.

- the adduser/deleteuser commands now support two classes of users:
  read-only users and read/write-users. The former is a simple way to
  generate a database account for users accessing your databases through
  SRU or via the web interface without permitting write access to
  your databases for everyone.

- the new CGI script refdbsru provides an optional SRU (Search and
  Retrieve via URL) interface to your databases.

- the checkref command can now be limited to check individual fields
  or combinations of fields.

- improved import of tagged Pubmed data (med2ris)

- digital object identifiers (DOI) are now harvested from Pubmed data
  and are stored properly. DOIs are presented as links (via
  dx.doi.org) by the (x)html backend.


Backwards-incompatible changes:

- the -i option of refdbjade was changed to I (capital i) to make it
  the same as in runbib (which uses -i for a different
  purpose). runbib in turn now uses -s instead of -I to skip the ID
  extraction step, and -I instead of -m to set include options


-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-users] Survey: most wanted features in a web interface

[Markus H. <mar...@mh...>]

Hi,

I've retrofitted the "tag cloud" (or whatever you like to call it)
display of author names, periodicals, and keywords to the new PHP
interface (it has been available in the getref html output for some
time). Of course there's an alternate stylesheet which suppresses this
effect if you don't need/want/like it.

A preview is right here:

http://refdb.sourceforge.net/preview/php-result-frequency.png

This is about all that I wanted to be available in the first release
of the PHP interface. If you need something else, please speak up
now. I'll get back to fixing some remaining glitches and hope to
finish 0.9.9 fairly soon.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-users] Screenshots of new PHP web interface

[<ste...@jo...>]

Markus Hoenicka a écrit :
> Quoting Stéphane Téletchéa <ste...@jo...>:
> 
>>> - Simple search: type in a word or a phrase, and RefDB returns matches
>>>   in any field.
>> I'm not sure if this one is that usable or not, do you provide a 'à la
>> google' search field ?
>>
> 
> That's why I'd appreciate to get some feedback of users actually  
> trying out the interface. With a few hundred sample references, typing  
> in a keyword, a name, or a year returns a useful subset (usually  
> around 10 or less, unless the search term is "and" or "a") of the  
> references. I can't comment on the quality of the search though,  
> because I assume everyone's expectations are different here.
> 
> And no, there is no google-esque logic behind the search. It just  
> matches the search string in any field. Remember that Google tries to  
> find needles in the haystack of a gazillion unstructured documents. A  
> reference management software uses structured data on purpose, and  
> anything more elaborate than the simple search should involve  
> field-based search strategies (which the interface also offers).

Well, i was thinking about a match of the term in every part of the 
reference (not the title only, or so), linknig it to google was not a 
good comparison. Btw, you answered the question :-)

>>> - Search by unique identifier: retrieve a single dataset by citation
>>>   key or DOI
>> If possible, i'd like to see DOi first since this is more and more used
>>   and is more cross-discplines thant other identifiers (like pubmedID
>> for instance).
> 
> I'm not religious about the order, so I can change it anytime. But not  
> all entries have a DOI (many journals provide electronic versions only  
> for the past 10 years or so), or are suitable to have one  
> (unpublished, in press, emails, you name it). OTOH every entry must  
> have a citation key. I reckon its main purpose is to look up  
> references that you've cited in your own documents, and this is  
> usually done by citation key.

"Oups", i should have read it with this is mind, of course citation 
_from the database_ is more relevant, i thinked of external citations 
... I see now your perspective, you're right in that case, DOI is second :-)

>> I'll do my best to package it soon :-)
> 
> Much appreciated!

This weekend, presumably tonight of monday nigth at worst.

Cheers,
Stéphane
-- 
Stéphane Téletchéa, PhD.                  http://www.steletch.org
Unité Mathématique Informatique et Génome http://migale.jouy.inra.fr/mig
INRA, Domaine de Vilvert                  Tél : (33) 134 652 891
78352 Jouy-en-Josas cedex, France         Fax : (33) 134 652 901

Re: [Refdb-users] Screenshots of new PHP web interface

[Markus H. <mar...@mh...>]

Quoting St=E9phane T=E9letch=E9a <ste...@jo...>:

>> - Simple search: type in a word or a phrase, and RefDB returns matches
>>   in any field.
>
> I'm not sure if this one is that usable or not, do you provide a '=E0 la
> google' search field ?
>

That's why I'd appreciate to get some feedback of users actually =20
trying out the interface. With a few hundred sample references, typing =20
in a keyword, a name, or a year returns a useful subset (usually =20
around 10 or less, unless the search term is "and" or "a") of the =20
references. I can't comment on the quality of the search though, =20
because I assume everyone's expectations are different here.

And no, there is no google-esque logic behind the search. It just =20
matches the search string in any field. Remember that Google tries to =20
find needles in the haystack of a gazillion unstructured documents. A =20
reference management software uses structured data on purpose, and =20
anything more elaborate than the simple search should involve =20
field-based search strategies (which the interface also offers).

>> - Search by unique identifier: retrieve a single dataset by citation
>>   key or DOI
>
> If possible, i'd like to see DOi first since this is more and more used
>   and is more cross-discplines thant other identifiers (like pubmedID
> for instance).

I'm not religious about the order, so I can change it anytime. But not =20
all entries have a DOI (many journals provide electronic versions only =20
for the past 10 years or so), or are suitable to have one =20
(unpublished, in press, emails, you name it). OTOH every entry must =20
have a citation key. I reckon its main purpose is to look up =20
references that you've cited in your own documents, and this is =20
usually done by citation key.

>
> I'll do my best to package it soon :-)

Much appreciated!

Markus

--=20
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-users] Screenshots of new PHP web interface

[<ste...@jo...>]

Markus Hoenicka a écrit :
> Bruce D'Arcus writes:
>  > One comment is the "simple search" is not simple. Compare to, say,
>  > Google, or for reference management, Zotero, where there is a single
>  > field. I would think ideally "simple search" would be integrated into
>  > a main table view, and perhaps do updating using Ajax.
>  > 
> 
> and
> 
> Stéphane Téletchéa writes:
>  > On the main search page, i would like to see something closer to a 
>  > citation match (author, journal, year, page number, the later is missing).
> 
>  > I'm not usre if you index it or not, but getting a DOi field would be 
>  > very nice since those are more and more common those days.
> 
> I've revamped the main search page to include these suggestions. I
> hope the page isn't too cluttered, but now there are four search
> sections:
> 
> - Simple search: type in a word or a phrase, and RefDB returns matches
>   in any field.

I'm not sure if this one is that usable or not, do you provide a 'à la 
google' search field ?

> - Search by unique identifier: retrieve a single dataset by citation
>   key or DOI

If possible, i'd like to see DOi first since this is more and more used 
  and is more cross-discplines thant other identifiers (like pubmedID 
for instance).

> - Simple field-based search: same as "Simple search" of the previous
>   version, plus fields for volume, issue, startpage

I think DOI and this one are sufficient for most of the queries, but it 
depends on other PoV :-)

> - Advanced query: same as before

I think you could keep this one and drop the simple search.


> An updated screenshot is right here:
> 
> http://refdb.sourceforge.net/preview/php-search.png
> 

I'll do my best to package it soon :-)
Stéphane

-- 
Stéphane Téletchéa, PhD.                  http://www.steletch.org
Unité Mathématique Informatique et Génome http://migale.jouy.inra.fr/mig
INRA, Domaine de Vilvert                  Tél : (33) 134 652 891
78352 Jouy-en-Josas cedex, France         Fax : (33) 134 652 901

Re: [Refdb-users] Screenshots of new PHP web interface

[Markus H. <mar...@mh...>]

Bruce D'Arcus writes:
 > One comment is the "simple search" is not simple. Compare to, say,
 > Google, or for reference management, Zotero, where there is a single=

 > field. I would think ideally "simple search" would be integrated int=
o
 > a main table view, and perhaps do updating using Ajax.
 >=20

and

St=E9phane T=E9letch=E9a writes:
 > On the main search page, i would like to see something closer to a=20=

 > citation match (author, journal, year, page number, the later is mis=
sing).

 > I'm not usre if you index it or not, but getting a DOi field would b=
e=20
 > very nice since those are more and more common those days.

I've revamped the main search page to include these suggestions. I
hope the page isn't too cluttered, but now there are four search
sections:

- Simple search: type in a word or a phrase, and RefDB returns matches
  in any field.

- Search by unique identifier: retrieve a single dataset by citation
  key or DOI

- Simple field-based search: same as "Simple search" of the previous
  version, plus fields for volume, issue, startpage

- Advanced query: same as before

An updated screenshot is right here:

http://refdb.sourceforge.net/preview/php-search.png


regards,
Markus

--=20
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-users] Survey: most wanted features in a web interface

[Dan O'D. <dan...@ul...>]

That looks great. I'll look at the code, but this import engine may also
greatly help the integration of the "search external databases" tab.
I'll see what I can do with it on the weekend. Unless TEI business takes
over.

On Thu, 2007-25-10 at 01:24 +0200, Markus Hoenicka wrote:
> Jus...@Pi... writes:
>  > My greatest worry is simple batch entry of bibliographic data for
>  > people totally ignorant of RefDB and its functioning. For this the
>  > following would be highly desirable:
>  > 
>  > - server-side conversion of the most important bib data formats (so
>  >   people can directly upload their BibTeX files),
>  > 
> 
> I've extended the "Upload from file" dialog accordingly. The PHP
> interface now determines at runtime which converters are available,
> and displays a list of these supported formats. This includes those
> formats that require bibutils. I've posted an updated screenshot at
> 
> http://refdb.sourceforge.net/preview/php-add-file.png
> 
> A few minor problems remain to be solved. E.g. we can't auto-detect
> the tagged EndNote RIS export as it resembles "real" RIS but has a
> couple of bugs which are not necessarily detectable in each
> dataset. It will be imported as RIS, warts and all. Also, I haven't
> found the header structure of binary UKMARC datasets to see whether we
> can tell them apart from the other MARC variants (if anyone uses this
> format at all, that is). Finally, I'd like to see a sample of
> EndNote's XML export, if someone has one handy. I'd like to see how to
> auto-detect this format.
> 
> regards,
> Markus
> 
-- 
Daniel Paul O'Donnell, PhD
Chair, Text Encoding Initiative <http://www.tei-c.org/>
Director, Digital Medievalist Project <http://www.digitalmedievalist.org/>
Associate Professor and Chair of English
University of Lethbridge
Lethbridge AB T1K 3M4
Vox: +1 403 329 2378
Fax: +1 403 382-7191
Homepage: http://people.uleth.ca/~daniel.odonnell/