refdb-users — List for RefDB users

Re: [Refdb-users] forced to use sqlite

[Z F <mai...@ya...>]

Dear Markus
 
> Are you sure there is no daemon running when you test the edited
> refdbdrc file? I mean if Debian installed and started refdbd as a
> daemon, it will answer the client requests even if you start a second
> refdbd instance. The daemon will of course use the refdbdrc settings
> before you edited the config file. Use refdbctl reload to reconfigure
> the running daemon.
> 
> If that is not the reason, where is refdbdrc located? What happens if
> you use refdbd -s -e 0 -l 7 -D mysql -b 3306?

This is exactly what I did. I followed the manual :-)
killed the daemon and used refdbd -s -e 0 -l 7 -D mysql
in one window and refdba in another.

when I execute viewstat, it sais that I am connected to sqlite
database...

I can not find what the problem is. Where should I look?
mysql data base seems to be working fine. I can connect to it. I
created
the database as descrebed in the manual..

Thank you for your kind help

Lazar

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

[Refdb-users] forced to use sqlite

[Markus H. <mar...@mh...>]

Z F writes:
 > I do not understand what I did wrong. Could some one point out
 > this to me?
 > 

Are you sure there is no daemon running when you test the edited
refdbdrc file? I mean if Debian installed and started refdbd as a
daemon, it will answer the client requests even if you start a second
refdbd instance. The daemon will of course use the refdbdrc settings
before you edited the config file. Use refdbctl reload to reconfigure
the running daemon.

If that is not the reason, where is refdbdrc located? What happens if
you use refdbd -s -e 0 -l 7 -D mysql -b 3306?

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-users] forced to use sqlite

[Z F <mai...@ya...>]

Hello everybody,

I just installed refdb (debian packages) and can not seem to 
configure it properly.

I am trying to use a mysql data base.

I have set 

dbserver  mysql

in refdbdrc

refdbd -s -e 0 -l 7 sais:

dbi_driver_dir went to:
/usr/lib/dbd
dbi is up using driver dir:
/usr/lib/dbd
Available libdbi database drivers:
mysql
sqlite
Requested libdbi driver found
Database directory:
/var/lib/refdb/db
application server started
share extended notes by default
use /tmp/refdbd_fifo9905 as fifo
server waiting n_max_fd=4

which says to me that the mysql driver is found, but

but when I try to use refbda viewstat
I get  connected to sqlite....

You are served by: refdb 0.9.6-pre3
Client IP: 127.0.0.1
Connected via sqlite driver (dbd_sqlite v0.8.1)
to: 2.8.16
db version: 1
serverip: localhost
timeout: 180
dbs_port: 
logfile: /var/log/refdbd.log
logdest: FILE
loglevel: INFO
remoteadmin: off
pidfile: /var/run/refdbd.pid



If I configure everything for a real IP address (the mysql server
and the refdb) than refdba can non connect to the server at all.

skip-networking is off in mysql configuration.

mysql database seems to be working. I have created refdb database
as described in the manual.

I do not understand what I did wrong. Could some one point out
this to me?

Thank you very much.

Lazar

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: [Refdb-users] first install of RefDB .debs

[Markus H. <mar...@mh...>]

Doug du Boulay <dou...@gm...> was heard to say:

> I don't know if this is relevant, but apparently I have
>
> -rwxr-xr-x    1 root     root   1265136 Aug 30 01:45 /lib/tls/libc-2.3.5.so*
> lrwxrwxrwx    1 root     root        13 Sep 27 11:45 /lib/tls/libc.so.6 ->
> libc-2.3.5.so*
>
> and also:
> -rwxr-xr-x    1 root     root   1142620 Aug 30 01:45 /lib/libc-2.3.5.so*
> lrwxrwxrwx    1 root     root        13 Sep 27 11:45 /lib/libc.so.6 ->
> libc-2.3.5.so*
>
> but I don't know the significance of the tls directory.
>

Googling tells me that this is apparently part of the Transport Layer Security
Library (http://www.gnu.org/software/gnutls/). There may be a few fixes in the
C library for performance reasons. It is quite possible that the modified C
library breaks RefDB. However, simply moving /lib/tls out of your library path
may break other applications that depend on it. Your safest bet is still to
compile RefDB from the sources.

If you're daring, you can temporarily rename /lib/tls to /lib/tls.hidden or
whatever and see whether it fixes the bib2ris crash. Google tells me that mv
still works after renaming /lib/tls :-) so this is not just a one-way road.

> refdbc: listdb
> 999:0
> refdbc: whichdb
> No default database. Select one with selectdb. Stop.
> refdbc: selectdb refdb
> not a RefDB database
> refdbc: quit
>
> The file /etc/refdb/refdbdrc contains
> dbpath          /var/lib/refdb/db
>
> so I can only assume  that it is looking at the
> database /var/lib/refdb/db/refdb and something is wrong with it?
>

No, there is just a misunderstanding: You've created the system database refdb
ok, but this is not a database that will contain reference data. It is for
style data and reserved journal words, i.e. stuff that all reference databases
share.

Next you should fire up refdba and run "createdb DBNAME" (replace DBNAME with
your preferred database name). If you're using SQLite you should then be able
to access DBNAME from refdbc. You should also read the manual section about
configuration files. You may want to set your new database as the default
database.

hope this helps
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-users] first install of RefDB .debs

[Doug du B. <dou...@gm...>]

On Saturday 14 January 2006 23:39, Markus Hoenicka wrote:
> Doug du Boulay writes:
>  > Program received signal SIGSEGV, Segmentation fault.
>  > 0xb7f44777 in confstr () from /lib/tls/libc.so.6
>  > (gdb) bt
>  > #0  0xb7f44777 in confstr () from /lib/tls/libc.so.6
>  > #1  0xb7f45801 in confstr () from /lib/tls/libc.so.6
>  > #2  0xb7f4586d in getopt () from /lib/tls/libc.so.6
>  > #3  0x0804c4b0 in ?? ()
>  > #4  0x72326269 in ?? ()
>
>
> This doesn't look like a RefDB problem to me. If bib2ris segfaults
> while running getopt, there must be some incompatibility between your
> installed version of the C library and the version the package was
> built with. 

I don't know if this is relevant, but apparently I have 

-rwxr-xr-x    1 root     root   1265136 Aug 30 01:45 /lib/tls/libc-2.3.5.so*
lrwxrwxrwx    1 root     root        13 Sep 27 11:45 /lib/tls/libc.so.6 -> 
libc-2.3.5.so*

and also: 
-rwxr-xr-x    1 root     root   1142620 Aug 30 01:45 /lib/libc-2.3.5.so*
lrwxrwxrwx    1 root     root        13 Sep 27 11:45 /lib/libc.so.6 -> 
libc-2.3.5.so*

but I don't know the significance of the tls directory.

> I'm sorry for the inconvenience, but David seems to be occupied with
> other stuff, and he is the one who knew how to fix it.

No problem. It keeps life interesting.

Anyway, going back to this:
> This is also correct. The size should be >0, so there must have something
> gone wrong when creating the database. I'm not sure if and how the package
> install script attempts to create the database. If it does attempt this,
> David would have to fix it. But to get you up and running, please follow
> the instructions about creating the main database in the manual. This
> process will modify only this particular file and should not confuse the
> package manager.

I ran these commands:
sqlite /var/lib/refdb/db/refdb < /usr/share/refdb/sql/refdb.dump.sqlite
ls -la /var/lib/refdb/db/refdb
-rw-r--r--    1 root     root   41984 Jan 15 23:04 /var/lib/refdb/db/refdb

which seemed to work.  Then I tried 
/etc/init.d/refdb restart
Restarting bibliography tool application server: refdb.
/usr/bin/refdbctl restart: bibliography tool application server restarted

Then I ran refdbc
refdbc: listdb
999:0
refdbc: whichdb
No default database. Select one with selectdb. Stop.
refdbc: selectdb refdb
not a RefDB database
refdbc: quit
 
The file /etc/refdb/refdbdrc contains
dbpath          /var/lib/refdb/db

so I can only assume  that it is looking at the 
database /var/lib/refdb/db/refdb and something is wrong with it?

Should I build refdb myself at this point?
Thanks

Doug

Re: [Refdb-users] first install of RefDB .debs

[Markus H. <mar...@mh...>]

Doug du Boulay writes:
 > Sorry, I forgot to do that.

Not a problem. In fact I can't reproduce this behaviour on FreeBSD.

 > (gdb) run
 > Starting program: /usr/bin/bib2ris
 > /usr/share/refdb/examples
 > (no debugging symbols found)...(no debugging symbols found)...(no debugging 
 > symbols found)...(no debugging symbols found)...
 > Program received signal SIGSEGV, Segmentation fault.
 > 0xb7f44777 in confstr () from /lib/tls/libc.so.6
 > (gdb) bt
 > #0  0xb7f44777 in confstr () from /lib/tls/libc.so.6
 > #1  0xb7f45801 in confstr () from /lib/tls/libc.so.6
 > #2  0xb7f4586d in getopt () from /lib/tls/libc.so.6
 > #3  0x0804c4b0 in ?? ()
 > #4  0x72326269 in ?? ()

Oh, waitaminute. I somehow thought you were talking about refdbib. If
bib2ris segfaults, the main database doesn't have anything to do with
it. Just forget my comments about that.

This doesn't look like a RefDB problem to me. If bib2ris segfaults
while running getopt, there must be some incompatibility between your
installed version of the C library and the version the package was
built with. I'm afraid your best bet is to compile and install RefDB
from the sources. As David has already identified possible name
clashes while developing the packages, these were removed
upstream. You should be able to deinstall your homebuilt installation
with "make uninstall" without confusing apt. You'd have to follow the
setup instructions in the manual for creating the system database and
for setting your environment.

I'm sorry for the inconvenience, but David seems to be occupied with
other stuff, and he is the one who knew how to fix it.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-users] first install of RefDB .debs

[Doug du B. <dou...@gm...>]

On Friday 13 January 2006 16:40, Markus Hoenicka wrote:
> > Then I tried bib2ris and even with no arguments it gives a Segmentation
> > fault.
>
> bib2ris should not segfault even if the main database is missing (which I
> think is the problem here). I'll try to reproduce this on my box after
> removing the main database. If I can't reproduce it on my system, would you
> be able to run bib2ris in gdb and send me the backtrace (the gdb command is
> bt) after it segfaults?

Sorry, I forgot to do that.
Thanks again
Doug

(gdb) run
Starting program: /usr/bin/bib2ris
/usr/share/refdb/examples
(no debugging symbols found)...(no debugging symbols found)...(no debugging 
symbols found)...(no debugging symbols found)...
Program received signal SIGSEGV, Segmentation fault.
0xb7f44777 in confstr () from /lib/tls/libc.so.6
(gdb) bt
#0  0xb7f44777 in confstr () from /lib/tls/libc.so.6
#1  0xb7f45801 in confstr () from /lib/tls/libc.so.6
#2  0xb7f4586d in getopt () from /lib/tls/libc.so.6
#3  0x0804c4b0 in ?? ()
#4  0x72326269 in ?? ()
(gdb)

Re: [Refdb-users] first install of RefDB .debs

[Markus H. <mar...@mh...>]

Hi,

Doug du Boulay <dd...@ow...> was heard to say:

> I reran /etc/init.d/refdb start/stop a couple of times
> before I realised that the script was doing a silent exit 1
> on account of the spurious local/ in the init script path names:
>
> # the full path to the binary that is to be started as a daemon
> DAEMON=/usr/local/bin/refdbd
> # the full path to the script that actually starts and stops the application
> REFDBCTL=/usr/local/bin/refdbctl
>

Yep, this is obviously wrong for Debian. David would have to fix this in his
.debs, but I haven't seen him on the list for a while.

>    "main database is too old or corrupt"
>
> where I assume the main database is the SQLite file:
> -rw-r--r--    1 root     root      0 Jan 11  22:05 /var/lib/refdb/db/refdb
>

This is also correct. The size should be >0, so there must have something gone
wrong when creating the database. I'm not sure if and how the package install
script attempts to create the database. If it does attempt this, David would
have to fix it. But to get you up and running, please follow the instructions
about creating the main database in the manual. This process will modify only
this particular file and should not confuse the package manager.

> Then I tried bib2ris and even with no arguments it gives a Segmentation
> fault.
>

bib2ris should not segfault even if the main database is missing (which I think
is the problem here). I'll try to reproduce this on my box after removing the
main database. If I can't reproduce it on my system, would you be able to run
bib2ris in gdb and send me the backtrace (the gdb command is bt) after it
segfaults?

regards,
Markus


-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-users] first install of RefDB .debs

[Doug du B. <dd...@ow...>]

Hi folks,

I am using an /etc/apt/sources.list with the following:

deb http://ftp.au.debian.org/debian/ unstable main non-free contrib
deb http://refdb.sourceforge.net/debian/release unstable main

I just did a pristine, default installation of the debian package 
refdb_0.9.6-pre2_i386.deb

Then I tried to run refdbc and I got the response:

     "could not establish server connection"

I reran /etc/init.d/refdb start/stop a couple of times
before I realised that the script was doing a silent exit 1
on account of the spurious local/ in the init script path names:

# the full path to the binary that is to be started as a daemon
DAEMON=/usr/local/bin/refdbd
# the full path to the script that actually starts and stops the application
REFDBCTL=/usr/local/bin/refdbctl

I set the paths correctly and tried refdbc again and got:

   "main database is too old or corrupt"

where I assume the main database is the SQLite file:
-rw-r--r--    1 root     root      0 Jan 11  22:05 /var/lib/refdb/db/refdb

Then I tried bib2ris and even with no arguments it gives a Segmentation fault.

So I'm guessing something has gone tragically wrong with the 
installation process?

Any ideas? 
Thanks
Doug

P.S.

gdb says:
Program received signal SIGSEGV, Segmentation fault.
0xb7f44777 in confstr () from /lib/tls/libc.so.6

 
and from an strace bib2ris dump:
execve("/usr/bin/bib2ris", ["bib2ris"], [/* 49 vars */]) = 0
uname({sys="Linux", node="timeout", ...}) = 0
brk(0)                                  = 0x8054000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or 
directory)
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 
0xb7fea000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or 
directory)
open("/etc/ld.so.cache", O_RDONLY)      = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=66607, ...}) = 0
old_mmap(NULL, 66607, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fd9000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or 
directory)
open("/usr/lib/libbtparse.so.0", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\3006\0"..., 512) = 
512
fstat64(3, {st_mode=S_IFREG|0644, st_size=65808, ...}) = 0
old_mmap(NULL, 77244, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 
0xb7fc6000
old_mmap(0xb7fd5000, 4096, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xf000) =0xb7fd5000
old_mmap(0xb7fd6000, 11708, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7fd6000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or 
directory)
open("/lib/tls/libc.so.6", O_RDONLY)    = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\300O\1"..., 512) = 
512
fstat64(3, {st_mode=S_IFREG|0755, st_size=1265136, ...}) = 0
old_mmap(NULL, 1275196, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) 
= 0xb7e8e000
old_mmap(0xb7fbf000, 16384, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x130000) = 0xb7fbf000
old_mmap(0xb7fc3000, 9532, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7fc3000
close(3)                                = 0
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 
0xb7e8d000
mprotect(0xb7fbf000, 4096, PROT_READ)   = 0
set_thread_area(0xbffff62c)             = 0
munmap(0xb7fd9000, 66607)               = 0
ioctl(0, TCGETS, {B38400 opost isig icanon echo ...}) = 0
brk(0)                                  = 0x8054000
brk(0x8075000)                          = 0x8075000
open("/etc/refdb/bib2risrc", O_RDONLY)  = -1 ENOENT (No such file or 
directory)
open("/home/timeout/ddb/.bib2risrc", O_RDONLY) = -1 ENOENT (No such file or 
directory)
open("/home/timeout/ddb/bib2risrc", O_RDONLY) = -1 ENOENT (No such file or 
directory)
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
+++ killed by SIGSEGV +++

Re: [Refdb-users] names again (was Re: RISY Report)

[Bruce D'A. <bda...@gm...>]

On 1/10/06, Bruce D'Arcus <bda...@gm...> wrote:

> > For whom? For the coder or for the valued end-user? I guess the end-use=
r who
> > uses 98% simple western names will appreciate if he doesn't have to key=
 in 37
> > display variants for each name.
>
> I just don't accept it's necessary to do more than one display name;
> ever. It's certainly not with any of my data at least (though the
> requirements for funky name handling are).

OK, I changed my mind. I do think it valuable to at least break apart
the primary pieces (though less so to break apart, say, a givenname
into pieces).

I guess I'd favor something like this (following from your's, Markus,
but modifying slightly):

agents: id, type, sort_key, display_name

names: agent_id, givenname, familyname, prefix, suffix, articular,
language, script

A few notes:

Yes, in my case, I think it's fine to put "J. Edgar" in the givenname
column and not worry about having a separate table for name parts.=20
YMMV.

I have agents just to include organizations (organtional authors,
publishers, legislative bodies, etc.). I suppose that could be a
separate ttable (?).

I moved language to the names because it is a property of a name.

This has me wiondeiring if having the sort and display names on the
agent might be a problem for some (international) contexts (??), even
if it is otherwise convenient.

I added the script just because there needs to be some way to get --
for those that need it -- name formatting of the form [transliterated
Western] ([original Kanji]). There may be a simpler way to do that;
not sure.

There proabably might also be some flag that indicates the sorting
algortihm to use?

Bruce

[Refdb-users] RE: Start selecting health commodities in the easiest way.

[Eavan <jer...@kj...>]

Stop being boring and come try something new with me- I heard this e-store
is tremendous!

Let's talk later,

Eavan

 -------Original Message-------
 
From: Wallie [mailto:sco...@li...] 
Sent: Tuesday, January 03, 2005 11:34 PM
To: Eavan
Subject: Start selecting health commodities in the easiest way.

What's up, Eavan

Since you are such a great companion, I want to inform you about this great
chance that you can save on performance enhancers. 

This store makes available the same compound you'd purchase at the local
store, only cheaper. Because I feel for you so much, I have searched a lot
of sites until I bumped into the most inexpensive price on health products.
You have to be loathing not acquiring the medications you want. 
http://im.aU8NF.rebootofemagic.com


not evidence there. result in desire  have filled her with diary delight,
now it maintenance grateful depressed her.  
not evolution there. comedian reveal 

I look forward to talking to you later,

Wallie

Re: [Refdb-users] names again (was Re: RISY Report)

[Bruce D'A. <bda...@gm...>]

On 1/10/06, Markus Hoenicka <mar...@mh...> wrote:
> Bruce D'Arcus <bda...@gm...> was heard to say:
>
> > So then if you want to be international, you need to be able to track
> > both the name part type (forename, surname, articular, prefix, suffix,
> > etc.) AND their position, AND the language (because sorting varies by
> > language).
> >
> > That suggests not a single table (say agents), nor even two joined
> > tables (agents and names), but rather more.
> >
>
> A person would be represented as an entry in a table like this:
>
> ID | sort-name | display-name | language
>
> The name parts go into another table like this:
>
> ID | person-ID | chunk | type | position
>
> I guess this is all it takes. sort-name is a convenience field which is,
> strictly speaking, redundant (as it can be assembled from the chunks), bu=
t it
> makes queries speedier as name queries are so common.

Nice :-)

> You'll only need three tables if you want to handle aliases, like a perso=
n who
> changed his/her name after marrying, or who simply can't make up his mind=
 (John
> Cougar -> John Cougar Mellencamp -> John Mellencamp, a well-known horror =
in
> record stores).

Also good.

> > Your formatting code then needs to be able to understand all that
> > logic (e.g the sorting).
> >
> > Is that really easier?
> >
>
> For whom? For the coder or for the valued end-user? I guess the end-user =
who
> uses 98% simple western names will appreciate if he doesn't have to key i=
n 37
> display variants for each name.

I just don't accept it's necessary to do more than one display name;
ever. It's certainly not with any of my data at least (though the
requirements for funky name handling are).

Bruce

Re: [Refdb-users] names again (was Re: RISY Report)

[Markus H. <mar...@mh...>]

Bruce D'Arcus <bda...@gm...> was heard to say:

> So then if you want to be international, you need to be able to track
> both the name part type (forename, surname, articular, prefix, suffix,
> etc.) AND their position, AND the language (because sorting varies by
> language).
>
> That suggests not a single table (say agents), nor even two joined
> tables (agents and names), but rather more.
>

A person would be represented as an entry in a table like this:

ID | sort-name | display-name | language

The name parts go into another table like this:

ID | person-ID | chunk | type | position

I guess this is all it takes. sort-name is a convenience field which is,
strictly speaking, redundant (as it can be assembled from the chunks), but it
makes queries speedier as name queries are so common. RefDB already uses this
cheap trick.

You'll only need three tables if you want to handle aliases, like a person who
changed his/her name after marrying, or who simply can't make up his mind (John
Cougar -> John Cougar Mellencamp -> John Mellencamp, a well-known horror in
record stores).


> Your formatting code then needs to be able to understand all that
> logic (e.g the sorting).
>
> Is that really easier?
>

For whom? For the coder or for the valued end-user? I guess the end-user who
uses 98% simple western names will appreciate if he doesn't have to key in 37
display variants for each name.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-users] names again (was Re: RISY Report)

[Bruce D'A. <bda...@gm...>]

On 1/10/06, Markus Hoenicka <mar...@mh...> wrote:

> > Also, one of the reasons why current solutions don't get it right is
> > because they try to have a single representation (whether a chopped up
> > text string, or XML tags) cover all cases. My argument -- which I've
> > been slowly stewing on over the past few months -- is that by focusing
> > on role-based name representations, you solve a lot of these problems
>
> But to cover all the border cases you'd need a lot of these representatio=
ns,
> essentially one for every possible name formatting style that any journal=
 out
> there may have figured out.  This looks harder and more tedious to me tha=
n
> marking up the names.

So then if you want to be international, you need to be able to track
both the name part type (forename, surname, articular, prefix, suffix,
etc.) AND their position, AND the language (because sorting varies by
language).

That suggests not a single table (say agents), nor even two joined
tables (agents and names), but rather more.

Your formatting code then needs to be able to understand all that
logic (e.g the sorting).

Is that really easier?

Bruce

Re: [Refdb-users] names again (was Re: RISY Report)

[Markus H. <mar...@mh...>]

Bruce D'Arcus <bda...@gm...> was heard to say:

> My classic example of a simple Western name that is problematic is "J.
> Edgar Hoover". I'd be fine entering and storing his name as "Hoover,
> J. Edgar" and having a rule that says that a single capitalized
> character followed by a period ia an initial, and leave it at that. In
> some styles you'd end up with "J. Edgar Hoover" in others "J. E.
> Hoover", and in still others "JE Hoover."
>
> That's fine; it works. And it's easy to process, and -- more
> importantly -- to handle in a GUI.
>
> It would not allow you to take "Jane Susan Smith" ("Smith, Jane
> Susan") and to somehow abbreviate one of those names, though, but is
> that really a practical problem? Probably this person woudn't want
> either name privlieged anyway,
>

This nicely illustrates the problems that arise if you abuse representation for
markup. In order to tell the system that Jane wants to go by her first
firstname, you'd have to enter her as "Smith, Jane S.". You lose information as
you couldn't properly spell her name in styles that ask for full firstnames. If
you enter her as "Smith, Jane Susan" the system can guess she goes by her first
firstname just to find out that she calls herself "J. Susan Smith". The system
could try to avoid this by spelling out both firstnames, but then you may end
up with an entry "Jane Susan Smith" which makes "Susan" look like a middle name
if the style asks for spelled out middle names.

My solution to this problem uses the Perlish approach: make simple things simple
and hard things possible (or so it goes). If RefDB uses a richer markup to enter
author names, you will be able to hand-code the names in XML if they don't fit
the standard western scheme. But no one is going to keep you from designing a
GUI which accepts names in the simple, position/comma/period-based "markup"
that you mentioned above, and then converts them properly into the XML markup.
It'll work just fine for the simple names. However, if you base the data model
on the simple string approach, you're screwed if you need the additional
information required to format the hard names.


> Also, one of the reasons why current solutions don't get it right is
> because they try to have a single representation (whether a chopped up
> text string, or XML tags) cover all cases. My argument -- which I've
> been slowly stewing on over the past few months -- is that by focusing
> on role-based name representations, you solve a lot of these problems.
>

But to cover all the border cases you'd need a lot of these representations,
essentially one for every possible name formatting style that any journal out
there may have figured out. This looks harder and more tedious to me than
marking up the names.

regards,
Markus


-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-users] names again (was Re: RISY Report)

[Bruce D'A. <bda...@gm...>]

On 1/10/06, Markus Hoenicka <mar...@mh...> wrote:
> Bruce D'Arcus <bda...@gm...> was heard to say:
>
>
> > sort: Madonna
> > display: Madonna
> >
> [...]
> > sort: Patterson, Tom
> > display: Tom Patterson
> [...]
> > sort: Chiang Lui
> > display: Chiang Lui
> [...]
> > informalname: David Doe
>
> I'm sorry but I don't see how this gets us much further. We've found out =
in the
> past that we have to record a display name as some bibliography styles wa=
nt the
> name printed as it appears on the work, so adding a displayname field is =
a good
> thing. But the sort name is as good or as bad as the RIS name fields are
> currently. It does not help to decide whether a capital letter is an
> abbreviation or a single-letter name. It does not help to decide which na=
me
> part should be abbreviated if the style calls for abbreviated firstnames.=
 It
> does not help to decide which first/middlename should not be abbreviated =
if the
> style asks for only one spelled-out first/middlename.

But let's step back and consider the context in which these
applications will typically be used: in GUI interfaces. We need a
solution that will work in that real world context.

My classic example of a simple Western name that is problematic is "J.
Edgar Hoover". I'd be fine entering and storing his name as "Hoover,
J. Edgar" and having a rule that says that a single capitalized
character followed by a period ia an initial, and leave it at that. In
some styles you'd end up with "J. Edgar Hoover" in others "J. E.
Hoover", and in still others "JE Hoover."

That's fine; it works. And it's easy to process, and -- more
importantly -- to handle in a GUI.

It would not allow you to take "Jane Susan Smith" ("Smith, Jane
Susan") and to somehow abbreviate one of those names, though, but is
that really a practical problem? Probably this person woudn't want
either name privlieged anyway,

> > I'll just take a German example from my field, so that maybe Markus
> > can correct me:
> >
> > sort: Humbolt, Alexander von
> > display: Alexander von Humbolt
> >
>
> I'm afraid this is a border case. The "von" is certainly derived from a
> honorific, but it is close to being treated as a part of the surname. If =
you
> had a chance to talk to this person you'd address him by last name as "He=
rr von
> Humboldt". This is even more apparent in Dutch where the "van" or "van de=
n" et
> al. may either be capitalized ("Van de Voorde,J" is in my database), or i=
t is
> even contracted with the surname, resulting in "Vanhoutte, Paul M." and s=
imilar
> names. I guess other languages have more striking examples for middle hon=
orifics
> although I can't present one right now. But that's just a side note.

OK, thanks for the clarification. But this is exactly the case where
the display/sort distinction becomes useful.

I asked abotu von Humbolt because typically in English texts the "von"
is dropped, at leat for sorting, and is treated as an articular.

> > It even works nicely for formatting, because when sorting a reference
> > list you just do a sort on a concatenated string of the
> > creator.agent.sortname values, and if you need to format them as the
> > sort key, just print directly.
>
> Basically RefDB does just this, as it uses a concatenation of the name pa=
rts for
> sorting. However, to get the bibliographic output right it is essential t=
o
> identify the name parts properly.
> >
> > You'd still need code to parse and process initials and such, but that
> > seems like a problem with known solutions.
> >
>
> Unfortunately not. I daresay none of the commercial packages, let alone R=
efDB,
> gets this right for names which do not stick slavishly to the implicit ma=
rkup
> rules of RIS names. I still maintain that you need a richer name markup t=
o help
> an app get this right.

But I still come back to the problem of if you have ths rich markup,
how do you actually create it?  99.99 % (or whatever) of the world
does not know what XML or emacs is. If they use RefDB, they want a
nice user interface. So how would you design a form that allowed a
user to accomplish this? And say you want to make your users' lives
easier by allowing those forms to autocomplete based on existing
objects?

Also, one of the reasons why current solutions don't get it right is
because they try to have a single representation (whether a chopped up
text string, or XML tags) cover all cases. My argument -- which I've
been slowly stewing on over the past few months -- is that by focusing
on role-based name representations, you solve a lot of these problems.

Bruce

Re: [Refdb-users] names again (was Re: RISY Report)

[Markus H. <mar...@mh...>]

Bruce D'Arcus <bda...@gm...> was heard to say:


> sort: Madonna
> display: Madonna
>
[...]
> sort: Patterson, Tom
> display: Tom Patterson
[...]
> sort: Chiang Lui
> display: Chiang Lui
[...]
> informalname: David Doe

I'm sorry but I don't see how this gets us much further. We've found out in the
past that we have to record a display name as some bibliography styles want the
name printed as it appears on the work, so adding a displayname field is a good
thing. But the sort name is as good or as bad as the RIS name fields are
currently. It does not help to decide whether a capital letter is an
abbreviation or a single-letter name. It does not help to decide which name
part should be abbreviated if the style calls for abbreviated firstnames. It
does not help to decide which first/middlename should not be abbreviated if the
style asks for only one spelled-out first/middlename.

> I'll just take a German example from my field, so that maybe Markus
> can correct me:
>
> sort: Humbolt, Alexander von
> display: Alexander von Humbolt
>

I'm afraid this is a border case. The "von" is certainly derived from a
honorific, but it is close to being treated as a part of the surname. If you
had a chance to talk to this person you'd address him by last name as "Herr von
Humboldt". This is even more apparent in Dutch where the "van" or "van den" et
al. may either be capitalized ("Van de Voorde,J" is in my database), or it is
even contracted with the surname, resulting in "Vanhoutte, Paul M." and similar
names. I guess other languages have more striking examples for middle honorifics
although I can't present one right now. But that's just a side note.

> It even works nicely for formatting, because when sorting a reference
> list you just do a sort on a concatenated string of the
> creator.agent.sortname values, and if you need to format them as the
> sort key, just print directly.

Basically RefDB does just this, as it uses a concatenation of the name parts for
sorting. However, to get the bibliographic output right it is essential to
identify the name parts properly.
>
> You'd still need code to parse and process initials and such, but that
> seems like a problem with known solutions.
>

Unfortunately not. I daresay none of the commercial packages, let alone RefDB,
gets this right for names which do not stick slavishly to the implicit markup
rules of RIS names. I still maintain that you need a richer name markup to help
an app get this right.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-users] names again (was Re: RISY Report)

[Bruce D'A. <bda...@gm...>]

Am posting this back to the list, for the archive.

Just as an experiment, let me try with the sort/display name solution.

On 1/10/06, David Mundie <mu...@an...> wrote:

> * What are you going to do about people who only have one name?
>
> <one-and-only-name>Madonna</one-and-only-name>

sort: Madonna
display: Madonna

> * What are you going to do about people who have exactly two names?
>
> <first-of-two-names>Tom</first-of-two-names><second-of-two-names>Patterso=
n</second-of-two-names>

sort: Patterson, Tom
display: Tom Patterson

> * Do "first" and "last" really mean "written first" and "written last"
>   or are they code-words for "personal" and "family" name?
>
> <personal-name written=3D"first">Liu</personal-name>
> <family-name written=3D"second">Chiang</family-name>

The language issue is solved intrinsically here.

sort: Chiang Lui
display: Chiang Lui

[aside from the fact that sometimes people with Asian names reverse
the "proper" sort order to accommodate Western audiences; and then
,what if you need to store the original Mandarin name in the original
script?]

> * In general, what about people who have three names, but don't go
>   by their first name...
> <first>John</first><second>David</second><third>Doe</third>
> <goes-by>david</goes-by>

<foaf:nick>David</foaf:nick>

.. or adapted to this approach:

informalname: David Doe

> * What about people whose names have an honorific in the middle?
>   For example, an Irish speaker told me that my name would be written,
>   in Irish, as "richard ursul o caiv" where the "ursul" bit means,
>   IIRC, "young knight" and functions like "Mr".
> <first>Richard</first><honorific>ursul</honorific><last>o
> caiv</last>

I'll just take a German example from my field, so that maybe Markus
can correct me:

sort: Humbolt, Alexander von
display: Alexander von Humbolt

Does't stripping it down to this contextual focus -- what do you want
to *do* with a name (and secondarily, for what context?)? --
dramatically simplify things, while also making it much more flexible
and general?

It even works nicely for formatting, because when sorting a reference
list you just do a sort on a concatenated string of the
creator.agent.sortname values, and if you need to format them as the
sort key, just print directly.

You'd still need code to parse and process initials and such, but that
seems like a problem with known solutions.

Bruce

Re: [Refdb-users] RISY Report

[David M. <mu...@an...>]

On 2006-Jan-09, at 16:54, Markus Hoenicka wrote:

> You basically reduce the reference types to simple forms.

I didn't "reduce" the reference types to simple forms - that's what  
they *already were*. All I did was to capture in XML the content  
models implicit in the Reference Manager manual.

> This has its charms as it would neatly transform to e.g. HTML forms  
> for entering
> data,

I suppose, but that thought was the furthest from my mind. I just  
wanted a set of convenient reference types that I could use to  
validate my submissions to RefDB. I guess the result looks more "form- 
like" than it should because I made no attempt to indicate which  
elements could be repeated.

> but from a markup perspective I just don't like the result.

I had to remind Bruce, and I'll remind you: I was not doing DTD  
design. What *I* think the right markup for bibliographies is didn't  
enter into it. My only claim that is that RISY is a faithful, even  
slavish rendition of RIS. You can't make a silk purse out of a sow's  
ear. :-)

>> The current release makes all fields required, just because I wanted
>> to postpone thinking about which ones should be optional. (Reference
>> Manager is no help here - the only field it requires is the date,
>> presumably for some implementation reason.)
>
> As far as I understand the "spec", only TI and ER are required.

Yeah, that was an overgeneralization on my part. I meant "the only  
field other than TY and ER". There are asterisks after "Pub Date" on  
Abstract, Art Work, Audiovisual Material, and most of the other dates  
I looked at, indicating they are mandatory. But reading the document  
more closely, I see that they are in fact optionally mandatory - you  
can use the Reference Editor to make them optional. Only TY and ER  
are mandatorily mandatory, so to speak.

> You said "finally" but I think we're only halfway done. The
> interesting part is to extend this mapping to the SQL
> storage.

Yes, but that's *your* problem, not mine. :-)

> Currently I use the (sometimes straightforward, sometimes
> butt ugly) mapping to the 37 default fields that RIS uses.

It only gets butt ugly if you think semantically. If you think of it  
from a purely engineering standpoint, it makes no difference where  
the fields get allocated - the labels attached to those 37 default  
fields are just arbitrary names. As long as each Reference Type can  
map its elements onto those 37 locations in memory, everything is  
copecetic.

> And yes, would you mind sharing your CSV file containing the RIS
> "spec"? I'd like to play with this a little towards RelaxNG.

No problem. I've attached, not the original CSV file, but rather a  
version that's been massaged using emacs macros, to propagate the  
field id's down into the field names, and with the beginnings of  
tagging in it. Let me know if you'd rather have the pristine version.

  - dam

Re: [Refdb-users] risx + ELEC

[Bruce D'A. <bda...@gm...>]

On 1/9/06, Markus Hoenicka <mar...@mh...> wrote:

>  > I think the level metaphor of TEI and RIS is wrong. Indeed, look at
>  > the revised bib model as implemented (I think!) in TEI 5, which is
>  > more like MODS or DC in structure.
>
> But you use it implicitly. All general types that you mentioned can be
> mapped to risx, and none of them requires the russian doll-like
> nesting that MODS allows.

It's true I use *a* level structure there, but it's different than the
RIS/TEI < 5 structure.  The primary level in this approach is always
the refernce item.

As for the > three-level cases, it's true they're not that common, but
they do exist. A part or chapter in a multi-volume book is an example
that I've seen in the Chicago Manual.

Bruce

[Refdb-users] preview of the XML RefDB manual

[Markus H. <mar...@mh...>]

Hi,

in an attempt to streamline the documentation, I've converted the SGML
manual as well as the man pages to XML. Now I can build the manual and
the man pages from the same DocBook sources. Besides, the tool
documentation is a bit more consistent due to the manpage
requirements.

If you have some time on your hands, please browse the new version:

http://refdb.sourceforge.net/refdb-manual-0.9.7/index.html

All relevant changes are in the Reference Manual (Section IV).

Package builders may also want to update their CVS and check whether
everything works ok. I'm afraid that I've broken something as the
changes of the build system were pretty fundamental.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[Refdb-users] RISY Report

[Markus H. <mar...@mh...>]

David Mundie writes:
 > The "obvious" problem with RIS is how to abstract away from the  
 > specific reference types that RIS defines. As Bruce put it, "there  
 > are general structures underneath the type-specific language", and I  
 > spent a frustrating couple of hours attempting to capture those  
 > abstractions in a RIS-compatible DTD. Finally, though, I had a  
 > breakthrough. Maybe that "obvious" problem isn't really a problem at  
 > all. Maybe the right thing to do is to treat the 33 RIS reference  
 > type as 33 distinct content models. Don't worry about inserting  
 > analytic, monograph, and serial containers. Don't play the old  
 > "<author role='Composer'" trick to collapse similar elements while  
 > retaining the distinctions in attributes. Heck, don't even insert a  
 > container "extent" element around the start page and end page elements.
 > 

You basically reduce the reference types to simple forms. This has its
charms as it would neatly transform to e.g. HTML forms for entering
data, but from a markup perspective I just don't like the result.

 > The current release makes all fields required, just because I wanted  
 > to postpone thinking about which ones should be optional. (Reference  
 > Manager is no help here - the only field it requires is the date,  
 > presumably for some implementation reason.)

As far as I understand the "spec", only TI and ER are required.

 > Finally, as I said earlier, it is trivial to generate a RIS document  
 > from a RISY document. For example, feeding the template document into  
 > the conversion script yields the following RIS declaration:
 > 
 > 	TY - ELEC
 > 	T1 - title
 > 	A1 - authors
 > 	Y1 - last_update
 > 	A2 - editors
 > 	SP - start_page
 > 	EP - end_page
 > 	JO - source
 > 	VL - volume
 > 	IS - edition
 > 	PB - publisher
 > 	Y2 - access_date
 > 	M1 - medium
 > 	M2 - unique_id
 > 	M3 - unique_id_doi
 > 

You said "finally" but I think we're only halfway done. The
interesting part is to extend this mapping to the SQL
storage. Currently I use the (sometimes straightforward, sometimes
butt ugly) mapping to the 37 default fields that RIS uses. I'd like to
change this and store data which are in fact different in different
tables/columns. This would include an extension to the query
language. The current method to query :M3: if you want to search for
the media type of audiovisual material is, at best, cryptic.

 > All of this in 35 lines of Gema code. I'm attaching the template  
 > document, the DTD, and the stylesheet. I'd be happy to share the  
 > scripts as well if anyone's interested.
 > 

Would it be hard to reimplement in a language that someone else has
interpreters for? 

And yes, would you mind sharing your CSV file containing the RIS
"spec"? I'd like to play with this a little towards RelaxNG.

regards,
Markus


-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-users] risx + ELEC

[Markus H. <mar...@mh...>]

Bruce D'Arcus writes:
 > This is true of RELAX NG, but not XML Schema.
 > 

I've read something about RelaxNG, but next to nothing about
Schema. For some reason I thought this was the case for all Schema
languages. I stand corrected.

 > > In fact, I believe the main distinction is whether a reference type
 > > requires or allows part and series data.
 > 
[...]
 > monograph (standalone item, published singly; not all electronic
 > resources fit here)

equivalent to publication

 > serial (someting published continually; periodicals, court reporters, etc.)

equivalent to serial

 > part-inMonograph

equivalent to publication with part info

 > part-inSerial

equivalent to publication with serial info

 > I think the level metaphor of TEI and RIS is wrong. Indeed, look at
 > the revised bib model as implemented (I think!) in TEI 5, which is
 > more like MODS or DC in structure.
 > 

But you use it implicitly. All general types that you mentioned can be
mapped to risx, and none of them requires the russian doll-like
nesting that MODS allows.

 > I think BibTeX is a hack form a data modelling perspective. It only
 > really works if you're a hard scientist, and even then can be a
 > problem.
 > 

I guess this depends on the branch you're in. Lots of journals
publishing papers in maths or physics accept LaTeX, and I can't
imagine they demand fancier bibliography data than bibtex can
handle. But I agree, the further you move from math and physics, the
less suited bibtex is.

regards
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [Refdb-users] RISY Report

[Bruce D'A. <bda...@gm...>]

On 1/9/06, David Mundie <mu...@an...> wrote:

> Does it include all of the data from the RefMan manual?
>
> Yes. As I say, it was *all* generated automatically from the text of the
> Reference Manager documentation. I may have introduced errors when
> transcribing it from the blinkety-blank PDF into Excel, but the intent wa=
s
> to be exhaustive.

Good.

I had this thought that I mentioned to the author of bibutils that it
might be nice to have soemthing like this to represent the (c-based)
mapping tables he has, and to automatically generate the code from
them.  That way, people could port it to other languages, while
maintaining consistent logic (which is the hard part anyway).

> I'll try to re-open my mind on this issue. At the time it came out, it
> certainly seemed to me that what they were saying was "Let's throw out th=
e
> DDC, the AACR2, and USMARC, and start over building a dumbed-down set of
> simple primitives that we think the average computer jock could use." I
> never doubted that it would do a better job than RIS, just that it would =
do
> a better job than the professional cataloguing tools.
>
> And I'm sorry to hear you describe MODS as "DC on steroids". Up until I r=
ead
> that, I had thought of MODS as "USMARC on steroids", which I find more
> appealing.

Actually, come to think of it, I'd say we're both right (though MODS
is rather slimmer than MARCXML). I was just referrring to the basic
structrure, but there's clearly a lot from MARC as well.

Bruce

Re: [Refdb-users] RISY Report

[David M. <mu...@an...>]

On Jan 9, 2006, at 9:36 AM, Bruce D'Arcus wrote:

> Your template is useful.

Thanks! It was certainly fun to implement.

> Does it include all of the data from the RefMan manual?

Yes. As I say, it was *all* generated automatically from the text of  
the Reference Manager documentation. I may have introduced errors  
when transcribing it from the blinkety-blank PDF into Excel, but the  
intent was to be exhaustive.

> I really dislike names like "book_title" and "chapter_title."  Why not
> just call them both "title" and, in the case of the chapter, call the
> book title a "containerr_title"?

See above. The Reference Manager Manual calls it a "Book Title", so  
RISY calls it a "book_title". I would be really loathe to give up the  
isomorphism between RISY and RIS - it's part of its charm.

> If you think terms of scripting code to access this stuff, compare:
>
> print book.title
>
> ... with:
>
> print book.book_title

Agreed. I know all about context-sensitive markup, and realize its  
value. But again, the transparent mapping onto RIS trumps it.

> And if you're going to use XML, get rid of the obtuse type acronyms.

Funny you should mention that. About 20 minutes after I posted to the  
list, I did an internal release of version 1.1 of RISY, which retains  
the expanded names ("Electronic Communication" instead of "ELEC",  
etc.) as an attribute and displays it in the styled output. To  
preserve RIS equivalence, I obviously can't get rid of the "obtuse  
type acronyms" entirely, but they bothered me in the displayed output  
as much as they did you.

> In any case, I still think there's somerthing fundamentally wrong
> about the RIS model ;-)

Dunno. Having a fixed-length set of fields that you overload for  
different purposes is certainly straight out of the 1960's. To me,  
USMARC has a lot of the same flavor to it - designed back in a  
different era.

But to repeat, RIS provides a convenient set of reference types at an  
intermediate level of specificity whose equivalent I haven't found  
elsewhere.

> I'm sorry, but you have this totally wrong. DC is mostly a product of
> the library community, and is a rich collabrative effort. It's basic
> data model is actually pretty good for a baseline standard (you could
> think of MODS as DC on steroids), and when used with RDF (where you
> can mix in additional structures as needed)I think does a better job
> than RIS, which most definitely was not designed by library people.

I'll try to re-open my mind on this issue. At the time it came out,  
it certainly seemed to me that what they were saying was "Let's throw  
out the DDC, the AACR2, and USMARC, and start over building a dumbed- 
down set of simple primitives that we think the average computer jock  
could use." I never doubted that it would do a better job than RIS,  
just that it would do a better job than the professional cataloguing  
tools.

And I'm sorry to hear you describe MODS as "DC on steroids". Up until  
I read that, I had thought of MODS as "USMARC on steroids", which I  
find more appealing.

  - dam