refdb-users

[Refdb-users] ref ID, user defined? + some plans (long)

[Paul B. <pbo...@sc...>]

Hi Markus & other developers,

Thanks for your refdb open source project, it appears to get most of the 
basic features we may need :-)

We want to implement a "distributed" system on top of a central 
repository (implemented with refdb) with the following properties:

- store all our organisations references centrally in refdb with a 
unique persistent ID
- provide a service to end-users to keep there existing reference 
applications (reference manager, endnote and bibtex) and word processors
- use the central repository of references also in a library portal and 
in the future also use them inside xml documents which will reside in 
different portals.

1) To reach this goal there are is one major issue:
We need both import and export from/to *existing* reference systems and 
databases ( these range from personal "databases" to groups who are 
using a common database/system.). To maintain backwards compatibility, 
the existing keys should be preserved in order not to break references 
in existing documents. I read in previous posts and the manual that user 
defined ID's are not supported or retained when importing.

Do you plan to integrate this in some way? Perhaps as an additional 
database field other than the default one and optionally coupled to the 
user? Of course this should be also an export option to select the key 
to use with a fall back to the refdb ID if no such key exists

A workaround for us could be to import/export them from refdb and use 
these databases only for new documents, but this is rather confusing and 
some users tend to define keys which hint to the contents of a certain 
reference (mainly latex users). Or do you think the use of U1..U5 and 
/or M1..M3 could already serve this purpose (in both directions) with 
additional filter stages / config options (for bibtex this seems to be 
the case, but what about the others)?

2) We will also develop a refdb SOAP service (in php, calling the refdb 
client utilities) which can be called by various applications (like the 
portals we plan to develop). Initially this will be a read-only service. 
I think other people may be interested in this as well (and we'll 
release the code of course).

I would welcome feedback, especially on the ID/key issues

Best regards

Paul

[Refdb-users] ref ID, user defined? + some plans (long)

[Markus H. <hoe...@co...>]

Hi Paul,

Paul Borgermans writes:
 > 1) To reach this goal there are is one major issue:
 > We need both import and export from/to *existing* reference systems and 
 > databases ( these range from personal "databases" to groups who are 
 > using a common database/system.). To maintain backwards compatibility, 
 > the existing keys should be preserved in order not to break references 
 > in existing documents. I read in previous posts and the manual that user 
 > defined ID's are not supported or retained when importing.
 > 
 > Do you plan to integrate this in some way? Perhaps as an additional 
 > database field other than the default one and optionally coupled to the 
 > user? Of course this should be also an export option to select the key 
 > to use with a fall back to the refdb ID if no such key exists
 > 

I don't plan to add this feature to the 0.8.x series, but it is
actually implemented in the libdbi-pre1 CVS branch that will
eventually be released as 0.9. I checked in the changes just today.

The current implementation will try to use existing citation keys
(supplied as an alphanumeric string in the ID field of the RIS
dataset) and generate an error if the citation key already exists (it
has to be unique to make sense). If no citation key is supplied by the
user, RefDB will automatically generate one using the first author and
the publication year, followed by a sequential suffix to make it
unique if necessary. The only restriction is that the citation keys
must not be entirely numeric, i.e. "44" would not be allowed but
"Miller1999" would be. Please let me know whether this implementation
would match your needs.

 > 2) We will also develop a refdb SOAP service (in php, calling the refdb 
 > client utilities) which can be called by various applications (like the 
 > portals we plan to develop). Initially this will be a read-only service. 
 > I think other people may be interested in this as well (and we'll 
 > release the code of course).

I thought about SOAP myself, but I'd be more than happy if someone
beats me at it. Let me know if you need any changes on the RefDB side
to make this work.

BTW, what organization are you working for? Is that some University
project or something similar?

regards,
Markus

-- 
Markus Hoenicka <hoe...@co...>
http://ourworld.compuserve.com/homepages/hoenicka_markus/

Re: [Refdb-users] ref ID, user defined? + some plans (long)

[Paul B. <pbo...@sc...>]

Hi Markus

Markus Hoenicka wrote:
> Hi Paul,
> 
> Paul Borgermans writes:
>  > 1) To reach this goal there are is one major issue:
>  > We need both import and export from/to *existing* reference systems and 
>  > databases ( these range from personal "databases" to groups who are 
>  > using a common database/system.). To maintain backwards compatibility, 
>  > the existing keys should be preserved in order not to break references 
>  > in existing documents. I read in previous posts and the manual that user 
>  > defined ID's are not supported or retained when importing.
>  > 
>  > Do you plan to integrate this in some way? Perhaps as an additional 
>  > database field other than the default one and optionally coupled to the 
>  > user? Of course this should be also an export option to select the key 
>  > to use with a fall back to the refdb ID if no such key exists
>  > 
> 
> I don't plan to add this feature to the 0.8.x series, but it is
> actually implemented in the libdbi-pre1 CVS branch that will
> eventually be released as 0.9. I checked in the changes just today.
> 
> The current implementation will try to use existing citation keys
> (supplied as an alphanumeric string in the ID field of the RIS
> dataset) and generate an error if the citation key already exists (it
> has to be unique to make sense). If no citation key is supplied by the
> user, RefDB will automatically generate one using the first author and
> the publication year, followed by a sequential suffix to make it
> unique if necessary. The only restriction is that the citation keys
> must not be entirely numeric, i.e. "44" would not be allowed but
> "Miller1999" would be. Please let me know whether this implementation
> would match your needs.

Great! That comes close. I see some users here using reference manager 
and not caring about ID's, so they end up with an entirely numeric key. 
I'll write pre/post processing filters for it if necessary. It is going 
to be in some automated scripts for upload / download from the refdb 
repository after all.

>  > 2) We will also develop a refdb SOAP service (in php, calling the refdb 
>  > client utilities) which can be called by various applications (like the 
>  > portals we plan to develop). Initially this will be a read-only service. 
>  > I think other people may be interested in this as well (and we'll 
>  > release the code of course).
> 
> I thought about SOAP myself, but I'd be more than happy if someone
> beats me at it. Let me know if you need any changes on the RefDB side
> to make this work.

I don't think there need to be changes made for now. Let me stress it 
will be a read-only service to start with. It will be used in say a 
department portal for listing all their relevant publications, extracted 
from the refdb driven (central) repository.


> BTW, what organization are you working for? Is that some University
> project or something similar?
> 

Similar I guess: a nuclear (no military) research organisation.

More about our project (a multi-year thingie): we try to integrate and 
put on-line a lot of our documents and research data, transforming most 
to XML in the next years. On top of that, we are implementing an open 
source portal system (multiple actually) based on eZ publish 
<http://developer.ez.no> and later maybe also apache cocoon in the 
backend to do some special XSLT. The portal systems should make it 
easier to find information and will also be used to capture as much 
knowledge as we can (300 scientists, 300 staff personnel). This is an 
increasing problem for universities and large organisations: a lot gets 
lost over time. It may be there somewhere on paper, but if one does not 
know it's there, then nobody will find it.

The basis to choose XML formats as storage formats, beside its intrinsic 
advantages of structuring data and meta-data, is  also to let it remain 
useful over many years. Our hypothesis is that if it is in XML and we 
need other formats, just transform it. If they (XML variants) are clear 
and well thought, this process should be relatively painless and feasible.

Returning to refdb: we'll integrate it with the SOAP services and 
clients into the portal system as an ez publish module (client part that 
is). This means it will also need to be linked to the ez publish 
document (object) repository itself. The URL (location) field in the 
database propbably is OK for this.

I hope this clarifies a little bit

Regards

Paul