[VuFind-Tech] Using a defined format for display instead of MARC

[Jakob V. <jak...@gb...>]

Eric Lease Morgan wrote:

>> I like vufind very much, imported a few thousand MARC21 records  
>> successfully and want to import records from other data sources as  
>> well.

[...]

> It seems to me that VUFind was designed around MARC records. While I  
> have very seriously considered indexing stuff directly to the  
> underlying Solr index, I have also seen that many of the displays pull  
> from MARC-ish fields. Many of the specialized views (Holdings,  
> Description, and Staff) come to mind. Converting your metadata to MARC  
> (ick) and stuffing the result into VUFind seems like a viable  
> solution. Not necessarily ideal but functional.

As long as VuFind is stuck to MARC format, it will remain a dead end. 
For our applications we could use PICA+ format but I think it's the same 
garbage. A simple, general tree structure like JSON should be easy 
enough to work with - and it can be understood by non-library nerds. No 
subfields, indicators, controlfields and such - just fields and (single 
or multiple) values.

For most cases a flat structure of unordered fields with repeatable, 
sorted values should be enough:

<WS> ::= %x20 | %x09 | %x0A | %x0D

<RECORD> ::= <WS>* '{' <MEMBER>+ '}' <WS>*

<MEMBER> ::= <WS>* <FIELD> <WS>* ':' <WS>* ( <LIST> | <VALUE> ) <WS>*

<LIST> ::= '[' <WS>* <VALUE> ( <SEPARATOR> <VALUE> )* <WS>* ']'

<SEPARATOR> ::= <WS>* ',' <WS>*

<VALUE> ::= '"' <escaped-unicode-string> '"'

<FIELD> ::= '"' <escaped-unicode-string> '"'

FIELDs must be unique per record and their order is irrelevant. Lists 
must not be empty - just omit the field to become empty/undefined.

This is a pure subset of JSON. If you like you can define a similar 
structure in XML, YAML or even RDF. If should be good practise to use 
URIs as fields but you can also use plain strings. Your display 
component can pick and order the fields it knows. On worst case just 
display all fields and their values in random order.

If needed we could use nested structures (full JSON) or more specific 
formats defined by XML Schema, YAML Schema, RDFS, OWLS whatever. Unless 
there is no formal schema of MARC[*] or whatever format you should 
better use a simpler format.

Greetings,
Jakob

[*] There is no formal MARC schema. There is an XML Schema of MARCXML 
but this schema only describes the general structure of fields, but not 
*which* fields can be / must be present.

-- 
Jakob Voß <jak...@gb...>, skype: nichtich
Verbundzentrale des GBV (VZG) / Common Library Network
Platz der Goettinger Sieben 1, 37073 Göttingen, Germany
+49 (0)551 39-10242, http://www.gbv.de