From: Eoghan Ó C. <eog...@gm...> - 2011-07-27 15:31:00
|
Hi, We hope to do some work on a Collections Module over the next few weeks & wanted to run our current plans past the list, so hopefully our design will be reusable in the trunk. Requirements: 1) A splash page for the collection to include some or all of the following - Collection name and any available collection-level metadata/descriptions - Facets & facet counts for items in the collection, e.g. display all topics, persons, genre etc from Solr (something like what the Open Library display for their Work-level Records: http://openlibrary.org/works/OL8721462W/Great_Expectations) - Links to sub-collections where present (e.g. http://digital.library.villanova.edu/Catholica%20Collection/) - The first page of a result set for items in the collection (much like how the current author module works: e.g. http://vufind.org/demo_trunk/Author/Home?author=Yeats%2C%20W.%20B.%201865-1939 .) - An archival tree display where available (see http://vufind.org/jira/browse/VUFIND-421) - Human-readable collection URLS (e.g. http://www.vufind.org/Collections/The+VuFind+Archive - Any other widget ... 2) A means of browsing/searching a list of collections Approach: 1) The cleanest design seems to be to create a new Vufind module (ie /web/services/Collections). In terms of where to get the collection data from, we were thinking of having collection-level records (ie the absolute parent record for the collection and records for any sub-collections) in the standard Solr biblio core. These records/Solr-documents would carry all the collection-level metadata need by the splash page. They could either be hidden from the normal result set or, when presented in a normal result, would link to the collection splash page rather than a normal record page. We would also need to use two Solr fields, the existing "collection" field and a new "head_of_collection" field. The collection field will be populated for all items in a collection and will contain a string of the title of the parent collection (i.e. either the name of the overall collection or the name of the sub-collection the item is in). The "head_of_collection" field will only be populated by the collection-level record and any sub-collection-level records, & will again contain the name of that collection. These values should allow us to display the correct information on the collection splash page. They will also allow easy faceting on items in a given collection and, with some new apache rewrite rules, should allow urls of the form http://vufind.org/Collections/Name+From+Solr+Collection+Field. In MARC, we expect to populate these values using a custom beanshell script using values from 773 tags and/or the LEADER/07 (The National Library of Australia uses 773 and other tags to achieve something similar again with records in the biblio index, see http://catalogue.nla.gov.au/Record/4664435). Using Solr fields as the filter should hopefully make this pretty reusable for non-marc data, so long as you can map your collection info into the biblio Solr fields and populate these two filters fields discussed above. The Collection module uses CollectionRecord.php(like record uses record.php, but simpler in that we don’t envisiage different tabs at the moment) which will look up the item bib_id from the collection title in the url. CollectionRecord.php can then call a new RecordDriver function called getCollectionMetadata() from IndexRecord to retrieve information from the collection level record. CollectionRecord also has a Solr SearchObject child records for the initial result set (like the Author module) and things like facet counts etc. 2) A browse interface for collection-level recods should be reasonably straight-forward using the two Solr fields above. Any thoughts or observations very welcome. David, we'd be particularly interested to hear how this fits with VuDL/Vufind plans. We don't want to reinvent the wheel (or pre-invent it badly!). Thanks Eoghan |
From: Osullivan L. <L.O...@sw...> - 2011-07-27 16:03:39
|
Hi Eoghan, That looks great J Getting EAD records out of Calm is proving to be frustrating. We are therefore going to go with flat XML files which we will feed into solr and use to generate turn into tree (kind of the reverse of what Eric has been doing). I think that process should integrate with what you propose. Thanks, Luke From: Eoghan Ó Carragáin [mailto:eog...@gm...] Sent: 27 July 2011 16:31 To: vuf...@li...; David Lacy Subject: [VuFind-Tech] Collections Module outline Hi, We hope to do some work on a Collections Module over the next few weeks & wanted to run our current plans past the list, so hopefully our design will be reusable in the trunk. Requirements: 1) A splash page for the collection to include some or all of the following * Collection name and any available collection-level metadata/descriptions * Facets & facet counts for items in the collection, e.g. display all topics, persons, genre etc from Solr (something like what the Open Library display for their Work-level Records: http://openlibrary.org/works/OL8721462W/Great_Expectations) * Links to sub-collections where present (e.g. http://digital.library.villanova.edu/Catholica%20Collection/) * The first page of a result set for items in the collection (much like how the current author module works: e.g. http://vufind.org/demo_trunk/Author/Home?author=Yeats%2C%20W.%20B.%201865-1939.) * An archival tree display where available (see http://vufind.org/jira/browse/VUFIND-421) * Human-readable collection URLS (e.g. http://www.vufind.org/Collections/The+VuFind+Archive * Any other widget ... 2) A means of browsing/searching a list of collections Approach: 1) The cleanest design seems to be to create a new Vufind module (ie /web/services/Collections). In terms of where to get the collection data from, we were thinking of having collection-level records (ie the absolute parent record for the collection and records for any sub-collections) in the standard Solr biblio core. These records/Solr-documents would carry all the collection-level metadata need by the splash page. They could either be hidden from the normal result set or, when presented in a normal result, would link to the collection splash page rather than a normal record page. We would also need to use two Solr fields, the existing "collection" field and a new "head_of_collection" field. The collection field will be populated for all items in a collection and will contain a string of the title of the parent collection (i.e. either the name of the overall collection or the name of the sub-collection the item is in). The "head_of_collection" field will only be populated by the collection-level record and any sub-collection-level records, & will again contain the name of that collection. These values should allow us to display the correct information on the collection splash page. They will also allow easy faceting on items in a given collection and, with some new apache rewrite rules, should allow urls of the form http://vufind.org/Collections/Name+From+Solr+Collection+Field. In MARC, we expect to populate these values using a custom beanshell script using values from 773 tags and/or the LEADER/07 (The National Library of Australia uses 773 and other tags to achieve something similar again with records in the biblio index, see http://catalogue.nla.gov.au/Record/4664435). Using Solr fields as the filter should hopefully make this pretty reusable for non-marc data, so long as you can map your collection info into the biblio Solr fields and populate these two filters fields discussed above. The Collection module uses CollectionRecord.php(like record uses record.php, but simpler in that we don't envisiage different tabs at the moment) which will look up the item bib_id from the collection title in the url. CollectionRecord.php can then call a new RecordDriver function called getCollectionMetadata() from IndexRecord to retrieve information from the collection level record. CollectionRecord also has a Solr SearchObject child records for the initial result set (like the Author module) and things like facet counts etc. 2) A browse interface for collection-level recods should be reasonably straight-forward using the two Solr fields above. Any thoughts or observations very welcome. David, we'd be particularly interested to hear how this fits with VuDL/Vufind plans. We don't want to reinvent the wheel (or pre-invent it badly!). Thanks Eoghan |
From: Alan R. <ala...@mn...> - 2011-07-27 16:32:53
|
Hello Eoghan, Since we do something similar with our MnPALS implementation I'm very interested in this. I'm also looking at doing something similar for a library we will be bringing up shortly(mid August) on VuFind 1.1. I do have a couple of comments. Instead of creating an additional Solr field, couldn't you use some combination of the building, institution, and collection fields? Since you will need a record driver, using the current fields would allow searching everything at once. This is something that the library we are bringing up would like. It would still allow you to have the other features you are looking at. Searches could be limited by institution or something. We do that with something like this: http://plus.mnpals.net/catalog/MSU The record driver would know how to handle the fields. Adding the ability to limit searches to an institution(think collection level) and collection(think head-of-collection) would be great. I noticed someone commented about EAD records. This is the type of records I plan on adding to VuFind. So again, I'm very interested. al On Wed, 2011-07-27 at 16:30 +0100, Eoghan Ó Carragáin wrote: > Hi, > We hope to do some work on a Collections Module over the next few > weeks & wanted to run our current plans past the list, so hopefully > our design will be reusable in the trunk. > > Requirements: > 1) A splash page for the collection to include some or all of the > following > > * Collection name and any available collection-level > metadata/descriptions > * Facets & facet counts for items in the collection, e.g. > display all topics, persons, genre etc from Solr (something > like what the Open Library display for their Work-level > Records: > http://openlibrary.org/works/OL8721462W/Great_Expectations) > * Links to sub-collections where present (e.g. > http://digital.library.villanova.edu/Catholica%20Collection/) > * The first page of a result set for items in the collection > (much like how the current author module works: e.g. > http://vufind.org/demo_trunk/Author/Home?author=Yeats%2C%20W.% > 20B.%201865-1939.) > * An archival tree display where available (see > http://vufind.org/jira/browse/VUFIND-421) > * Human-readable collection URLS (e.g. > http://www.vufind.org/Collections/The+VuFind+Archive > * Any other widget ... > 2) A means of browsing/searching a list of collections > > Approach: > 1) > The cleanest design seems to be to create a new Vufind module > (ie /web/services/Collections). > > In terms of where to get the collection data from, we were thinking of > having collection-level records (ie the absolute parent record for the > collection and records for any sub-collections) in the standard Solr > biblio core. These records/Solr-documents would carry all the > collection-level metadata need by the splash page. They could either > be hidden from the normal result set or, when presented in a normal > result, would link to the collection splash page rather than a normal > record page. > > We would also need to use two Solr fields, the existing "collection" > field and a new "head_of_collection" field. The collection field will > be populated for all items in a collection and will contain a string > of the title of the parent collection (i.e. either the name of the > overall collection or the name of the sub-collection the item is in). > The "head_of_collection" field will only be populated by the > collection-level record and any sub-collection-level records, & will > again contain the name of that collection. These values should allow > us to display the correct information on the collection splash page. > They will also allow easy faceting on items in a given collection and, > with some new apache rewrite rules, should allow urls of the form > http://vufind.org/Collections/Name+From+Solr+Collection+Field. In > MARC, we expect to populate these values using a custom beanshell > script using values from 773 tags and/or the LEADER/07 (The National > Library of Australia uses 773 and other tags to achieve something > similar again with records in the biblio index, see > http://catalogue.nla.gov.au/Record/4664435). Using Solr fields as the > filter should hopefully make this pretty reusable for non-marc data, > so long as you can map your collection info into the biblio Solr > fields and populate these two filters fields discussed above. > > > The Collection module uses CollectionRecord.php(like record uses > record.php, but simpler in that we don’t envisiage different tabs at > the moment) which will look up the item bib_id from the collection > title in the url. CollectionRecord.php can then call a new > RecordDriver function called getCollectionMetadata() from IndexRecord > to retrieve information from the collection level record. > CollectionRecord also has a Solr SearchObject child records for the > initial result set (like the Author module) and things like facet > counts etc. > > 2) A browse interface for collection-level recods should be reasonably > straight-forward using the two Solr fields above. > > Any thoughts or observations very welcome. David, we'd be particularly > interested to hear how this fits with VuDL/Vufind plans. We don't want > to reinvent the wheel (or pre-invent it badly!). > > Thanks > Eoghan > ------------------------------------------------------------------------------ > Got Input? Slashdot Needs You. > Take our quick survey online. Come on, we don't ask for help often. > Plus, you'll get a chance to win $100 to spend on ThinkGeek. > http://p.sf.net/sfu/slashdot-survey > _______________________________________________ > Vufind-tech mailing list > Vuf...@li... > https://lists.sourceforge.net/lists/listinfo/vufind-tech -- Alan Rykhus PALS, A Program of the Minnesota State Colleges and Universities (507)389-1975 ala...@mn... "It's hard to lead a cavalry charge if you think you look funny on a horse" ~ Adlai Stevenson |
From: Demian K. <dem...@vi...> - 2011-07-27 16:33:57
|
It sounds like you're on the right track. A few thoughts that might be helpful: - Is it safe to use the existing Solr collection facet? This has been a general-purpose facet for so long that I'm concerned that various institutions may have come up with different uses for the field. Repurposing it as something more specific may cause some conflicts that could be avoided by simply adding a new field - maybe "parent_collection". (I'm not totally convinced that this is necessary... but bringing it up for consideration). - How do you deal with duplicate collection names? For example, suppose you have a "John Smith" collection and a "Jane Doe" collection, and both have subcollections called "Letters." Do these collections essentially get merged together because they share the same name? Do you ensure uniqueness by creating (potentially very long) delimited strings? Am I completely missing the point here? It just seems to me that using descriptive strings as identifiers is potentially hazardous... but I understand why it is tempting. - I'd definitely encourage you to take a look at how VuFind 2.0 is shaping up; if you eventually plan on upgrading, it may be worth thinking about how the new code could fit into the new model. I'm sure there won't be any major problems in porting things over (after all, the whole point is to make the new version more developer-friendly and extensible than the old one)... but it never hurts to plan ahead a little bit. - Demian From: Eoghan Ó Carragáin [mailto:eog...@gm...] Sent: Wednesday, July 27, 2011 11:31 AM To: vuf...@li...; David Lacy Subject: [VuFind-Tech] Collections Module outline Hi, We hope to do some work on a Collections Module over the next few weeks & wanted to run our current plans past the list, so hopefully our design will be reusable in the trunk. Requirements: 1) A splash page for the collection to include some or all of the following * Collection name and any available collection-level metadata/descriptions * Facets & facet counts for items in the collection, e.g. display all topics, persons, genre etc from Solr (something like what the Open Library display for their Work-level Records: http://openlibrary.org/works/OL8721462W/Great_Expectations) * Links to sub-collections where present (e.g. http://digital.library.villanova.edu/Catholica%20Collection/) * The first page of a result set for items in the collection (much like how the current author module works: e.g. http://vufind.org/demo_trunk/Author/Home?author=Yeats%2C%20W.%20B.%201865-1939.) * An archival tree display where available (see http://vufind.org/jira/browse/VUFIND-421) * Human-readable collection URLS (e.g. http://www.vufind.org/Collections/The+VuFind+Archive * Any other widget ... 2) A means of browsing/searching a list of collections Approach: 1) The cleanest design seems to be to create a new Vufind module (ie /web/services/Collections). In terms of where to get the collection data from, we were thinking of having collection-level records (ie the absolute parent record for the collection and records for any sub-collections) in the standard Solr biblio core. These records/Solr-documents would carry all the collection-level metadata need by the splash page. They could either be hidden from the normal result set or, when presented in a normal result, would link to the collection splash page rather than a normal record page. We would also need to use two Solr fields, the existing "collection" field and a new "head_of_collection" field. The collection field will be populated for all items in a collection and will contain a string of the title of the parent collection (i.e. either the name of the overall collection or the name of the sub-collection the item is in). The "head_of_collection" field will only be populated by the collection-level record and any sub-collection-level records, & will again contain the name of that collection. These values should allow us to display the correct information on the collection splash page. They will also allow easy faceting on items in a given collection and, with some new apache rewrite rules, should allow urls of the form http://vufind.org/Collections/Name+From+Solr+Collection+Field. In MARC, we expect to populate these values using a custom beanshell script using values from 773 tags and/or the LEADER/07 (The National Library of Australia uses 773 and other tags to achieve something similar again with records in the biblio index, see http://catalogue.nla.gov.au/Record/4664435). Using Solr fields as the filter should hopefully make this pretty reusable for non-marc data, so long as you can map your collection info into the biblio Solr fields and populate these two filters fields discussed above. The Collection module uses CollectionRecord.php(like record uses record.php, but simpler in that we don't envisiage different tabs at the moment) which will look up the item bib_id from the collection title in the url. CollectionRecord.php can then call a new RecordDriver function called getCollectionMetadata() from IndexRecord to retrieve information from the collection level record. CollectionRecord also has a Solr SearchObject child records for the initial result set (like the Author module) and things like facet counts etc. 2) A browse interface for collection-level recods should be reasonably straight-forward using the two Solr fields above. Any thoughts or observations very welcome. David, we'd be particularly interested to hear how this fits with VuDL/Vufind plans. We don't want to reinvent the wheel (or pre-invent it badly!). Thanks Eoghan |
From: Eoghan Ó C. <eog...@gm...> - 2011-07-28 08:41:52
|
Thanks Demian. Comments inline below. Cheers, eoghan On 27 July 2011 17:33, Demian Katz <dem...@vi...> wrote: > It sounds like you’re on the right track. A few thoughts that might be > helpful:**** > > ** ** > > - Is it safe to use the existing Solr collection facet? This has been a > general-purpose facet for so long that I’m concerned that various > institutions may have come up with different uses for the field. > Repurposing it as something more specific may cause some conflicts that > could be avoided by simply adding a new field – maybe “parent_collection”. > (I’m not totally convinced that this is necessary… but bringing it up for > consideration). > ---- This makes sense. Two new fields it is. > **** > > - How do you deal with duplicate collection names? For example, suppose > you have a “John Smith” collection and a “Jane Doe” collection, and both > have subcollections called “Letters.” Do these collections essentially get > merged together because they share the same name? Do you ensure uniqueness > by creating (potentially very long) delimited strings? Am I completely > missing the point here? It just seems to me that using descriptive strings > as identifiers is potentially hazardous… but I understand why it is > tempting. > ---- Hmmm, I see your point. We don't expect to have any subcollections in our own catalogue, so made the assumption that collection names would always be unique. While this may be manageable in a given institution, it probably isn't safe for the trunk especially with consortial implementations etc. One thing that might be worth looking into is a wikipedia-esque disambiguation page. We'll have a think about how this might work > **** > > - I’d definitely encourage you to take a look at how VuFind 2.0 is shaping > up; if you eventually plan on upgrading, it may be worth thinking about how > the new code could fit into the new model. I’m sure there won’t be any > major problems in porting things over (after all, the whole point is to make > the new version more developer-friendly and extensible than the old one)… > but it never hurts to plan ahead a little bit. > ---- I failed, for the second time, to get Vufind 2.0 running on windows. I agree it would be good to plan ahead. Can you say a little bit more about how the architecture changes might apply here? Either way, we'll certainly develop this agains 1.x to begin with. > **** > > ** ** > > - Demian**** > > ** ** > > *From:* Eoghan Ó Carragáin [mailto:eog...@gm...] > *Sent:* Wednesday, July 27, 2011 11:31 AM > > *To:* vuf...@li...; David Lacy > *Subject:* [VuFind-Tech] Collections Module outline**** > > ** ** > > Hi, > > We hope to do some work on a Collections Module over the next few weeks & > wanted to run our current plans past the list, so hopefully our design will > be reusable in the trunk. > > Requirements: > 1) A splash page for the collection to include some or all of the following > **** > > > - Collection name and any available collection-level > metadata/descriptions**** > - Facets & facet counts for items in the collection, e.g. display all > topics, persons, genre etc from Solr (something like what the Open Library > display for their Work-level Records: > http://openlibrary.org/works/OL8721462W/Great_Expectations)**** > - Links to sub-collections where present (e.g. > http://digital.library.villanova.edu/Catholica%20Collection/)**** > - The first page of a result set for items in the collection (much like > how the current author module works: e.g. > http://vufind.org/demo_trunk/Author/Home?author=Yeats%2C%20W.%20B.%201865-1939 > .)**** > - An archival tree display where available (see > http://vufind.org/jira/browse/VUFIND-421)**** > - Human-readable collection URLS (e.g. > http://www.vufind.org/Collections/The+VuFind+Archive**** > - Any other widget ...**** > > 2) A means of browsing/searching a list of collections > > Approach: > 1) > The cleanest design seems to be to create a new Vufind module (ie > /web/services/Collections). > > In terms of where to get the collection data from, we were thinking of > having collection-level records (ie the absolute parent record for the > collection and records for any sub-collections) in the standard Solr biblio > core. These records/Solr-documents would carry all the collection-level > metadata need by the splash page. They could either be hidden from the > normal result set or, when presented in a normal result, would link to the > collection splash page rather than a normal record page. > > We would also need to use two Solr fields, the existing "collection" field > and a new "head_of_collection" field. The collection field will be populated > for all items in a collection and will contain a string of the title of the > parent collection (i.e. either the name of the overall collection or the > name of the sub-collection the item is in). The "head_of_collection" field > will only be populated by the collection-level record and any > sub-collection-level records, & will again contain the name of that > collection. These values should allow us to display the correct information > on the collection splash page. They will also allow easy faceting on items > in a given collection and, with some new apache rewrite rules, should allow > urls of the form > http://vufind.org/Collections/Name+From+Solr+Collection+Field. In MARC, > we expect to populate these values using a custom beanshell script using > values from 773 tags and/or the LEADER/07 (The National Library of Australia > uses 773 and other tags to achieve something similar again with records in > the biblio index, see http://catalogue.nla.gov.au/Record/4664435). Using > Solr fields as the filter should hopefully make this pretty reusable for > non-marc data, so long as you can map your collection info into the biblio > Solr fields and populate these two filters fields discussed above.**** > > The Collection module uses CollectionRecord.php(like record uses > record.php, but simpler in that we don’t envisiage different tabs at the > moment) which will look up the item bib_id from the collection title in the > url. CollectionRecord.php can then call a new RecordDriver function called > getCollectionMetadata() from IndexRecord to retrieve information from the > collection level record. CollectionRecord also has a Solr SearchObject > child records for the initial result set (like the Author module) and things > like facet counts etc. > > 2) A browse interface for collection-level recods should be reasonably > straight-forward using the two Solr fields above. > > Any thoughts or observations very welcome. David, we'd be particularly > interested to hear how this fits with VuDL/Vufind plans. We don't want to > reinvent the wheel (or pre-invent it badly!). > > Thanks > Eoghan **** > |
From: Demian K. <dem...@vi...> - 2011-07-27 16:37:44
|
Regarding EAD files, have you seen Eric Lease Morgan's work on the subject? http://www.catholicresearch.net/blog/2010/10/indexing-ead/ It uses Perl scripts and in some ways is tailored to the CRRA portal, but perhaps you can reuse some of it. If you come up with a more generic, PHP-oriented variation, I'd definitely be interested in getting more EAD tools into the trunk. - Demian > -----Original Message----- > From: Alan Rykhus [mailto:ala...@mn...] > Sent: Wednesday, July 27, 2011 12:33 PM > To: vuf...@li... > Subject: Re: [VuFind-Tech] Collections Module outline > > Hello Eoghan, > > Since we do something similar with our MnPALS implementation I'm very > interested in this. I'm also looking at doing something similar for a > library we will be bringing up shortly(mid August) on VuFind 1.1. > > I do have a couple of comments. > > Instead of creating an additional Solr field, couldn't you use some > combination of the building, institution, and collection fields? Since > you will need a record driver, using the current fields would allow > searching everything at once. This is something that the library we are > bringing up would like. It would still allow you to have the other > features you are looking at. Searches could be limited by institution > or > something. We do that with something like this: > http://plus.mnpals.net/catalog/MSU > > The record driver would know how to handle the fields. Adding the > ability to limit searches to an institution(think collection level) and > collection(think head-of-collection) would be great. > > I noticed someone commented about EAD records. This is the type of > records I plan on adding to VuFind. So again, I'm very interested. > > al > > On Wed, 2011-07-27 at 16:30 +0100, Eoghan Ó Carragáin wrote: > > Hi, > > We hope to do some work on a Collections Module over the next few > > weeks & wanted to run our current plans past the list, so hopefully > > our design will be reusable in the trunk. > > > > Requirements: > > 1) A splash page for the collection to include some or all of the > > following > > > > * Collection name and any available collection-level > > metadata/descriptions > > * Facets & facet counts for items in the collection, e.g. > > display all topics, persons, genre etc from Solr (something > > like what the Open Library display for their Work-level > > Records: > > http://openlibrary.org/works/OL8721462W/Great_Expectations) > > * Links to sub-collections where present (e.g. > > http://digital.library.villanova.edu/Catholica%20Collection/) > > * The first page of a result set for items in the collection > > (much like how the current author module works: e.g. > > > http://vufind.org/demo_trunk/Author/Home?author=Yeats%2C%20W.% > > 20B.%201865-1939.) > > * An archival tree display where available (see > > http://vufind.org/jira/browse/VUFIND-421) > > * Human-readable collection URLS (e.g. > > http://www.vufind.org/Collections/The+VuFind+Archive > > * Any other widget ... > > 2) A means of browsing/searching a list of collections > > > > Approach: > > 1) > > The cleanest design seems to be to create a new Vufind module > > (ie /web/services/Collections). > > > > In terms of where to get the collection data from, we were thinking > of > > having collection-level records (ie the absolute parent record for > the > > collection and records for any sub-collections) in the standard Solr > > biblio core. These records/Solr-documents would carry all the > > collection-level metadata need by the splash page. They could either > > be hidden from the normal result set or, when presented in a normal > > result, would link to the collection splash page rather than a normal > > record page. > > > > We would also need to use two Solr fields, the existing "collection" > > field and a new "head_of_collection" field. The collection field will > > be populated for all items in a collection and will contain a string > > of the title of the parent collection (i.e. either the name of the > > overall collection or the name of the sub-collection the item is in). > > The "head_of_collection" field will only be populated by the > > collection-level record and any sub-collection-level records, & will > > again contain the name of that collection. These values should allow > > us to display the correct information on the collection splash page. > > They will also allow easy faceting on items in a given collection > and, > > with some new apache rewrite rules, should allow urls of the form > > http://vufind.org/Collections/Name+From+Solr+Collection+Field. In > > MARC, we expect to populate these values using a custom beanshell > > script using values from 773 tags and/or the LEADER/07 (The National > > Library of Australia uses 773 and other tags to achieve something > > similar again with records in the biblio index, see > > http://catalogue.nla.gov.au/Record/4664435). Using Solr fields as the > > filter should hopefully make this pretty reusable for non-marc data, > > so long as you can map your collection info into the biblio Solr > > fields and populate these two filters fields discussed above. > > > > > > The Collection module uses CollectionRecord.php(like record uses > > record.php, but simpler in that we don’t envisiage different tabs at > > the moment) which will look up the item bib_id from the collection > > title in the url. CollectionRecord.php can then call a new > > RecordDriver function called getCollectionMetadata() from IndexRecord > > to retrieve information from the collection level record. > > CollectionRecord also has a Solr SearchObject child records for the > > initial result set (like the Author module) and things like facet > > counts etc. > > > > 2) A browse interface for collection-level recods should be > reasonably > > straight-forward using the two Solr fields above. > > > > Any thoughts or observations very welcome. David, we'd be > particularly > > interested to hear how this fits with VuDL/Vufind plans. We don't > want > > to reinvent the wheel (or pre-invent it badly!). > > > > Thanks > > Eoghan > > --------------------------------------------------------------------- > --------- > > Got Input? Slashdot Needs You. > > Take our quick survey online. Come on, we don't ask for help often. > > Plus, you'll get a chance to win $100 to spend on ThinkGeek. > > http://p.sf.net/sfu/slashdot-survey > > _______________________________________________ > > Vufind-tech mailing list > > Vuf...@li... > > https://lists.sourceforge.net/lists/listinfo/vufind-tech > > -- > Alan Rykhus > PALS, A Program of the Minnesota State Colleges and Universities > (507)389-1975 > ala...@mn... > "It's hard to lead a cavalry charge if you think you look funny on a > horse" ~ Adlai Stevenson > > > ----------------------------------------------------------------------- > ------- > Got Input? Slashdot Needs You. > Take our quick survey online. Come on, we don't ask for help often. > Plus, you'll get a chance to win $100 to spend on ThinkGeek. > http://p.sf.net/sfu/slashdot-survey > _______________________________________________ > Vufind-tech mailing list > Vuf...@li... > https://lists.sourceforge.net/lists/listinfo/vufind-tech |
From: Eoghan Ó C. <eog...@gm...> - 2011-07-28 08:31:56
|
Hi Alan, Thanks for the feedback. We were thinking more on the level of sets of records forming part of a collection within a single institution rather than collections being at institution level, but I see that it might make sense to handle them the same way. Could you say a little bit more about your use case & your current implementation? Re reusing the building, institution, collection fields, Demian's point about possibly breaking some people's catalogue because of a particular use they've applied to these old fields seems to apply here too. Probably safest to create new solr fields altogether. Re EAD & display of archives, I'm aware of at least 4 institutions currently using or intending to use Vufind for archical material. We're hoping to get a standard way of displaying the hierarchical relationship into the trunk too. See http://vufind.org/jira/browse/VUFIND-421 Cheers, Eoghan On 27 July 2011 17:32, Alan Rykhus <ala...@mn...> wrote: > Hello Eoghan, > > Since we do something similar with our MnPALS implementation I'm very > interested in this. I'm also looking at doing something similar for a > library we will be bringing up shortly(mid August) on VuFind 1.1. > > I do have a couple of comments. > > Instead of creating an additional Solr field, couldn't you use some > combination of the building, institution, and collection fields? Since > you will need a record driver, using the current fields would allow > searching everything at once. This is something that the library we are > bringing up would like. It would still allow you to have the other > features you are looking at. Searches could be limited by institution or > something. We do that with something like this: > http://plus.mnpals.net/catalog/MSU > > The record driver would know how to handle the fields. Adding the > ability to limit searches to an institution(think collection level) and > collection(think head-of-collection) would be great. > > I noticed someone commented about EAD records. This is the type of > records I plan on adding to VuFind. So again, I'm very interested. > > al > > On Wed, 2011-07-27 at 16:30 +0100, Eoghan Ó Carragáin wrote: > > Hi, > > We hope to do some work on a Collections Module over the next few > > weeks & wanted to run our current plans past the list, so hopefully > > our design will be reusable in the trunk. > > > > Requirements: > > 1) A splash page for the collection to include some or all of the > > following > > > > * Collection name and any available collection-level > > metadata/descriptions > > * Facets & facet counts for items in the collection, e.g. > > display all topics, persons, genre etc from Solr (something > > like what the Open Library display for their Work-level > > Records: > > http://openlibrary.org/works/OL8721462W/Great_Expectations) > > * Links to sub-collections where present (e.g. > > http://digital.library.villanova.edu/Catholica%20Collection/) > > * The first page of a result set for items in the collection > > (much like how the current author module works: e.g. > > http://vufind.org/demo_trunk/Author/Home?author=Yeats%2C%20W.% > > 20B.%201865-1939.) > > * An archival tree display where available (see > > http://vufind.org/jira/browse/VUFIND-421) > > * Human-readable collection URLS (e.g. > > http://www.vufind.org/Collections/The+VuFind+Archive > > * Any other widget ... > > 2) A means of browsing/searching a list of collections > > > > Approach: > > 1) > > The cleanest design seems to be to create a new Vufind module > > (ie /web/services/Collections). > > > > In terms of where to get the collection data from, we were thinking of > > having collection-level records (ie the absolute parent record for the > > collection and records for any sub-collections) in the standard Solr > > biblio core. These records/Solr-documents would carry all the > > collection-level metadata need by the splash page. They could either > > be hidden from the normal result set or, when presented in a normal > > result, would link to the collection splash page rather than a normal > > record page. > > > > We would also need to use two Solr fields, the existing "collection" > > field and a new "head_of_collection" field. The collection field will > > be populated for all items in a collection and will contain a string > > of the title of the parent collection (i.e. either the name of the > > overall collection or the name of the sub-collection the item is in). > > The "head_of_collection" field will only be populated by the > > collection-level record and any sub-collection-level records, & will > > again contain the name of that collection. These values should allow > > us to display the correct information on the collection splash page. > > They will also allow easy faceting on items in a given collection and, > > with some new apache rewrite rules, should allow urls of the form > > http://vufind.org/Collections/Name+From+Solr+Collection+Field. In > > MARC, we expect to populate these values using a custom beanshell > > script using values from 773 tags and/or the LEADER/07 (The National > > Library of Australia uses 773 and other tags to achieve something > > similar again with records in the biblio index, see > > http://catalogue.nla.gov.au/Record/4664435). Using Solr fields as the > > filter should hopefully make this pretty reusable for non-marc data, > > so long as you can map your collection info into the biblio Solr > > fields and populate these two filters fields discussed above. > > > > > > The Collection module uses CollectionRecord.php(like record uses > > record.php, but simpler in that we don’t envisiage different tabs at > > the moment) which will look up the item bib_id from the collection > > title in the url. CollectionRecord.php can then call a new > > RecordDriver function called getCollectionMetadata() from IndexRecord > > to retrieve information from the collection level record. > > CollectionRecord also has a Solr SearchObject child records for the > > initial result set (like the Author module) and things like facet > > counts etc. > > > > 2) A browse interface for collection-level recods should be reasonably > > straight-forward using the two Solr fields above. > > > > Any thoughts or observations very welcome. David, we'd be particularly > > interested to hear how this fits with VuDL/Vufind plans. We don't want > > to reinvent the wheel (or pre-invent it badly!). > > > > Thanks > > Eoghan > > > ------------------------------------------------------------------------------ > > Got Input? Slashdot Needs You. > > Take our quick survey online. Come on, we don't ask for help often. > > Plus, you'll get a chance to win $100 to spend on ThinkGeek. > > http://p.sf.net/sfu/slashdot-survey > > _______________________________________________ > > Vufind-tech mailing list > > Vuf...@li... > > https://lists.sourceforge.net/lists/listinfo/vufind-tech > > -- > Alan Rykhus > PALS, A Program of the Minnesota State Colleges and Universities > (507)389-1975 > ala...@mn... > "It's hard to lead a cavalry charge if you think you look funny on a > horse" ~ Adlai Stevenson > > > > ------------------------------------------------------------------------------ > Got Input? Slashdot Needs You. > Take our quick survey online. Come on, we don't ask for help often. > Plus, you'll get a chance to win $100 to spend on ThinkGeek. > http://p.sf.net/sfu/slashdot-survey > _______________________________________________ > Vufind-tech mailing list > Vuf...@li... > https://lists.sourceforge.net/lists/listinfo/vufind-tech > |
From: Alan R. <ala...@mn...> - 2011-07-28 14:31:57
|
Hello Eoghan, As you might know, our implementation is based on a branch from 0.8. I just took a harder look at our setup, it was done 2 or 3 years ago. I see I added 2 new fields. The following is what I use. collection - hard coded to "Catalog" as it still is today. institution - sub-library names - used in display and faceting building - library and sub-library codes - used to pre-limit searches to libraries and/or sub-libraries in our consortia I added the following shelf - collection name - used for display and faceting floor - collection code - used to pre-limit searches to sections of libraries I haven't done my research on EAD records yet. It is a project we are looking at for later this year. I should dig into this further before commenting too much. al On Thu, 2011-07-28 at 09:31 +0100, Eoghan Ó Carragáin wrote: > Hi Alan, > Thanks for the feedback. > > We were thinking more on the level of sets of records forming part of > a collection within a single institution rather than collections being > at institution level, but I see that it might make sense to handle > them the same way. Could you say a little bit more about your use case > & your current implementation? > > Re reusing the building, institution, collection fields, Demian's > point about possibly breaking some people's catalogue because of a > particular use they've applied to these old fields seems to apply here > too. Probably safest to create new solr fields altogether. > > Re EAD & display of archives, I'm aware of at least 4 institutions > currently using or intending to use Vufind for archical material. > We're hoping to get a standard way of displaying the hierarchical > relationship into the trunk too. See > http://vufind.org/jira/browse/VUFIND-421 > > Cheers, > Eoghan > > On 27 July 2011 17:32, Alan Rykhus <ala...@mn...> wrote: > Hello Eoghan, > > Since we do something similar with our MnPALS implementation > I'm very > interested in this. I'm also looking at doing something > similar for a > library we will be bringing up shortly(mid August) on VuFind > 1.1. > > I do have a couple of comments. > > Instead of creating an additional Solr field, couldn't you use > some > combination of the building, institution, and collection > fields? Since > you will need a record driver, using the current fields would > allow > searching everything at once. This is something that the > library we are > bringing up would like. It would still allow you to have the > other > features you are looking at. Searches could be limited by > institution or > something. We do that with something like this: > http://plus.mnpals.net/catalog/MSU > > The record driver would know how to handle the fields. Adding > the > ability to limit searches to an institution(think collection > level) and > collection(think head-of-collection) would be great. > > I noticed someone commented about EAD records. This is the > type of > records I plan on adding to VuFind. So again, I'm very > interested. > > al > > > On Wed, 2011-07-27 at 16:30 +0100, Eoghan Ó Carragáin wrote: > > Hi, > > We hope to do some work on a Collections Module over the > next few > > weeks & wanted to run our current plans past the list, so > hopefully > > our design will be reusable in the trunk. > > > > Requirements: > > 1) A splash page for the collection to include some or all > of the > > following > > > > * Collection name and any available collection-level > > metadata/descriptions > > * Facets & facet counts for items in the collection, > e.g. > > display all topics, persons, genre etc from Solr > (something > > like what the Open Library display for their > Work-level > > Records: > > > http://openlibrary.org/works/OL8721462W/Great_Expectations) > > * Links to sub-collections where present (e.g. > > http://digital.library.villanova.edu/Catholica% > 20Collection/) > > * The first page of a result set for items in the > collection > > (much like how the current author module works: e.g. > > > http://vufind.org/demo_trunk/Author/Home?author=Yeats%2C%20W.% > > 20B.%201865-1939.) > > * An archival tree display where available (see > > http://vufind.org/jira/browse/VUFIND-421) > > * Human-readable collection URLS (e.g. > > http://www.vufind.org/Collections/The+VuFind+Archive > > * Any other widget ... > > 2) A means of browsing/searching a list of collections > > > > Approach: > > 1) > > The cleanest design seems to be to create a new Vufind > module > > (ie /web/services/Collections). > > > > In terms of where to get the collection data from, we were > thinking of > > having collection-level records (ie the absolute parent > record for the > > collection and records for any sub-collections) in the > standard Solr > > biblio core. These records/Solr-documents would carry all > the > > collection-level metadata need by the splash page. They > could either > > be hidden from the normal result set or, when presented in a > normal > > result, would link to the collection splash page rather than > a normal > > record page. > > > > We would also need to use two Solr fields, the existing > "collection" > > field and a new "head_of_collection" field. The collection > field will > > be populated for all items in a collection and will contain > a string > > of the title of the parent collection (i.e. either the name > of the > > overall collection or the name of the sub-collection the > item is in). > > The "head_of_collection" field will only be populated by the > > collection-level record and any sub-collection-level > records, & will > > again contain the name of that collection. These values > should allow > > us to display the correct information on the collection > splash page. > > They will also allow easy faceting on items in a given > collection and, > > with some new apache rewrite rules, should allow urls of the > form > > http://vufind.org/Collections/Name+From+Solr+Collection > +Field. In > > MARC, we expect to populate these values using a custom > beanshell > > script using values from 773 tags and/or the LEADER/07 (The > National > > Library of Australia uses 773 and other tags to achieve > something > > similar again with records in the biblio index, see > > http://catalogue.nla.gov.au/Record/4664435). Using Solr > fields as the > > filter should hopefully make this pretty reusable for > non-marc data, > > so long as you can map your collection info into the biblio > Solr > > fields and populate these two filters fields discussed > above. > > > > > > The Collection module uses CollectionRecord.php(like record > uses > > record.php, but simpler in that we don’t envisiage different > tabs at > > the moment) which will look up the item bib_id from the > collection > > title in the url. CollectionRecord.php can then call a new > > RecordDriver function called getCollectionMetadata() from > IndexRecord > > to retrieve information from the collection level record. > > CollectionRecord also has a Solr SearchObject child records > for the > > initial result set (like the Author module) and things like > facet > > counts etc. > > > > 2) A browse interface for collection-level recods should be > reasonably > > straight-forward using the two Solr fields above. > > > > Any thoughts or observations very welcome. David, we'd be > particularly > > interested to hear how this fits with VuDL/Vufind plans. We > don't want > > to reinvent the wheel (or pre-invent it badly!). > > > > Thanks > > Eoghan > > > > ------------------------------------------------------------------------------ > > Got Input? Slashdot Needs You. > > Take our quick survey online. Come on, we don't ask for > help often. > > Plus, you'll get a chance to win $100 to spend on ThinkGeek. > > http://p.sf.net/sfu/slashdot-survey > > _______________________________________________ > > Vufind-tech mailing list > > Vuf...@li... > > https://lists.sourceforge.net/lists/listinfo/vufind-tech > > -- > Alan Rykhus > PALS, A Program of the Minnesota State Colleges and > Universities > (507)389-1975 > ala...@mn... > "It's hard to lead a cavalry charge if you think you look > funny on a > horse" ~ Adlai Stevenson > > > ------------------------------------------------------------------------------ > Got Input? Slashdot Needs You. > Take our quick survey online. Come on, we don't ask for help > often. > Plus, you'll get a chance to win $100 to spend on ThinkGeek. > http://p.sf.net/sfu/slashdot-survey > _______________________________________________ > Vufind-tech mailing list > Vuf...@li... > https://lists.sourceforge.net/lists/listinfo/vufind-tech > -- Alan Rykhus PALS, A Program of the Minnesota State Colleges and Universities (507)389-1975 ala...@mn... "It's hard to lead a cavalry charge if you think you look funny on a horse" ~ Adlai Stevenson |
From: Demian K. <dem...@vi...> - 2011-07-28 13:04:21
|
- I'd definitely encourage you to take a look at how VuFind 2.0 is shaping up; if you eventually plan on upgrading, it may be worth thinking about how the new code could fit into the new model. I'm sure there won't be any major problems in porting things over (after all, the whole point is to make the new version more developer-friendly and extensible than the old one)... but it never hurts to plan ahead a little bit. ---- I failed, for the second time, to get Vufind 2.0 running on windows. I agree it would be good to plan ahead. Can you say a little bit more about how the architecture changes might apply here? Either way, we'll certainly develop this agains 1.x to begin with. What sort of trouble did you run into with VuFind 2.0 on Windows? I might be able to find some time to test it out myself and see if I can figure out what's wrong.... As for how architecture changes might apply, I don't really expect anything too dramatic; the main things are that 2.0 will have stricter MVC compliance (i.e. some things currently in the model layer - for example, record driver methods that return template names - need to be refactored to use view helpers instead) and that controllers work a little differently (VuFind 1.x has a separate PHP script for every action, whereas Zend Framework uses a single class with methods for each action... this means that situations where VuFind 1.x uses inheritance to share common setup functionality across actions in a module can now be done in a slightly easier-to-understand way). But these are fairly trivial changes - hopefully there's nothing more dramatic that I'm failing to think of! - Demian |
From: Eoghan Ó C. <eog...@gm...> - 2011-07-28 13:40:34
|
Hi, It's a while since I tried it but I just got a white screen. I thought it was having difficulty with some paths & tried hardcoding some paths in /public/index.php but gave up. I'm sure it is something wrong with my setup but I didn't have time to dig further. Any pointers much appreciated. Yes, the architecture changes sound as if they'll make implementing a new module easier. We'll probably fire ahead with 1.x, though, but hopefully the port won't be too hard, as you say. Cheers, Eoghan 2011/7/28 Demian Katz <dem...@vi...> > - I’d definitely encourage you to take a look at how VuFind 2.0 is > shaping up; if you eventually plan on upgrading, it may be worth thinking > about how the new code could fit into the new model. I’m sure there won’t > be any major problems in porting things over (after all, the whole point is > to make the new version more developer-friendly and extensible than the old > one)… but it never hurts to plan ahead a little bit.**** > > ---- I failed, for the second time, to get Vufind 2.0 running on windows. > I agree it would be good to plan ahead. Can you say a little bit more about > how the architecture changes might apply here? Either way, we'll certainly > develop this agains 1.x to begin with. > **** > > What sort of trouble did you run into with VuFind 2.0 on Windows? I might > be able to find some time to test it out myself and see if I can figure out > what’s wrong….**** > > ** ** > > As for how architecture changes might apply, I don’t really expect anything > too dramatic; the main things are that 2.0 will have stricter MVC compliance > (i.e. some things currently in the model layer – for example, record driver > methods that return template names – need to be refactored to use view > helpers instead) and that controllers work a little differently (VuFind 1.x > has a separate PHP script for every action, whereas Zend Framework uses a > single class with methods for each action… this means that situations where > VuFind 1.x uses inheritance to share common setup functionality across > actions in a module can now be done in a slightly easier-to-understand > way). But these are fairly trivial changes – hopefully there’s nothing more > dramatic that I’m failing to think of!**** > > ** ** > > - Demian**** > |