From: Demian K. <dem...@vi...> - 2009-10-29 16:36:38
|
After a bit of a struggle, I've decided to give up on getting "edit record" working. The problem here is that in order to update a Solr entry, you need to rewrite all of the fields -- there's no true update, just a replace. Since VuFind can only easily write to Solr using the XML interface, and the MARC data in the "fullrecord" field of the index contains control characters that cannot be transported by XML, it appears to be impossible to save changes to most records. I've opened VUFIND-155 to track this -- if Solr opens up new HTTP-based protocols for adding records, this problem could become much easier to fix in the future. And if there's already some capability that I'm not aware of, please point me in the right direction! Technical problems aside, I'm not sure that there's much value to this form of index editing anyway -- it might be useful for testing purposes, but in a production environment, direct manipulation of Solr entries seems more dangerous than helpful. For the moment, I have replaced "Edit" with a "View" function -- at least this gives us a way to examine Solr records without having to go through the Solr interface; this might be useful in some scenarios. I believe this puts an end to VUFIND-10, so I have closed it. I've opened a number of more granular tickets for various shortcomings of the admin interface, but I don't think any of these are important enough to worry about in the near term -- they're all currently assigned to the wish list. - Demian From: Demian Katz Sent: Wednesday, October 28, 2009 3:55 PM To: vuf...@li... Subject: Progress on VUFIND-10 (Finalize Admin module for 1.0 iteration) I've spent some time today cleaning up issues in the Admin module: 1.) Fixed XSS vulnerabilities. 2.) Commented out completely non-functional features that aren't ready to be implemented (i.e. authority maintenance) 3.) Replaced incomplete file edit modules (config.ini / elevated terms) with a less ambitious generic text editor -- this gives easy access to all the key config files without all the time it would take to implement advanced web forms to allow editing of all the individual settings. I'm not sure if advanced web-based editing of these files even makes sense at this point, but I've opened JIRA tickets as reminders that we might want to think about it in the future. 4.) Cosmetic fixes -- the default template version of the admin module had a number of problems that I've done my best to clean up. 5.) Better feedback in some places (report success or failure of file saves, record edits/deletes, etc.). I believe at this point that the only non-functional admin feature that remains is the "edit record" screen. I'll see if I can fix that tomorrow morning. Once that's done, I think we can close VUFIND-10. Please let me know if you are aware of anything else that I'm missing. thanks, Demian |
From: Chris D. <ce...@ui...> - 2009-10-29 16:50:00
|
Not only can you not update a record in SOLR directly, but I also think the ³poor man¹s update² method (retrieve current record, manipulate it, then add i.e., replace the modified record) would fail too, no? I¹m thinking about the non-stored fields; they would be lost if you attempted to do this, wouldn¹t they? I¹m not 100% sure and would love to be proved wrong here :-) Chris From: Demian Katz <dem...@vi...> Date: Thu, 29 Oct 2009 12:36:20 -0400 To: "vuf...@li..." <vuf...@li...> Subject: Re: [VuFind-Tech] Progress on VUFIND-10 (Finalize Admin module for 1.0 iteration) After a bit of a struggle, I've decided to give up on getting "edit record" working. The problem here is that in order to update a Solr entry, you need to rewrite all of the fields -- there's no true update, just a replace. Since VuFind can only easily write to Solr using the XML interface, and the MARC data in the "fullrecord" field of the index contains control characters that cannot be transported by XML, it appears to be impossible to save changes to most records. I've opened VUFIND-155 to track this -- if Solr opens up new HTTP-based protocols for adding records, this problem could become much easier to fix in the future. And if there's already some capability that I¹m not aware of, please point me in the right direction! Technical problems aside, I'm not sure that there's much value to this form of index editing anyway -- it might be useful for testing purposes, but in a production environment, direct manipulation of Solr entries seems more dangerous than helpful. For the moment, I have replaced "Edit" with a "View" function -- at least this gives us a way to examine Solr records without having to go through the Solr interface; this might be useful in some scenarios. I believe this puts an end to VUFIND-10, so I have closed it. I've opened a number of more granular tickets for various shortcomings of the admin interface, but I don't think any of these are important enough to worry about in the near term -- they're all currently assigned to the wish list. - Demian From: Demian Katz Sent: Wednesday, October 28, 2009 3:55 PM To: vuf...@li... Subject: Progress on VUFIND-10 (Finalize Admin module for 1.0 iteration) I've spent some time today cleaning up issues in the Admin module: 1.) Fixed XSS vulnerabilities. 2.) Commented out completely non-functional features that aren't ready to be implemented (i.e. authority maintenance) 3.) Replaced incomplete file edit modules (config.ini / elevated terms) with a less ambitious generic text editor -- this gives easy access to all the key config files without all the time it would take to implement advanced web forms to allow editing of all the individual settings. I'm not sure if advanced web-based editing of these files even makes sense at this point, but I've opened JIRA tickets as reminders that we might want to think about it in the future. 4.) Cosmetic fixes -- the default template version of the admin module had a number of problems that I've done my best to clean up. 5.) Better feedback in some places (report success or failure of file saves, record edits/deletes, etc.). I believe at this point that the only non-functional admin feature that remains is the "edit record" screen. I'll see if I can fix that tomorrow morning. Once that's done, I think we can close VUFIND-10. Please let me know if you are aware of anything else that I'm missing. thanks, Demian ---------------------------------------------------------------------------- -- Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference _______________________________________________ Vufind-tech mailing list Vuf...@li... https://lists.sourceforge.net/lists/listinfo/vufind-tech |
From: Demian K. <dem...@vi...> - 2009-10-29 16:52:14
|
You're absolutely right -- another significant reason why this really isn't a good idea to begin with. I'll add a comment to the JIRA ticket pointing this out. - Demian From: Chris Delis [mailto:ce...@ui...] Sent: Thursday, October 29, 2009 12:50 PM To: Demian Katz; vuf...@li... Subject: Re: [VuFind-Tech] Progress on VUFIND-10 (Finalize Admin module for 1.0 iteration) Not only can you not update a record in SOLR directly, but I also think the "poor man's update" method (retrieve current record, manipulate it, then add - i.e., replace - the modified record) would fail too, no? I'm thinking about the non-stored fields; they would be lost if you attempted to do this, wouldn't they? I'm not 100% sure and would love to be proved wrong here :-) Chris ________________________________ From: Demian Katz <dem...@vi...> Date: Thu, 29 Oct 2009 12:36:20 -0400 To: "vuf...@li..." <vuf...@li...> Subject: Re: [VuFind-Tech] Progress on VUFIND-10 (Finalize Admin module for 1.0 iteration) After a bit of a struggle, I've decided to give up on getting "edit record" working. The problem here is that in order to update a Solr entry, you need to rewrite all of the fields -- there's no true update, just a replace. Since VuFind can only easily write to Solr using the XML interface, and the MARC data in the "fullrecord" field of the index contains control characters that cannot be transported by XML, it appears to be impossible to save changes to most records. I've opened VUFIND-155 to track this -- if Solr opens up new HTTP-based protocols for adding records, this problem could become much easier to fix in the future. And if there's already some capability that I'm not aware of, please point me in the right direction! Technical problems aside, I'm not sure that there's much value to this form of index editing anyway -- it might be useful for testing purposes, but in a production environment, direct manipulation of Solr entries seems more dangerous than helpful. For the moment, I have replaced "Edit" with a "View" function -- at least this gives us a way to examine Solr records without having to go through the Solr interface; this might be useful in some scenarios. I believe this puts an end to VUFIND-10, so I have closed it. I've opened a number of more granular tickets for various shortcomings of the admin interface, but I don't think any of these are important enough to worry about in the near term -- they're all currently assigned to the wish list. - Demian From: Demian Katz Sent: Wednesday, October 28, 2009 3:55 PM To: vuf...@li... Subject: Progress on VUFIND-10 (Finalize Admin module for 1.0 iteration) I've spent some time today cleaning up issues in the Admin module: 1.) Fixed XSS vulnerabilities. 2.) Commented out completely non-functional features that aren't ready to be implemented (i.e. authority maintenance) 3.) Replaced incomplete file edit modules (config.ini / elevated terms) with a less ambitious generic text editor -- this gives easy access to all the key config files without all the time it would take to implement advanced web forms to allow editing of all the individual settings. I'm not sure if advanced web-based editing of these files even makes sense at this point, but I've opened JIRA tickets as reminders that we might want to think about it in the future. 4.) Cosmetic fixes -- the default template version of the admin module had a number of problems that I've done my best to clean up. 5.) Better feedback in some places (report success or failure of file saves, record edits/deletes, etc.). I believe at this point that the only non-functional admin feature that remains is the "edit record" screen. I'll see if I can fix that tomorrow morning. Once that's done, I think we can close VUFIND-10. Please let me know if you are aware of anything else that I'm missing. thanks, Demian ________________________________ ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference ________________________________ _______________________________________________ Vufind-tech mailing list Vuf...@li... https://lists.sourceforge.net/lists/listinfo/vufind-tech |