Dear Jennifer,

I seem to have been mistaken in my understanding that the all_ac contains only metadata values that come from the "solr.search.filters", every metadata field seems to end up in there.

You could put the all_ac to "ignore" but then the "full text" option in discovery won't have an autocomplete.

What I would suggest is altering the SolrServiceImpl.java class.
In this class I would suggest moving the following lines:

//Add the field to all for autocomplete so our autocomplete works for all fields
doc.addField("all_ac", value);

Into the following if block:

if(SearchUtils.getSearchFilters().contains(field) || SearchUtils.getSearchFilters().contains(unqualifiedField + "." + Item.ANY)){
    //Add a dynamic fields for autocomplete in search
    doc.addField(field + "_ac", value); 
}

So you would get something like:

if(SearchUtils.getSearchFilters().contains(field) || SearchUtils.getSearchFilters().contains(unqualifiedField + "." + Item.ANY)){
    //Add a dynamic fields for autocomplete in search`
    doc.addField("all_ac", value);
    doc.addField(field + "_ac", value); 
}

After that clean package and use ant update and re index discovery with the -f option. That way only metadata fields that are in the configuration end up in the autocomplete.


Kind regards,

Kevin Van de Velde
@mire
Esperantolaan 4 - 3001 Heverlee - Belgium
2888 Loker Avenue East, Suite 305 - Carlsbad, CA 92010 - USA
atmire.com - Institutional Repository Solutions



On 25 May 2011 01:14, Jennifer Whalan <Jennifer.Whalan@nt.gov.au> wrote:
Kevin,
 
 
It doesn't appear that this is what happens (at least on my instance here). Our property in dspace-solr-seach.cfg is:
 
solr.search.filters=dc.title, dc.contributor.author, subject, dc.date.issued.year, coverage
but the actual data in the solr database is: (apologies for the font size).
 
<?xml version="1.0" encoding="UTF-8"?>
<response>

<lst name="responseHeader">
 <int name="status">0</int>
 <int name="QTime">421</int>
 <lst name="params">
  <str name="indent">on</str>
  <str name="start">0</str>
  <str name="q">10070/58278</str>
  <str name="rows">10</str>
  <str name="version">2.2</str>
 </lst>
</lst>
<result name="response" numFound="1" start="0">
 <doc>
  <date name="SolrIndexer.lastIndexed">2011-05-24T22:29:53.253Z</date>
  <arr name="all_ac"><str/><str>2007-11-27T00:00:40Z</str><str>2007-11-27T00:00:40Z</str>
	<str>Unknown</str><str>PH0077/0028</str><str>http://hdl.handle.net/10070/58278</str><str/>
	<str/><str/><str/><str>Northern Territory. Dept. of the Community Development Collection.</str>
	<str>077-0028</str><str>Northern Territory Library.</str><str>Unknown.</str><str/>
	<str>Withdrawn due to migration issues, Jennifer Whalan, 20/04/2009.</str></arr>
  <arr name="coverage_filter"><str>|||</str></arr>
  <arr name="dc.coverage.spatial"><str/></arr>
  <arr name="dc.coverage.spatial.en"><str/></arr>
  <arr name="dc.coverage.spatial_filter"><str>|||</str></arr>
  <arr name="dc.date.accessioned.year"><int>2007</int></arr>
  <arr name="dc.date.accessioned_dt"><date>2007-11-27T00:00:40Z</date></arr>
  <arr name="dc.date.available.year"><int>2007</int></arr>
  <arr name="dc.date.available_dt"><date>2007-11-27T00:00:40Z</date></arr>
  <arr name="dc.description"><str/></arr>
  <arr name="dc.description.en"><str/></arr>
  <arr name="dc.format.extent"><str/></arr>
  <arr name="dc.format.extent.en"><str/></arr>
  <arr name="dc.format.medium"><str/></arr>
  <arr name="dc.format.medium.en"><str/></arr>
  <arr name="dc.format.mimetype"><str/></arr>
  <arr name="dc.format.mimetype.en"><str/></arr>
  <arr name="dc.identifier.other"><str>ph0077/0028</str></arr>
  <arr name="dc.identifier.other.en"><str>PH0077/0028</str></arr>
  <arr name="dc.identifier.uri"><str>http://hdl.handle.net/10070/58278</str></arr>
  <arr name="dc.relation.ispartof"><str>northern territory. dept. of the community development collection.</str></arr>
  <arr name="dc.relation.ispartof.en"><str>Northern Territory. Dept. of the Community Development Collection.</str></arr>
  <arr name="dc.rights.holder"><str>northern territory library.</str></arr>
  <arr name="dc.rights.holder.en"><str>Northern Territory Library.</str></arr>
  <arr name="dc.title"><str>077-0028</str></arr>
  <arr name="dc.title.en"><str>077-0028</str></arr>
  <arr name="dc.title_ac"><str>077-0028</str></arr>
  <str name="dc.title_sort">077-0028</str>
  <str name="handle">10070/58278</str>
  <arr name="location"><str>m2</str><str>l8</str></arr>
  <arr name="location.coll"><str>8</str></arr>
  <arr name="location.comm"><str>2</str></arr>
  <arr name="publication_grp"><str>Northern Territory. Dept. of the Community Development Collection.</str></arr>
  <int name="search.resourceid">58379</int>
  <int name="search.resourcetype">2</int>
 </doc>
</result>
</response>
I'm assuming that the all_ac field is the one that is causing the issues, since it has fields in there that are not part of the above config setting. I suppose I could just ignore this field using
 
   <field name="all_ac" type="ignored" multiValued="true"/>
like how I ignored other fields?
 
I have a title_ac, and I know that I get author_ac (this record just doesn't have any), but for some reason my subject and coverage fields don't get _ac fields, so I assume that if I did ignore the all_ac, I wouldn't not get a autocomplete for them?
 
 
Thanks
Jennifer

Jennifer Whalan
Systems Support Officer |
Innovation & Access
Northern Territory Library 'Celebrating 30 Years'

Department of Natural Resources, Environment, The Arts and Sport
Northern Territory Government


Phone:  (08) 8922 0752
Fax:    (08) 8922 0722
Email: 
jennifer.whalan@nt.gov.au
Web:    www.ntl.nt.gov.au

The information contained in this message and any attachments may be confidential information and may be subject to legal privilege, public interest or legal profession privilege. If you are not the intended recipient, any use, disclosure or copying of this message or any attachments is unauthorised. If you have received this document in error, please advise the sender. No representation or warranty is given that attached files are free from viruses or other defects. The recipient assumes all responsibility for any loss or damage resulting directly or indirectly from the use of any attached files.

 


From: kevin.van.de.velde@gmail.com [mailto:kevin.van.de.velde@gmail.com] On Behalf Of Kevin Van de velde
Sent: Thursday, 5 May 2011 4:48 PM
To: Jennifer Whalan
Cc: dspace-tech@lists.sourceforge.net

Subject: Re: [Dspace-tech] Discovery and metadata.hide option in 1.7.1

Dear Jennifer,

The dropdown suggestions only show words that appear in one of the configured metadata fields in the "solr.search.filters" property in the {dspace.dir}/config/dspace-solr-search.cfg file so I would suggest looking there to ensure that none of the non indexed fields appear in that property.

Kind regards,

Kevin Van de Velde
@mire
Esperantolaan 4 - 3001 Heverlee - Belgium
2888 Loker Avenue East, Suite 305 - Carlsbad, CA 92010 - USA
atmire.com - Institutional Repository Solutions



On 3 May 2011 00:48, Jennifer Whalan <Jennifer.Whalan@nt.gov.au> wrote:
Bram,
 
Unfortunately, we had no luck with the following. After adding the fields to the schema.xml and deleting the index and reindexing, we found that they still were appearing in the dropdown suggestions. I've included what we added below, and the options around it as well, to show the general location we added it. I had quick read through the schema.xml documentation on the apache site as well, and so I can understand why it is supposed to work. My only guess would be that if the order that things are added to the schema.xml is important, that it is in the wrong place.
 
 
Thanks
Jennifer
 
     <!--Date matching-->
    <dynamicField name="*.year" type="sint" indexed="true" stored="true" multiValued="true" omitNorms="true" />
    <dynamicField name="*_dt" type="date" indexed="true" stored="true" multiValued="true" omitNorms="true" />

   <field name="dc.description.provenance" type="ignored" multiValued="true"/>
   <field name="dc.rights" type="ignored" multiValued="true"/>
   <field name="dc.description.provenancedonor" type="ignored" multiValued="true"/>
   <field name="dc.date.review" type="ignored" multiValued="true"/>
   <field name="dc.identifier.email" type="ignored" multiValued="true"/>
   <field name="dc.identifier.address" type="ignored" multiValued="true"/>
   <field name="dc.identifier.phone" type="ignored" multiValued="true"/>
   <field name="dc.ntl.rating" type="ignored" multiValued="true"/>
   <field name="dc.ntl.withdrawn" type="ignored" multiValued="true"/>
   <field name="dc.ntl.terms" type="ignored" multiValued="true"/>
   <field name="dc.ntl.lift" type="ignored" multiValued="true"/>

    <!--Used for matching on all other fields-->
     <dynamicField name="*" type="text" multiValued="true"/>
 

Jennifer Whalan
Systems Support Officer |
Innovation & Access
Northern Territory Library 'Celebrating 30 Years'


Department of Natural Resources, Environment, The Arts and Sport
Northern Territory Government


Phone:  (08) 8922 0752
Fax:    (08) 8922 0722
Email: 
jennifer.whalan@nt.gov.au
Web:    www.ntl.nt.gov.au

The information contained in this message and any attachments may be confidential information and may be subject to legal privilege, public interest or legal profession privilege. If you are not the intended recipient, any use, disclosure or copying of this message or any attachments is unauthorised. If you have received this document in error, please advise the sender. No representation or warranty is given that attached files are free from viruses or other defects. The recipient assumes all responsibility for any loss or damage resulting directly or indirectly from the use of any attached files.

 


From: bluyten@gmail.com [mailto:bluyten@gmail.com] On Behalf Of Bram Luyten
Sent: Thursday, 21 April 2011 9:28 PM
To: Jennifer Whalan
Cc: Kevin Van de velde
Subject: Re: [Dspace-tech] Discovery and metadata.hide option in 1.7.1

Dear Jennifer,

recently, one of our developers responded to your question on the DSpace mailing list. As @mire originally developed Mirage & Discovery for DSpace 1.7, I would be very interested to learn if the given advice has helped you to resolve your issue.

Furthermore, if you experience any other issue with Discovery and Mirage, we can always take a look at it for resolving in one of the upcoming releases of DSpace. Likewise, if you would have any challenging migration, upgrade or customization tasks, I'd be happy to investigate whether our professional services could apply in your context.

with kindest regards,

Bram Luyten

@mire

Technologielaan 9 - 3001 Heverlee - Belgium
2888 Loker Avenue East, Suite 305 - Carlsbad, CA 92010 - USA

atmire.com - Institutional Repository Solutions


On Thu, Apr 14, 2011 at 3:43 PM, Kevin Van de velde <  > wrote:
Hi Jennifer,

The disallowing of indexing certain fields in discovery has not yet been implemented. But if you want you can do this yourself by altering a single file.
The file that needs to be edited is the following: {dspace.dir}/solr/search/conf/schema.xml. In the "fields" tag (it should start somewhere around line 490 of the file) you just need to add the following line:

<field name="{field.not.to.index}" type="ignored" multiValued="true"/>

So for example if you do not want the title field to be indexed add:

<field name="dc.title" type="ignored" multiValued="true"/>

After this is done just restart the tomcat and re index discovery and the field should be removed from the index.

I really hope this helps.

Ps: I do plan to create a patch to fix this issue (so that non indexable fields can be configured) as soon as I find the time to create one.

Kind regards,

Kevin Van de Velde
@mire - www.atmire.com
533 2nd Street - Encinitas, CA 92024 - USA
Technologielaan 9 - 3001 Heverlee - Belgium


http://www.atmire.com - Institutional Repository Solutions
http://www.togather.eu - Before getting together, get Tog@ther


On 14 April 2011 08:45, Jennifer Whalan <Jennifer.Whalan@nt.gov.au> wrote:
Not sure if this is perhaps a bug or not, but currently in our dspace.cfg, we have about 8 metadata fields that we want to keep hidden from searches, item views, oai, etc. However I've noticed that these fields are indexed by Discovery, and so if doing a filter on Fulltext, the dropdown shows options that are in these particular fields.
 
Is there a way to remove these fields from being indexed by Discovery?
 
 
Thanks
Jennifer.
 

Jennifer Whalan
Territory Stories Administrator
Innovation & Access, Northern Territory Library
Department of Natural Resources, Environment, The Arts and Sport
Northern Territory Government


Phone:  (08) 8922 0752
Fax:    (08) 8922 0722
Email: 
jennifer.whalan@nt.gov.au
Web:    www.ntl.nt.gov.au

The information contained in this message and any attachments may be confidential information and may be subject to legal privilege, public interest or legal profession privilege. If you are not the intended recipient, any use, disclosure or copying of this message or any attachments is unauthorised. If you have received this document in error, please advise the sender. No representation or warranty is given that attached files are free from viruses or other defects. The recipient assumes all responsibility for any loss or damage resulting directly or indirectly from the use of any attached files.


------------------------------------------------------------------------------
Benefiting from Server Virtualization: Beyond Initial Workload
Consolidation -- Increasing the use of server virtualization is a top
priority.Virtualization can reduce costs, simplify management, and improve
application availability and disaster protection. Learn more about boosting
the value of server virtualization. http://p.sf.net/sfu/vmware-sfdev2dev
_______________________________________________
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech



------------------------------------------------------------------------------
Benefiting from Server Virtualization: Beyond Initial Workload
Consolidation -- Increasing the use of server virtualization is a top
priority.Virtualization can reduce costs, simplify management, and improve
application availability and disaster protection. Learn more about boosting
the value of server virtualization. http://p.sf.net/sfu/vmware-sfdev2dev
_______________________________________________
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech



------------------------------------------------------------------------------
WhatsUp Gold - Download Free Network Management Software
The most intuitive, comprehensive, and cost-effective network
management toolset available today.  Delivers lowest initial
acquisition cost and overall TCO of any competing solution.
http://p.sf.net/sfu/whatsupgold-sd
_______________________________________________
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech