From: Brian O'H. <bri...@ed...> - 2013-04-30 09:12:46
|
Simon, Your Geoportal REST harvester works very nicely thanks! I had to add 'sys.schema.key:iso19115-2' as a search criteria in the harvester config to just harvest records with this schema as I was getting errors in schema auto-detection when it came across RDF records in the NOAA catalogue. See https://github.com/Esri/geoportal-server/wiki/How-to-find-all-documents-of-a-particular-metadata-standard. thanks again, Brian On 18/04/13 00:37, Sim...@cs... wrote: > Just in case the gmiTogmd.xsl converter got blocked by some zealous email server, I've committed it to 2.8.x and trunk (see web/src/main/webapp/xsl/conversion/import/gmiTogmd.xsl). > > Cheers, > Simon > ________________________________________ > From: Pigot, Simon (CMAR, Hobart) > Sent: Thursday, 18 April 2013 9:29 AM > To: Brian O'Hare; Francois Prunayre > Cc:Geonetwork-Users@Lists.Sourceforge.Net > Subject: RE: [GeoNetwork-users] Geoportal issues > > Brian, > > GeoPortal REST harvester in GeoNetwork does seem to work ok on the NOAA site. I set up a harvester as shown in the image attached. I did have to create a gmiTogmd.xsl (attached), install it in web/geonetwork/xsl/conversion/import as the NOAA site is using ISO19115-2 (this xslt converts the top element and loses the rest of the gmi content) and select it in the harvester as an XSLT to apply to the content harvested from the geoportal site. > > Cheers, > Simon > ________________________________________ > From: Brian O'Hare [bri...@ed...] > Sent: Wednesday, 17 April 2013 6:51 PM > To: Francois Prunayre > Cc: Pigot, Simon (CMAR, Hobart);Geonetwork-Users@Lists.Sourceforge.Net;mho...@es...;pc...@pc... > Subject: Re: [GeoNetwork-users] Geoportal issues > > Many thanks everyone for your help. > > I've enabled HttpClient debug logging in Geonetwork but can't see the request going to geoportal. However I have used a debugger on the running tomcat and can see that the request is using Filter not CQL. It should just be a straightforward GetRecords request. The capabilities file is parsed without problems. Here's some logging output - you can see the Geoportal url here also: > > 2013-04-17 09:32:58,384 [schema: kb] DEBUG [org.apache.commons.httpclient.HttpConnection] - Open connection towww.ngdc.noaa.gov:80<http://www.ngdc.noaa.gov:80> > 2013-04-17 09:32:58,529 [schema: kb] DEBUG [org.apache.commons.httpclient.HttpMethodBase] - Adding Host request header > 2013-04-17 09:32:58,529 [schema: kb] WARN [org.apache.commons.httpclient.cookie.CookieSpec] - Invalid cookie state: domain not specified > 2013-04-17 09:32:58,529 [schema: kb] DEBUG [org.apache.commons.httpclient.methods.EntityEnclosingMethod] - Request body sent > 2013-04-17 09:32:58,760 [schema: kb] WARN [org.apache.commons.httpclient.HttpMethodBase] - Cookie rejected: "JSESSIONID=70FF886352F32B7A0FB5996515DFB29E". Illegal path attribute "/geoportal". Path of origin:"http://www.ngdc.noaa.gov/geoportal/csw"<http://www.ngdc.noaa.gov/geoportal/csw> > 2013-04-17 09:32:58,763 [schema: kb] DEBUG [org.apache.commons.httpclient.HttpMethodBase] - Should close connection in response to directive: close > 2013-04-17 09:32:58,763 [schema: kb] DEBUG [org.apache.commons.httpclient.HttpConnection] - Releasing connection back to connection manager. > 2013-04-17 09:32:58,763 [schema: kb] WARN [geonetwork.harvester] - Raised exception when searching : org.fao.geonet.csw.common.exceptions.NoApplicableCodeEx: code=NoApplicableCode, locator=null, message= > An exception occurred with no applicable code > > Looks like its to do with cookies. I've rebuilt geonetwork and set cookie policy to 'ignoreCookies' with no success - it still fails. Geoportal version is reported as. > > NGDC Geoportal Instance using ArcGIS Server Geoportal Extension 10 - OGC CSW 2.0.2 ISO AP > > I'll try Simon's restful harvester and see if I have more success. > > thanks again all, > > Brian. > > On 17/04/13 07:24, Francois Prunayre wrote: > Hi, GeoNetwork checks the capability in order to send CQL or FE request. I've been harvesting ESRI GeoPortal with CSW with no issue. > Could you share the URL to be harvested ? > > Cheers. > > Francois > > > 2013/4/17 <Sim...@cs...<mailto:Sim...@cs...>> > I implemented a harvester that uses the geoportal REST api instead of CSW (would prefer to use the standard yes but it could be an alternative if you need it to work and don't have time for sorting this out) so you could try this if you can't nail down the problems with CSW. The harvester is documented in the GeoNetwork 2.8.x manual athttp://geonetwork-opensource.org/manuals/2.8.0/eng/users/managing_metadata/harvesting/geoPREST/index.html#geoportal-rest-harvester - a lot depends on the version of the geoportal you are harvesting from (the open source geoportal should work with CSW, but for other versions such as 9.3.x, it might be better to use the geoportal REST api harvester in 2.8.x). > > Cheers, > Simon > ________________________________________ > From: Marten Hogeweg [mho...@es...<mailto:mho...@es...>] > Sent: Wednesday, 17 April 2013 4:10 AM > To:pc...@pc...<mailto:pc...@pc...>;geo...@li...<mailto:geo...@li...>; Brian O'Hare > Subject: Re: [GeoNetwork-users] Geoportal issues > > Can you share the CSW requests that Geonetwork sends? Note that Geoportal Server expects OGC Filter, not CQL. > > Marten Hogeweg > Esri > > -----Original Message----- > From:pc...@pc...<mailto:pc...@pc...> [mailto:pc...@pc...<mailto:pc...@pc...>] > Sent: Tuesday, April 16, 2013 7:48 PM > To:geo...@li...<mailto:geo...@li...>; Brian O'Hare > Subject: Re: [GeoNetwork-users] Geoportal issues > > Brian, > > I understand the Geonetwork catalogue behindhttp://www.geodata.govt.nz is harvesting records from the ESRI geoportal catalogue athttp://geoportal.doc.govt.nz/geoportal/catalog/main/home.page > > The person looking after the geodata catalogue is rainer (at) silverstripe.com<http://silverstripe.com> who may be able to advise you. I believe Simon Pigot was also involved in making this work successfully. > > Cheers, > > Brent Wood > > > --- On Tue, 4/16/13, Brian O'Hare <bri...@ed...<mailto:bri...@ed...>> wrote: > > From: Brian O'Hare <bri...@ed...<mailto:bri...@ed...>> > Subject: [GeoNetwork-users] Geoportal issues > To:geo...@li...<mailto:geo...@li...> > Date: Tuesday, April 16, 2013, 11:21 PM > > Hi > > Has anyone had any problems trying to run a CSW harvest against ESRI Geoportal. Geonetwork seems to catch a 'NoApplicableCodeException' from Geoportal when trying to do a GetRecordsRequest. I've only noticed this with requests to Geoportal. > > cheers, > > Brian. > > -- > Geospatial Software Developer (Geo Services) EDINA Causewayside House > 160 Causewayside > Edinburgh EH9 1PR > > p: +44 (0)131 5601184<tel:%2B44%20%280%29131%205601184> > e:bri...@ed...<mailto:bri...@ed...> > w:http://edina.ac.uk > > > The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. > > > ------------------------------------------------------------------------------ > Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! > http://www2.precog.com/precogplatform/slashdotnewsletter > _______________________________________________ > GeoNetwork-users mailing list > Geo...@li...<mailto:Geo...@li...> > https://lists.sourceforge.net/lists/listinfo/geonetwork-users > GeoNetwork OpenSource is maintained athttp://sourceforge.net/projects/geonetwork > ------------------------------------------------------------------------------ > Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! > http://www2.precog.com/precogplatform/slashdotnewsletter > _______________________________________________ > GeoNetwork-users mailing list > Geo...@li...<mailto:Geo...@li...> > https://lists.sourceforge.net/lists/listinfo/geonetwork-users > GeoNetwork OpenSource is maintained athttp://sourceforge.net/projects/geonetwork > > > > > ------------------------------------------------------------------------------ > Precog is a next-generation analytics platform capable of advanced > analytics on semi-structured data. The platform includes APIs for building > apps and a phenomenal toolset for data science. Developers can use > our toolset for easy data analysis & visualization. Get a free account! > http://www2.precog.com/precogplatform/slashdotnewsletter > _______________________________________________ > GeoNetwork-users mailing list > Geo...@li...<mailto:Geo...@li...> > https://lists.sourceforge.net/lists/listinfo/geonetwork-users > GeoNetwork OpenSource is maintained athttp://sourceforge.net/projects/geonetwork > > > > -- > Geospatial Software Developer (Geo Services) > EDINA > Causewayside House > 160 Causewayside > Edinburgh EH9 1PR > > p: +44 (0)131 5601184 > e:bri...@ed...<mailto:bri...@ed...> > w:http://edina.ac.uk > > > -- Geospatial Software Developer (Geo Services) EDINA Causewayside House 160 Causewayside Edinburgh EH9 1PR p: +44 (0)131 5601184 e:bri...@ed... w:http://edina.ac.uk The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. |