I've installed the latest wikipedia database successfully. I've also deployed the wikipedia-miner.war in tomcat (x64) on Windows 7 x64 using JRE 1.7 x64. On trying the search service with a keyword, I'm getting the following exception in the server log:
SEVERE: Servlet.service() for servlet in context with path threw exception with root cause
com.sleepycat.je.DatabaseNotFoundException: (JE 4.0.103) Database markup not found.
at com.sleepycat.je.Environment.setupDatabase(Environment.java:790)
at com.sleepycat.je.Environment.openDatabase(Environment.java:536)
at org.wikipedia.miner.db.WDatabase.getDatabase(WDatabase.java:573)
at org.wikipedia.miner.db.WDatabase.retrieve(WDatabase.java:311)
at org.wikipedia.miner.model.Page.getMarkup(Page.java:231)
at org.wikipedia.miner.model.Page.getFirstParagraphMarkup(Page.java:292)
at org.wikipedia.miner.service.ExploreArticleService.buildWrappedResponse(ExploreArticleService.java:201)
at org.wikipedia.miner.service.Service.doGet(Service.java:221)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:304)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.wikipedia.miner.service.CharsetFilter.doFilter(CharsetFilter.java:37)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:185)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:151)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:929)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:405)
at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:269)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:515)
at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:1773)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Appreciate some light shed on the same…
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Just to add more details to the same thread, when I run the search service, I hit getDatabase(boolean readOnly) method of WDatabase.java.
1. The first call is for the database 'page' - which succeeds.
2. The second call is also for the database 'page' - which again succeeds.
3. The third call is for the database 'markup' - which fails since there is no database by that name
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
It sounds like the database did not get built completely. Can you delete the current database directory and try the ant build-database target again? Keep a note of any problems that occur during this call, and post the stack-trace if you still have problems.
Cheers,
Dave
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
1. Deleted the entire DB folder and ran the build-database again
2. Found NO exceptions in the log of the build-database task
3. Created the WAR and deployed in Tomcat.
4. Tomcat started successfully and loaded the database (no exceptions there either)
5. When I access the application, I get the following exception in Tomcat:
Sep 23, 2011 4:29:27 PM org.apache.catalina.core.StandardContext loadOnStartup
SEVERE: Servlet /wikipediaminer threw load() exception
java.lang.NullPointerException
at java.io.FileInputStream.<init>(FileInputStream.java:134)
at weka.wrapper.Decider.load(Decider.java:168)
at org.wikipedia.miner.annotation.Disambiguator.loadClassifier(Disambiguator.java:289)
at org.wikipedia.miner.service.WikifyService.init(WikifyService.java:94)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1228)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1147)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1043)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4957)
at org.apache.catalina.core.StandardContext$3.call(StandardContext.java:5284)
at org.apache.catalina.core.StandardContext$3.call(StandardContext.java:5279)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Sep 23, 2011 4:29:27 PM org.apache.catalina.core.ApplicationContext log
INFO: Marking servlet correct as unavailable
Sep 23, 2011 4:29:27 PM org.apache.catalina.core.StandardContext loadOnStartup
SEVERE: Servlet /wikipediaminer threw load() exception
java.lang.ClassNotFoundException: org.wikipedia.miner.service.CorrectionService
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1678)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1523)
at org.apache.catalina.core.DefaultInstanceManager.loadClass(DefaultInstanceManager.java:415)
at org.apache.catalina.core.DefaultInstanceManager.loadClassMaybePrivileged(DefaultInstanceManager.java:397)
at org.apache.catalina.core.DefaultInstanceManager.newInstance(DefaultInstanceManager.java:118)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1099)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1043)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4957)
at org.apache.catalina.core.StandardContext$3.call(StandardContext.java:5284)
at org.apache.catalina.core.StandardContext$3.call(StandardContext.java:5279)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Sep 23, 2011 4:29:28 PM org.apache.catalina.core.ApplicationContext log
INFO: ContextListener: contextInitialized()
Sep 23, 2011 4:29:28 PM org.apache.catalina.core.ApplicationContext log
INFO: SessionListener: contextInitialized()
Sep 23, 2011 4:29:29 PM org.apache.catalina.core.ApplicationContext log
INFO: ContextListener: attributeAdded('org.apache.jasper.compiler.TldLocationsCache', 'org.apache.jasper.compiler.TldLocationsCache@4007ce9c')
Sep 23, 2011 4:46:36 PM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet in context with path threw exception with root cause
com.sleepycat.je.DatabaseNotFoundException: (JE 4.0.103) Database markup not found.
at com.sleepycat.je.Environment.setupDatabase(Environment.java:790)
at com.sleepycat.je.Environment.openDatabase(Environment.java:536)
at org.wikipedia.miner.db.WDatabase.getDatabase(WDatabase.java:573)
at org.wikipedia.miner.db.WDatabase.retrieve(WDatabase.java:311)
at org.wikipedia.miner.model.Page.getMarkup(Page.java:231)
at org.wikipedia.miner.model.Page.getFirstParagraphMarkup(Page.java:292)
at org.wikipedia.miner.service.ExploreArticleService.buildWrappedResponse(ExploreArticleService.java:201)
at org.wikipedia.miner.service.Service.doGet(Service.java:221)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:304)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.wikipedia.miner.service.CharsetFilter.doFilter(CharsetFilter.java:37)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:185)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:151)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:929)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:405)
at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:269)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:515)
at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:1773)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
One point of interest. When the build-database task was running, I saw several sub-tasks and I didn't find 'markup' database in it.
a. I'm using Java 1.7 x64 on Windows 7 x64
b. Using "enwiki-20110722-csv.tar.gz"
c. Using "wikipedia-miner-1.2.0.tar.gz"
d. Using the latest BerkeleyDB (Java version)
e. The following databases loaded in the build-database task - as you can see, I don't see the markup database
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
You didn't mention any xml dump. You need to grab this huge file, uncompress it in the same directory as the csv files, and run the build-database again.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Downloaded and completed the process. The "Search" webservice now works. However, the annotate service is throwing some exceptions (pasted below). The 'Compare' service is giving me NaN% for supposedly related topics (photon and tachyon, photon and meson, Obama and Politics, etc.) - but I don't see an exception in the log for the same though:
Sep 23, 2011 10:25:17 PM org.apache.catalina.core.StandardContext loadOnStartup
SEVERE: Servlet /wikipediaminer threw load() exception
java.lang.NullPointerException
at java.io.FileInputStream.<init>(FileInputStream.java:134)
at weka.wrapper.Decider.load(Decider.java:168)
at org.wikipedia.miner.annotation.Disambiguator.loadClassifier(Disambiguator.java:289)
at org.wikipedia.miner.service.WikifyService.init(WikifyService.java:94)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1228)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1147)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1043)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4957)
at org.apache.catalina.core.StandardContext$3.call(StandardContext.java:5284)
at org.apache.catalina.core.StandardContext$3.call(StandardContext.java:5279)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Sep 23, 2011 10:25:17 PM org.apache.catalina.core.ApplicationContext log
INFO: Marking servlet correct as unavailable
Sep 23, 2011 10:25:17 PM org.apache.catalina.core.StandardContext loadOnStartup
SEVERE: Servlet /wikipediaminer threw load() exception
java.lang.ClassNotFoundException: org.wikipedia.miner.service.CorrectionService
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1678)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1523)
at org.apache.catalina.core.DefaultInstanceManager.loadClass(DefaultInstanceManager.java:415)
at org.apache.catalina.core.DefaultInstanceManager.loadClassMaybePrivileged(DefaultInstanceManager.java:397)
at org.apache.catalina.core.DefaultInstanceManager.newInstance(DefaultInstanceManager.java:118)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1099)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1043)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4957)
at org.apache.catalina.core.StandardContext$3.call(StandardContext.java:5284)
at org.apache.catalina.core.StandardContext$3.call(StandardContext.java:5279)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Sep 23, 2011 10:25:20 PM org.apache.catalina.core.ApplicationContext log
INFO: ContextListener: contextInitialized()
Sep 23, 2011 10:25:20 PM org.apache.catalina.core.ApplicationContext log
INFO: SessionListener: contextInitialized()
Sep 23, 2011 10:25:20 PM org.apache.catalina.core.ApplicationContext log
INFO: ContextListener: attributeAdded('org.apache.jasper.compiler.TldLocationsCache', 'org.apache.jasper.compiler.TldLocationsCache@2c93829c')
Sep 23, 2011 10:34:02 PM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Allocate exception for servlet wikify
java.lang.NullPointerException
at java.io.FileInputStream.<init>(FileInputStream.java:134)
at weka.wrapper.Decider.load(Decider.java:168)
at org.wikipedia.miner.annotation.Disambiguator.loadClassifier(Disambiguator.java:289)
at org.wikipedia.miner.service.WikifyService.init(WikifyService.java:94)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1228)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1147)
at org.apache.catalina.core.StandardWrapper.allocate(StandardWrapper.java:836)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:135)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:185)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:151)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:929)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:405)
at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:269)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:515)
at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:1773)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Sep 23, 2011 10:38:01 PM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Allocate exception for servlet wikify
java.lang.NullPointerException
at java.io.FileInputStream.<init>(FileInputStream.java:134)
at weka.wrapper.Decider.load(Decider.java:168)
at org.wikipedia.miner.annotation.Disambiguator.loadClassifier(Disambiguator.java:289)
at org.wikipedia.miner.service.WikifyService.init(WikifyService.java:94)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1228)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1147)
at org.apache.catalina.core.StandardWrapper.allocate(StandardWrapper.java:836)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:135)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:185)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:151)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:929)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:405)
at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:269)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:515)
at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:1773)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Never mind - solved it myself. I just hadn't indicated the appropriate model files in the template.xml. Once I did that, things started working fine. Now, the only exception I see (which doesn't seem to affect the three services in any way) is indicated below. I'll try fixing it myself.
You know, it would be so much better if you could give a more detailed steps to setup and indicated it somewhere in the wiki. If you are okay, I can come up with the steps that I followed and if it is okay, you can publish the same in the wiki to help folks who might want to use this.
SEVERE: Servlet /wikipediaminer threw load() exception
java.lang.ClassNotFoundException: org.wikipedia.miner.service.CorrectionService
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1678)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1523)
at org.apache.catalina.core.DefaultInstanceManager.loadClass(DefaultInstanceManager.java:415)
at org.apache.catalina.core.DefaultInstanceManager.loadClassMaybePrivileged(DefaultInstanceManager.java:397)
at org.apache.catalina.core.DefaultInstanceManager.newInstance(DefaultInstanceManager.java:118)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1099)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1043)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4957)
at org.apache.catalina.core.StandardContext$3.call(StandardContext.java:5284)
at org.apache.catalina.core.StandardContext$3.call(StandardContext.java:5279)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Fixed that too - the web.xml had a wrong entry. The correct class is "org.wikipedia.miner.service.CorrectService". On correcting it, things started working.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The wiki is free for anyone to edit, but I felt it was pretty clear about the need for both xml dump and csv summaries. It is missing a detailed description of the wikipedia miner configuration, and I'm working on that.
Thanks for finding the bug with the CorrectService. I've fixed this in the project trunk, and it will be in the next release.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi, I tried to install Wikipedia Miner on Tomcat Server 6.0, but it gives me some errors.
One, apparently already resolved, was caused by CorrectionService instead of CorrectService (like is written above).
Another error is the following:
org.apache.catalina.core.StandardContext loadOnStartup
GRAVE: Servlet /wikipedia-miner threw load() exception
java.lang.NullPointerException
at java.io.FileInputStream.<init>(FileInputStream.java:116)
at weka.wrapper.Decider.load(Decider.java:168)
at org.wikipedia.miner.annotation.Disambiguator.loadClassifier(Unknown Source)
at org.wikipedia.miner.service.WikifyService.init(Unknown Source)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1173)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:993)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4420)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4733)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:840)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
at org.apache.catalina.core.StandardService.start(StandardService.java:525)
at org.apache.catalina.core.StandardServer.start(StandardServer.java:754)
at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
Another error occurs when I click on "wikify" button:
2-ago-2012 16.23.37 org.apache.catalina.core.StandardWrapperValve invoke
GRAVE: Allocate exception for servlet wikify
java.lang.NullPointerException
at java.io.FileInputStream.<init>(FileInputStream.java:116)
at weka.wrapper.Decider.load(Decider.java:168)
at org.wikipedia.miner.annotation.Disambiguator.loadClassifier(Unknown Source)
at org.wikipedia.miner.service.WikifyService.init(Unknown Source)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1173)
at org.apache.catalina.core.StandardWrapper.allocate(StandardWrapper.java:809)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:129)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:662)
I think that the cause is linked with models.
Can you explain us if these models have to be configured and how? And if it must be configured somewhat which is not explained in the official guide in
Then it starts loading the database ( seems work fine…) at the end of the loading the demos still don't work.
Does anybody knows which could be the problem?
Thanks Daniele
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I've installed the latest wikipedia database successfully. I've also deployed the wikipedia-miner.war in tomcat (x64) on Windows 7 x64 using JRE 1.7 x64. On trying the search service with a keyword, I'm getting the following exception in the server log:
SEVERE: Servlet.service() for servlet in context with path threw exception with root cause
com.sleepycat.je.DatabaseNotFoundException: (JE 4.0.103) Database markup not found.
at com.sleepycat.je.Environment.setupDatabase(Environment.java:790)
at com.sleepycat.je.Environment.openDatabase(Environment.java:536)
at org.wikipedia.miner.db.WDatabase.getDatabase(WDatabase.java:573)
at org.wikipedia.miner.db.WDatabase.retrieve(WDatabase.java:311)
at org.wikipedia.miner.model.Page.getMarkup(Page.java:231)
at org.wikipedia.miner.model.Page.getFirstParagraphMarkup(Page.java:292)
at org.wikipedia.miner.service.ExploreArticleService.buildWrappedResponse(ExploreArticleService.java:201)
at org.wikipedia.miner.service.Service.doGet(Service.java:221)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:304)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.wikipedia.miner.service.CharsetFilter.doFilter(CharsetFilter.java:37)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:185)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:151)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:929)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:405)
at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:269)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:515)
at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:1773)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Appreciate some light shed on the same…
Just to add more details to the same thread, when I run the search service, I hit getDatabase(boolean readOnly) method of WDatabase.java.
1. The first call is for the database 'page' - which succeeds.
2. The second call is also for the database 'page' - which again succeeds.
3. The third call is for the database 'markup' - which fails since there is no database by that name
It sounds like the database did not get built completely. Can you delete the current database directory and try the ant build-database target again? Keep a note of any problems that occur during this call, and post the stack-trace if you still have problems.
Cheers,
Dave
Sure - will try again and let you know.
Tried that and it still does not work.
1. Deleted the entire DB folder and ran the build-database again
2. Found NO exceptions in the log of the build-database task
3. Created the WAR and deployed in Tomcat.
4. Tomcat started successfully and loaded the database (no exceptions there either)
5. When I access the application, I get the following exception in Tomcat:
Sep 23, 2011 4:29:27 PM org.apache.catalina.core.ApplicationContext log
INFO: Marking servlet correct as unavailable
Sep 23, 2011 4:29:27 PM org.apache.catalina.core.StandardContext loadOnStartup
SEVERE: Servlet /wikipediaminer threw load() exception
java.lang.ClassNotFoundException: org.wikipedia.miner.service.CorrectionService
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1678)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1523)
at org.apache.catalina.core.DefaultInstanceManager.loadClass(DefaultInstanceManager.java:415)
at org.apache.catalina.core.DefaultInstanceManager.loadClassMaybePrivileged(DefaultInstanceManager.java:397)
at org.apache.catalina.core.DefaultInstanceManager.newInstance(DefaultInstanceManager.java:118)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1099)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1043)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4957)
at org.apache.catalina.core.StandardContext$3.call(StandardContext.java:5284)
at org.apache.catalina.core.StandardContext$3.call(StandardContext.java:5279)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Sep 23, 2011 4:29:28 PM org.apache.catalina.core.ApplicationContext log
INFO: ContextListener: contextInitialized()
Sep 23, 2011 4:29:28 PM org.apache.catalina.core.ApplicationContext log
INFO: SessionListener: contextInitialized()
Sep 23, 2011 4:29:29 PM org.apache.catalina.core.ApplicationContext log
INFO: ContextListener: attributeAdded('org.apache.jasper.compiler.TldLocationsCache', 'org.apache.jasper.compiler.TldLocationsCache@4007ce9c')
Sep 23, 2011 4:46:36 PM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet in context with path threw exception with root cause
com.sleepycat.je.DatabaseNotFoundException: (JE 4.0.103) Database markup not found.
at com.sleepycat.je.Environment.setupDatabase(Environment.java:790)
at com.sleepycat.je.Environment.openDatabase(Environment.java:536)
at org.wikipedia.miner.db.WDatabase.getDatabase(WDatabase.java:573)
at org.wikipedia.miner.db.WDatabase.retrieve(WDatabase.java:311)
at org.wikipedia.miner.model.Page.getMarkup(Page.java:231)
at org.wikipedia.miner.model.Page.getFirstParagraphMarkup(Page.java:292)
at org.wikipedia.miner.service.ExploreArticleService.buildWrappedResponse(ExploreArticleService.java:201)
at org.wikipedia.miner.service.Service.doGet(Service.java:221)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:304)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.wikipedia.miner.service.CharsetFilter.doFilter(CharsetFilter.java:37)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:185)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:151)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:929)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:405)
at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:269)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:515)
at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:1773)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
One point of interest. When the build-database task was running, I saw several sub-tasks and I didn't find 'markup' database in it.
a. I'm using Java 1.7 x64 on Windows 7 x64
b. Using "enwiki-20110722-csv.tar.gz"
c. Using "wikipedia-miner-1.2.0.tar.gz"
d. Using the latest BerkeleyDB (Java version)
e. The following databases loaded in the build-database task - as you can see, I don't see the markup database
Ah.
You didn't mention any xml dump. You need to grab this huge file, uncompress it in the same directory as the csv files, and run the build-database again.
Downloading it - will let you know how it goes.
Downloaded and completed the process. The "Search" webservice now works. However, the annotate service is throwing some exceptions (pasted below). The 'Compare' service is giving me NaN% for supposedly related topics (photon and tachyon, photon and meson, Obama and Politics, etc.) - but I don't see an exception in the log for the same though:
Sep 23, 2011 10:25:17 PM org.apache.catalina.core.ApplicationContext log
INFO: Marking servlet correct as unavailable
Sep 23, 2011 10:25:17 PM org.apache.catalina.core.StandardContext loadOnStartup
SEVERE: Servlet /wikipediaminer threw load() exception
java.lang.ClassNotFoundException: org.wikipedia.miner.service.CorrectionService
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1678)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1523)
at org.apache.catalina.core.DefaultInstanceManager.loadClass(DefaultInstanceManager.java:415)
at org.apache.catalina.core.DefaultInstanceManager.loadClassMaybePrivileged(DefaultInstanceManager.java:397)
at org.apache.catalina.core.DefaultInstanceManager.newInstance(DefaultInstanceManager.java:118)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1099)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1043)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4957)
at org.apache.catalina.core.StandardContext$3.call(StandardContext.java:5284)
at org.apache.catalina.core.StandardContext$3.call(StandardContext.java:5279)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Sep 23, 2011 10:25:20 PM org.apache.catalina.core.ApplicationContext log
INFO: ContextListener: contextInitialized()
Sep 23, 2011 10:25:20 PM org.apache.catalina.core.ApplicationContext log
INFO: SessionListener: contextInitialized()
Sep 23, 2011 10:25:20 PM org.apache.catalina.core.ApplicationContext log
INFO: ContextListener: attributeAdded('org.apache.jasper.compiler.TldLocationsCache', 'org.apache.jasper.compiler.TldLocationsCache@2c93829c')
Sep 23, 2011 10:34:02 PM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Allocate exception for servlet wikify
java.lang.NullPointerException
at java.io.FileInputStream.<init>(FileInputStream.java:134)
at weka.wrapper.Decider.load(Decider.java:168)
at org.wikipedia.miner.annotation.Disambiguator.loadClassifier(Disambiguator.java:289)
at org.wikipedia.miner.service.WikifyService.init(WikifyService.java:94)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1228)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1147)
at org.apache.catalina.core.StandardWrapper.allocate(StandardWrapper.java:836)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:135)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:185)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:151)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:929)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:405)
at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:269)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:515)
at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:1773)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Sep 23, 2011 10:38:01 PM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Allocate exception for servlet wikify
java.lang.NullPointerException
at java.io.FileInputStream.<init>(FileInputStream.java:134)
at weka.wrapper.Decider.load(Decider.java:168)
at org.wikipedia.miner.annotation.Disambiguator.loadClassifier(Disambiguator.java:289)
at org.wikipedia.miner.service.WikifyService.init(WikifyService.java:94)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1228)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1147)
at org.apache.catalina.core.StandardWrapper.allocate(StandardWrapper.java:836)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:135)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:185)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:151)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:929)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:405)
at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:269)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:515)
at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:1773)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Never mind - solved it myself. I just hadn't indicated the appropriate model files in the template.xml. Once I did that, things started working fine. Now, the only exception I see (which doesn't seem to affect the three services in any way) is indicated below. I'll try fixing it myself.
You know, it would be so much better if you could give a more detailed steps to setup and indicated it somewhere in the wiki. If you are okay, I can come up with the steps that I followed and if it is okay, you can publish the same in the wiki to help folks who might want to use this.
Fixed that too - the web.xml had a wrong entry. The correct class is "org.wikipedia.miner.service.CorrectService". On correcting it, things started working.
HI Jayaram,
The wiki is free for anyone to edit, but I felt it was pretty clear about the need for both xml dump and csv summaries. It is missing a detailed description of the wikipedia miner configuration, and I'm working on that.
Thanks for finding the bug with the CorrectService. I've fixed this in the project trunk, and it will be in the next release.
Hi, I tried to install Wikipedia Miner on Tomcat Server 6.0, but it gives me some errors.
One, apparently already resolved, was caused by CorrectionService instead of CorrectService (like is written above).
Another error is the following:
org.apache.catalina.core.StandardContext loadOnStartup
GRAVE: Servlet /wikipedia-miner threw load() exception
java.lang.NullPointerException
at java.io.FileInputStream.<init>(FileInputStream.java:116)
at weka.wrapper.Decider.load(Decider.java:168)
at org.wikipedia.miner.annotation.Disambiguator.loadClassifier(Unknown Source)
at org.wikipedia.miner.service.WikifyService.init(Unknown Source)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1173)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:993)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4420)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4733)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:840)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
at org.apache.catalina.core.StandardService.start(StandardService.java:525)
at org.apache.catalina.core.StandardServer.start(StandardServer.java:754)
at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
Another error occurs when I click on "wikify" button:
2-ago-2012 16.23.37 org.apache.catalina.core.StandardWrapperValve invoke
GRAVE: Allocate exception for servlet wikify
java.lang.NullPointerException
at java.io.FileInputStream.<init>(FileInputStream.java:116)
at weka.wrapper.Decider.load(Decider.java:168)
at org.wikipedia.miner.annotation.Disambiguator.loadClassifier(Unknown Source)
at org.wikipedia.miner.service.WikifyService.init(Unknown Source)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1173)
at org.apache.catalina.core.StandardWrapper.allocate(StandardWrapper.java:809)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:129)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:662)
I think that the cause is linked with models.
Can you explain us if these models have to be configured and how? And if it must be configured somewhat which is not explained in the official guide in
?
Thanks you a lot,
I think this has something to do with files not being found. Can you post your config files?
Edgar
Thank you so much, I probably did something wrong here:
https://dl.dropbox.com/u/13667204/Default/configs.rar
I would be very grateful if you could help me to figure out what's missing in configs…
I solved the errors, i forgot to insert the models paths…
My new configs are: https://dl.dropbox.com/u/13667204/Default/configs2.rar
Everything seems to be ok, but demos in the deployed web service still don't work!
I got this in the Eclipse Console:
Then it starts loading the database ( seems work fine…) at the end of the loading the demos still don't work.
Does anybody knows which could be the problem?
Thanks Daniele
Solved,
here I explain how: http://www.youtube.com/watch?v=HKsiXlrtEIk for those who would need it.,
Daniele