Re: [photospace-developer] problems building index on
Status: Alpha
Brought to you by:
jahlonzo
Menu
▾
▴
Re: [photospace-developer] problems building index on
|
From: alon s. <al...@sa...> - 2005-11-11 20:59:34
|
I just checked in this fix. SearchIndex.update() now looks like:
public void update(File updated) throws IOException
{
synchronized(this)
{
closeReader();
if (index.exists()) FileUtils.cleanDirectory(index);
FileUtils.copyDirectory(updated, index);
}
}
Alon
On 11/10/05, alon salant <al...@sa...> wrote:
> Cool. I'll make this fix in CVS.
>
> Alon
>
> On 11/10/05, Tobias SCHOESSLER <Tob...@un...> wrote:
> > ok, I got it fixed.
> >
> > It comes down to this jdk bug
> >
> > http://h21007.www2.hp.com/cmdspp/QuestionAnswer/1,1764,95
> > F27FD6-4941-11D6-B748-0003470C0153,00.html
> >
> > renameTo seems to have problems renaming files across filesystems.
> >
> > In my case we have tomcat and its temp dir on /usr/local and our web
> > deployment on something like /web on different partitions.
> >
> > I patched SearchIndex.update(File updated) like this.
> >
> > public void update(File updated) throws IOException {
> > synchronized (this) {
> > closeReader();
> > FileUtils.cleanDirectory(index);
> > FileUtils.copyDirectory(updated, index);
> > }
> > }
> >
> > I had to upgrade the commons.io to 1.1 to support the directory copy.
> > Commons copyDirectory is probably less efficient than the filesystem's
> > rename implementation but this probably works on a broader range of
> > plattforms.
> >
> > I tested windows 2000 and hp machine with some debian linux.
> >
> >
> > thanks alon salant
> > thanks to Philip S. Constantinou pointing out the filesystem issues wit=
h
> > renameTo.
> >
> >
> >
> > ______________________________________
> > Tobias Schoessler, Java Developer
> > Information Management Unit
> > Information Technology Service
> > United Nations Office on Drugs and Crime
> >
> > Tel: (+43-1) 26060-5173
> > Websites: www.unov.org, www.unodc.org
> > ______________________________________
> > Impossible is not a fact, only an opinion
> >
> >
> >
> > alon salant
> > <al...@sa...>
> > Sent by: =
To
> > photospace-develo pho...@li...=
efo
> > per-admin@lists.s rge.net
> > ourceforge.net =
cc
> >
> > Subj=
ect
> > Wednesday, 9 Re: [photospace-developer] probl=
ems
> > November 2005 building index on
> > 21:20
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Ah, didn't realize I wasn't replying to the list. The project has
> > actually not been very active lately - mostly b/c I have been working
> > on other things.
> >
> > Did you try using File.renameTo() in a different context like a JSP?
> > Looking at the source for File.renameTo() there's not much going on.
> > There is a check against the SecurityManager which would throw an
> > exception if it failed (I think) and then a native call to the file
> > system:
> >
> > public boolean renameTo(File dest) {
> > SecurityManager security =3D System.getSecurityManager();
> > if (security !=3D null) {
> > security.checkWrite(path);
> > security.checkWrite(dest.path);
> > }
> > return fs.rename(this, dest);
> > }
> >
> > It looks like fs.rename() is returning false. Why? Hmmm...
> >
> >
> > On 11/9/05, Tobias SCHOESSLER <Tob...@un...> wrote:
> > > Ok, I am digging into this further.
> > >
> > > I added debug logs to SearchIndex.java
> > >
> > > public void update(File updated) throws IOException {
> > > synchronized (this) {
> > > closeReader();
> > > log.info("****************** Can I write on "+index=
+" ?
> > > "+ index.canWrite() );
> > > FileUtils.deleteDirectory(index);
> > > // FileUtils.cleanDirectory(index);
> > > log.info("****************** Cleaning "+index);
> > > log.info("****************** Does "+index+" still
> > exist?
> > > "+ index.exists() );
> > > log.info("****************** Can I write on "+updat=
ed+"
> > ?
> > > "+ updated.canWrite() );
> > > log.info("****************** Can I write on "+index=
+" ?
> > > "+ index.canWrite() );
> > >
> > > if (!updated.renameTo(index))
> > > throw new IOException("Unable to move tempora=
ry
> > > index "
> > > + updated + " to " + index);
> > > }
> > > }
> > >
> > >
> > > server log at context STARTUP - so context doesn't even start in this
> > case.
> > >
> > > INFO: Find registry server-registry.xml at classpath resource
> > > Nov 9, 2005 7:20:56 PM org.apache.catalina.startup.Catalina start
> > > INFO: Server startup in 34519 ms
> > > 19:20:59,799 DEBUG [TP-Processor3]
> > > org.apache.catalina.core.StandardWrapper:748 - Allocating non-STM
> > instance
> > > Nov 9, 2005 7:20:59 PM org.apache.catalina.core.ApplicationContext lo=
g
> > > INFO: HTMLManager: init: Associated with Deployer
> > > 'Catalina:type=3DDeployer,host=3Dtomcat5.unodc.org'
> > > Nov 9, 2005 7:20:59 PM org.apache.catalina.core.ApplicationContext lo=
g
> > > INFO: HTMLManager: init: Global resources are available
> > > 19:20:59,824 DEBUG [TP-Processor3]
> > > org.apache.catalina.core.StandardWrapper:763 - Returning non-STM
> > instance
> > > Nov 9, 2005 7:20:59 PM org.apache.catalina.core.ApplicationContext lo=
g
> > > INFO: HTMLManager: list: Listing contexts for virtual host
> > > 'tomcat5.unodc.org'
> > > 19:21:08,987 DEBUG [TP-Processor3]
> > > org.apache.catalina.core.StandardWrapper:763 - Returning non-STM
> > instance
> > > 19:21:12,703 DEBUG [TP-Processor3]
> > > org.apache.catalina.core.StandardWrapper:763 - Returning non-STM
> > instance
> > > INFO [2005-11-09 19:21:12,723] [Thread-47] (IndexingJob.java:71) -
> > > Indexing ... 1 files
> > > 19:21:12,773 DEBUG [TP-Processor3]
> > > org.apache.catalina.core.StandardWrapper:763 - Returning non-STM
> > instance
> > > INFO [2005-11-09 19:21:12,841] [Thread-47] (SearchIndex.java:55) -
> > > ****************** Can I write on /web/photospace/data/index ? true
> > > INFO [2005-11-09 19:21:12,849] [Thread-47] (SearchIndex.java:58) -
> > > ****************** Deleting /web/photospace/data/index
> > > INFO [2005-11-09 19:21:12,849] [Thread-47] (SearchIndex.java:59) -
> > > ****************** Does /web/photospace/data/index still exist? false
> > > INFO [2005-11-09 19:21:12,849] [Thread-47] (SearchIndex.java:60) -
> > > ****************** Can I write on
> > /usr/local/tomcat2/temp/photospace-index
> > > ? true
> > > INFO [2005-11-09 19:21:12,849] [Thread-47] (SearchIndex.java:61) -
> > > ****************** Can I write on /web/photospace/data/index ? false
> > > ERROR [2005-11-09 19:21:12,856] [Thread-47] (IndexingJob.java:109) -
> > Error
> > > indexing
> > > java.io.IOException: Unable to move temporary index
> > > /usr/local/tomcat2/temp/photospace-index to /web/photospace/data/inde=
x
> > > at photospace.search.SearchIndex.update(SearchIndex.java:64)
> > > at photospace.search.IndexerImpl.index(IndexerImpl.java:78)
> > > at photospace.search.IndexingJob.run(IndexingJob.java:99)
> > > at java.lang.Thread.run(Thread.java:595)
> > > 19:21:42,910 DEBUG [TP-Processor2]
> > > org.apache.catalina.core.StandardWrapper:763 - Returning non-STM
> > instance
> > > ERROR [2005-11-09 19:21:42,915] [TP-Processor2]
> > > (StandardWrapperValve.java:253) - Servlet.service() for servlet jsp t=
hrew
> > > exception
> > > java.lang.IllegalStateException: Search index has not been created ye=
t
> > > at photospace.search.SearchIndex.getReader(SearchIndex.java:2=
6)
> > > at photospace.search.Searcher.search(Searcher.java:233)
> > > at photospace.search.Searcher.search(Searcher.java:178)
> > > at photospace.search.Searcher.search(Searcher.java:163)
> > > at photospace.web.tags.SearchTag.doStartTag(SearchTag.java:22=
)
> > > at
> > >
> > org.apache.jsp.doc.index_002dstatus_jsp._jspx_meth_space_search_0(org.a=
pache.jsp.doc.index_002dstatus_jsp:140)
> >
> > > at
> > >
> > org.apache.jsp.doc.index_002dstatus_jsp.access$1(org.apache.jsp.doc.ind=
ex_002dstatus_jsp:128)
> >
> > > at
> > >
> > org.apache.jsp.doc.index_002dstatus_jsp$index_002dstatus_jspHelper.invo=
ke1(org.apache.jsp.doc.index_002dstatus_jsp:511)
> >
> > > at
> > >
> > org.apache.jsp.doc.index_002dstatus_jsp$index_002dstatus_jspHelper.invo=
ke(org.apache.jsp.doc.index_002dstatus_jsp:570)
> >
> > > at
> > >
> > org.apache.jsp.tag.web.main_002dlayout_tag.doTag(org.apache.jsp.tag.web=
.main_002dlayout_tag:281)
> >
> > > at
> > >
> > org.apache.jsp.doc.index_002dstatus_jsp._jspx_meth_tags_main$1layout_0(=
org.apache.jsp.doc.index_002dstatus_jsp:108)
> >
> > > at
> > >
> > org.apache.jsp.doc.index_002dstatus_jsp._jspService(org.apache.jsp.doc.=
index_002dstatus_jsp:82)
> >
> > > at
> > > org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
> > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:80=
2)
> > > at
> > >
> > org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.j=
ava:322)
> >
> > > at
> > > org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:2=
91)
> > > at
> > > org.apache.jasper.servlet.JspServlet.service(JspServlet.java:241)
> > >
> > >
> > > The debugs look fine and reasonable to me. Maybe the problem is that=
the
> > > Directory does not exists that it is trying to rename to.
> > >
> > > that why I tried the FileUtils.cleanDirectory(index); instead of
> > > FileUtils.deleteDirectory(index);
> > >
> > > in this case the context starts up, but still when clicking on build
> > index
> > > I get this:
> > >
> > > 19:33:16,816 DEBUG [TP-Processor1]
> > > org.apache.catalina.core.StandardWrapper:763 - Returning non-STM
> > instance
> > > 19:33:17,082 DEBUG [TP-Processor1]
> > > org.apache.catalina.core.StandardWrapper:763 - Returning non-STM
> > instance
> > > 19:33:17,093 DEBUG [TP-Processor1]
> > > org.apache.catalina.core.StandardWrapper:763 - Returning non-STM
> > instance
> > > 19:33:17,102 DEBUG [TP-Processor1]
> > > org.apache.catalina.core.StandardWrapper:763 - Returning non-STM
> > instance
> > > 19:33:17,111 DEBUG [TP-Processor1]
> > > org.apache.catalina.core.StandardWrapper:763 - Returning non-STM
> > instance
> > > 19:33:19,781 DEBUG [TP-Processor1]
> > > org.apache.catalina.core.StandardWrapper:763 - Returning non-STM
> > instance
> > > INFO [2005-11-09 19:33:19,796] [Thread-66] (IndexingJob.java:71) -
> > > Indexing ... 1 files
> > > 19:33:19,853 DEBUG [TP-Processor1]
> > > org.apache.catalina.core.StandardWrapper:763 - Returning non-STM
> > instance
> > > INFO [2005-11-09 19:33:19,878] [Thread-66] (SearchIndex.java:55) -
> > > ****************** Can I write on /web/photospace/data/index ? true
> > > INFO [2005-11-09 19:33:19,884] [Thread-66] (SearchIndex.java:58) -
> > > ****************** Cleaning /web/photospace/data/index
> > > INFO [2005-11-09 19:33:19,885] [Thread-66] (SearchIndex.java:59) -
> > > ****************** Does /web/photospace/data/index still exist? true
> > > INFO [2005-11-09 19:33:19,885] [Thread-66] (SearchIndex.java:60) -
> > > ****************** Can I write on
> > /usr/local/tomcat2/temp/photospace-index
> > > ? true
> > > INFO [2005-11-09 19:33:19,885] [Thread-66] (SearchIndex.java:61) -
> > > ****************** Can I write on /web/photospace/data/index ? true
> > > ERROR [2005-11-09 19:33:19,888] [Thread-66] (IndexingJob.java:109) -
> > Error
> > > indexing
> > > java.io.IOException: Unable to move temporary index
> > > /usr/local/tomcat2/temp/photospace-index to /web/photospace/data/inde=
x
> > > at photospace.search.SearchIndex.update(SearchIndex.java:64)
> > > at photospace.search.IndexerImpl.index(IndexerImpl.java:78)
> > > at photospace.search.IndexingJob.run(IndexingJob.java:99)
> > > at java.lang.Thread.run(Thread.java:595)
> > >
> > >
> > > Again the debugs look fine to me, no clue why the renameTo fails thou=
gh
> > ...
> > > ?
> > >
> > >
> > > thanks for taking the time.
> > > Why don't you reply to the mailing list ? Looks so dead on sourceforg=
e -
> > no
> > > posts, and yet its so active ?
> > >
> > >
> > >
> > > ______________________________________
> > > Tobias Schoessler, Java Developer
> > > Information Management Unit
> > > Information Technology Service
> > > United Nations Office on Drugs and Crime
> > >
> > > Tel: (+43-1) 26060-5173
> > > Websites: www.unov.org, www.unodc.org
> > > ______________________________________
> > > Impossible is not a fact, only an opinion
> > >
> > >
> > >
> > > alon salant
> > > <al...@sa...>
> > > Sent by:
> > To
> > > as...@gm... Tobias SCHOESSLER
> > > <Tob...@un...=
g>
> > >
> > cc
> > > Wednesday, 9
> > > November 2005
> > Subject
> > > 18:43 Re: [photospace-developer]
> > problems
> > > building index on
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > Ahh, sorry missed the end of your message. You have checked permissio=
ns.
> > >
> > > The problem can only be file system related. Here's the source for
> > > SearchIndex.update():
> > >
> > > public void update(File updated) throws IOException
> > > {
> > > synchronized(this)
> > > {
> > > closeReader();
> > > FileUtils.deleteDirectory(index);
> > > if (!updated.renameTo(index))
> > > throw new IOException("Unable to move temporary index " +
> > > updated + " to " + index);
> > > }
> > > }
> > >
> > > FileUtils is part of Jakarta Commons Lang and does a recursive delete=
.
> > > File.renameTo() is part of the Java core. File.renameTo() is failing
> > > either because the old index did not get deleted (in your case I
> > > believe it has not even been created yet) or because it can't write t=
o
> > > /web/photospace/data/index.
> > >
> > > If you're up for it, I'd recommend making sure that tomcat can write
> > > to that location. Just write a JSP that does File.renameTo() to that
> > > location and see if it works.
> > >
> > > Alon
> > >
> > > On 11/9/05, alon salant <al...@sa...> wrote:
> > > > The first thing I would look at are file system permissions. Can yo=
u
> > > > confirm that the user under which tomcat is running has permission =
to
> > > > write to /web/photospace/data/index?
> > > >
> > > > Alon
> > > >
> > > > On 11/9/05, Tobias SCHOESSLER <Tob...@un...> wrot=
e:
> > > > >
> > > > > I am trying to run photospace on a linux server.
> > > > >
> > > > > when building the index I get this exception:
> > > > >
> > > > > 18:04:12,494 DEBUG [TP-Processor3]
> > > > > org.apache.catalina.core.StandardWrapper:763 - Returning non-ST=
M
> > > instance
> > > > > 18:04:16,091 DEBUG [TP-Processor2]
> > > > > org.apache.catalina.core.StandardWrapper:763 - Returning non-ST=
M
> > > instance
> > > > > 18:04:21,718 DEBUG [TP-Processor3]
> > > > > org.apache.catalina.core.StandardWrapper:763 - Returning non-ST=
M
> > > instance
> > > > > INFO [2005-11-09 18:04:21,736] [Thread-47] (IndexingJob.java:71)=
-
> > > > > Indexing 1 files
> > > > > 18:04:21,748 DEBUG [TP-Processor2]
> > > > > org.apache.catalina.core.StandardWrapper:763 - Returning non-ST=
M
> > > instance
> > > > > ERROR [2005-11-09 18:04:21,857] [Thread-47] (IndexingJob.java:109=
) -
> > > Error
> > > > > indexing
> > > > > java.io.IOException: Unable to move temporary index
> > > > > /usr/local/tomcat2/temp/photospace-index to
> > /web/photospace/data/index
> > > > > at photospace.search.SearchIndex.update(SearchIndex.java:=
59)
> > > > > at photospace.search.IndexerImpl.index(IndexerImpl.java:7=
8)
> > > > > at photospace.search.IndexingJob.run(IndexingJob.java:99)
> > > > > at java.lang.Thread.run(Thread.java:595)
> > > >
> > >
> > >
> > >
> >
> >
> > -------------------------------------------------------
> > SF.Net email is sponsored by:
> > Tame your development challenges with Apache's Geronimo App Server.
> > Download
> > it for free - -and be entered to win a 42" plasma tv or your very own
> > Sony(tm)PSP. Click here to play: http://sourceforge.net/geronimo.php
> > _______________________________________________
> > photospace-developer mailing list
> > pho...@li...
> > https://lists.sourceforge.net/lists/listinfo/photospace-developer
> >
> >
> >
> >
> > -------------------------------------------------------
> > SF.Net email is sponsored by:
> > Tame your development challenges with Apache's Geronimo App Server. Dow=
nload
> > it for free - -and be entered to win a 42" plasma tv or your very own
> > Sony(tm)PSP. Click here to play: http://sourceforge.net/geronimo.php
> > _______________________________________________
> > photospace-developer mailing list
> > pho...@li...
> > https://lists.sourceforge.net/lists/listinfo/photospace-developer
> >
>
|
