Re: [photospace-developer] problems building index on

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Cool. I'll make this fix in CVS.

Alon

On 11/10/05, Tobias SCHOESSLER <Tob...@un...> wrote:
> ok, I got it fixed.
>
> It comes down to this jdk bug
>
> http://h21007.www2.hp.com/cmdspp/QuestionAnswer/1,1764,95
> F27FD6-4941-11D6-B748-0003470C0153,00.html
>
> renameTo seems to have problems renaming files across filesystems.
>
> In my case we have tomcat and its temp dir on /usr/local and our web
> deployment on something like /web on different partitions.
>
> I patched SearchIndex.update(File updated) like this.
>
>       public void update(File updated) throws IOException {
>             synchronized (this) {
>                   closeReader();
>                   FileUtils.cleanDirectory(index);
>                   FileUtils.copyDirectory(updated, index);
>             }
>       }
>
> I had to upgrade the commons.io to 1.1 to support the directory copy.
> Commons copyDirectory is probably less efficient than the filesystem's
> rename implementation but this probably works on a broader range of
> plattforms.
>
> I tested windows 2000 and hp machine with some debian linux.
>
>
> thanks alon salant
> thanks to Philip S. Constantinou pointing out the filesystem issues with
> renameTo.
>
>
>
> ______________________________________
> Tobias Schoessler, Java Developer
> Information Management Unit
> Information Technology Service
> United Nations Office on Drugs and Crime
>
> Tel:  (+43-1) 26060-5173
> Websites: www.unov.org, www.unodc.org
> ______________________________________
> Impossible is not a fact, only an opinion
>
>
>
>              alon salant
>              <al...@sa...>
>              Sent by:                                                   T=
o
>              photospace-develo         pho...@li...=
o
>              per-admin@lists.s         rge.net
>              ourceforge.net                                             c=
c
>
>                                                                    Subjec=
t
>              Wednesday, 9              Re: [photospace-developer] problem=
s
>              November 2005             building index on
>              21:20
>
>
>
>
>
>
>
>
>
> Ah, didn't realize I wasn't replying to the list. The project has
> actually not been very active lately - mostly b/c I have been working
> on other things.
>
> Did you try using File.renameTo() in a different context like a JSP?
> Looking at the source for File.renameTo() there's not much going on.
> There is a check against the SecurityManager which would throw an
> exception if it failed (I think) and then a native call to the file
> system:
>
>     public boolean renameTo(File dest) {
>              SecurityManager security =3D System.getSecurityManager();
>              if (security !=3D null) {
>                  security.checkWrite(path);
>                  security.checkWrite(dest.path);
>              }
>              return fs.rename(this, dest);
>     }
>
> It looks like fs.rename() is returning false. Why? Hmmm...
>
>
> On 11/9/05, Tobias SCHOESSLER <Tob...@un...> wrote:
> > Ok, I am digging into this further.
> >
> > I added debug logs to SearchIndex.java
> >
> >       public void update(File updated) throws IOException {
> >             synchronized (this) {
> >                   closeReader();
> >                   log.info("****************** Can I write on "+index+"=
 ?
> > "+ index.canWrite() );
> >                   FileUtils.deleteDirectory(index);
> > //                FileUtils.cleanDirectory(index);
> >                   log.info("****************** Cleaning "+index);
> >                   log.info("****************** Does "+index+" still
> exist?
> > "+ index.exists() );
> >                   log.info("****************** Can I write on "+updated=
+"
> ?
> > "+ updated.canWrite() );
> >                   log.info("****************** Can I write on "+index+"=
 ?
> > "+ index.canWrite() );
> >
> >                   if (!updated.renameTo(index))
> >                         throw new IOException("Unable to move temporary
> > index "
> >                                     + updated + " to " + index);
> >             }
> >       }
> >
> >
> > server log at context STARTUP - so context doesn't even start in this
> case.
> >
> > INFO: Find registry server-registry.xml at classpath resource
> > Nov 9, 2005 7:20:56 PM org.apache.catalina.startup.Catalina start
> > INFO: Server startup in 34519 ms
> > 19:20:59,799 DEBUG [TP-Processor3]
> > org.apache.catalina.core.StandardWrapper:748 - Allocating non-STM
> instance
> > Nov 9, 2005 7:20:59 PM org.apache.catalina.core.ApplicationContext log
> > INFO: HTMLManager: init: Associated with Deployer
> > 'Catalina:type=3DDeployer,host=3Dtomcat5.unodc.org'
> > Nov 9, 2005 7:20:59 PM org.apache.catalina.core.ApplicationContext log
> > INFO: HTMLManager: init: Global resources are available
> > 19:20:59,824 DEBUG [TP-Processor3]
> > org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
> instance
> > Nov 9, 2005 7:20:59 PM org.apache.catalina.core.ApplicationContext log
> > INFO: HTMLManager: list: Listing contexts for virtual host
> > 'tomcat5.unodc.org'
> > 19:21:08,987 DEBUG [TP-Processor3]
> > org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
> instance
> > 19:21:12,703 DEBUG [TP-Processor3]
> > org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
> instance
> >  INFO [2005-11-09 19:21:12,723] [Thread-47] (IndexingJob.java:71) -
> > Indexing ... 1 files
> > 19:21:12,773 DEBUG [TP-Processor3]
> > org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
> instance
> >  INFO [2005-11-09 19:21:12,841] [Thread-47] (SearchIndex.java:55) -
> > ****************** Can I write on /web/photospace/data/index ? true
> >  INFO [2005-11-09 19:21:12,849] [Thread-47] (SearchIndex.java:58) -
> > ****************** Deleting /web/photospace/data/index
> >  INFO [2005-11-09 19:21:12,849] [Thread-47] (SearchIndex.java:59) -
> > ****************** Does /web/photospace/data/index still exist? false
> >  INFO [2005-11-09 19:21:12,849] [Thread-47] (SearchIndex.java:60) -
> > ****************** Can I write on
> /usr/local/tomcat2/temp/photospace-index
> > ? true
> >  INFO [2005-11-09 19:21:12,849] [Thread-47] (SearchIndex.java:61) -
> > ****************** Can I write on /web/photospace/data/index ? false
> > ERROR [2005-11-09 19:21:12,856] [Thread-47] (IndexingJob.java:109) -
> Error
> > indexing
> > java.io.IOException: Unable to move temporary index
> > /usr/local/tomcat2/temp/photospace-index to /web/photospace/data/index
> >         at photospace.search.SearchIndex.update(SearchIndex.java:64)
> >         at photospace.search.IndexerImpl.index(IndexerImpl.java:78)
> >         at photospace.search.IndexingJob.run(IndexingJob.java:99)
> >         at java.lang.Thread.run(Thread.java:595)
> > 19:21:42,910 DEBUG [TP-Processor2]
> > org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
> instance
> > ERROR [2005-11-09 19:21:42,915] [TP-Processor2]
> > (StandardWrapperValve.java:253) - Servlet.service() for servlet jsp thr=
ew
> > exception
> > java.lang.IllegalStateException: Search index has not been created yet
> >         at photospace.search.SearchIndex.getReader(SearchIndex.java:26)
> >         at photospace.search.Searcher.search(Searcher.java:233)
> >         at photospace.search.Searcher.search(Searcher.java:178)
> >         at photospace.search.Searcher.search(Searcher.java:163)
> >         at photospace.web.tags.SearchTag.doStartTag(SearchTag.java:22)
> >         at
> >
> org.apache.jsp.doc.index_002dstatus_jsp._jspx_meth_space_search_0(org.apa=
che.jsp.doc.index_002dstatus_jsp:140)
>
> >         at
> >
> org.apache.jsp.doc.index_002dstatus_jsp.access$1(org.apache.jsp.doc.index=
_002dstatus_jsp:128)
>
> >         at
> >
> org.apache.jsp.doc.index_002dstatus_jsp$index_002dstatus_jspHelper.invoke=
1(org.apache.jsp.doc.index_002dstatus_jsp:511)
>
> >         at
> >
> org.apache.jsp.doc.index_002dstatus_jsp$index_002dstatus_jspHelper.invoke=
(org.apache.jsp.doc.index_002dstatus_jsp:570)
>
> >         at
> >
> org.apache.jsp.tag.web.main_002dlayout_tag.doTag(org.apache.jsp.tag.web.m=
ain_002dlayout_tag:281)
>
> >         at
> >
> org.apache.jsp.doc.index_002dstatus_jsp._jspx_meth_tags_main$1layout_0(or=
g.apache.jsp.doc.index_002dstatus_jsp:108)
>
> >         at
> >
> org.apache.jsp.doc.index_002dstatus_jsp._jspService(org.apache.jsp.doc.in=
dex_002dstatus_jsp:82)
>
> >         at
> > org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
> >         at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
> >         at
> >
> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.jav=
a:322)
>
> >         at
> > org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:291=
)
> >         at
> > org.apache.jasper.servlet.JspServlet.service(JspServlet.java:241)
> >
> >
> >  The debugs look fine and reasonable to me. Maybe the problem is that t=
he
> > Directory does not exists that it is trying to rename to.
> >
> > that why I tried the FileUtils.cleanDirectory(index); instead of
> > FileUtils.deleteDirectory(index);
> >
> > in this case the context starts up, but still when clicking on build
> index
> > I get this:
> >
> > 19:33:16,816 DEBUG [TP-Processor1]
> > org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
> instance
> > 19:33:17,082 DEBUG [TP-Processor1]
> > org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
> instance
> > 19:33:17,093 DEBUG [TP-Processor1]
> > org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
> instance
> > 19:33:17,102 DEBUG [TP-Processor1]
> > org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
> instance
> > 19:33:17,111 DEBUG [TP-Processor1]
> > org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
> instance
> > 19:33:19,781 DEBUG [TP-Processor1]
> > org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
> instance
> >  INFO [2005-11-09 19:33:19,796] [Thread-66] (IndexingJob.java:71) -
> > Indexing ... 1 files
> > 19:33:19,853 DEBUG [TP-Processor1]
> > org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
> instance
> >  INFO [2005-11-09 19:33:19,878] [Thread-66] (SearchIndex.java:55) -
> > ****************** Can I write on /web/photospace/data/index ? true
> >  INFO [2005-11-09 19:33:19,884] [Thread-66] (SearchIndex.java:58) -
> > ****************** Cleaning /web/photospace/data/index
> >  INFO [2005-11-09 19:33:19,885] [Thread-66] (SearchIndex.java:59) -
> > ****************** Does /web/photospace/data/index still exist? true
> >  INFO [2005-11-09 19:33:19,885] [Thread-66] (SearchIndex.java:60) -
> > ****************** Can I write on
> /usr/local/tomcat2/temp/photospace-index
> > ? true
> >  INFO [2005-11-09 19:33:19,885] [Thread-66] (SearchIndex.java:61) -
> > ****************** Can I write on /web/photospace/data/index ? true
> > ERROR [2005-11-09 19:33:19,888] [Thread-66] (IndexingJob.java:109) -
> Error
> > indexing
> > java.io.IOException: Unable to move temporary index
> > /usr/local/tomcat2/temp/photospace-index to /web/photospace/data/index
> >         at photospace.search.SearchIndex.update(SearchIndex.java:64)
> >         at photospace.search.IndexerImpl.index(IndexerImpl.java:78)
> >         at photospace.search.IndexingJob.run(IndexingJob.java:99)
> >         at java.lang.Thread.run(Thread.java:595)
> >
> >
> > Again the debugs look fine to me, no clue why the renameTo fails though
> ...
> > ?
> >
> >
> > thanks for taking the time.
> > Why don't you reply to the mailing list ? Looks so dead on sourceforge =
-
> no
> > posts, and yet its so active ?
> >
> >
> >
> > ______________________________________
> > Tobias Schoessler, Java Developer
> > Information Management Unit
> > Information Technology Service
> > United Nations Office on Drugs and Crime
> >
> > Tel:  (+43-1) 26060-5173
> > Websites: www.unov.org, www.unodc.org
> > ______________________________________
> > Impossible is not a fact, only an opinion
> >
> >
> >
> >              alon salant
> >              <al...@sa...>
> >              Sent by:
> To
> >              as...@gm...         Tobias SCHOESSLER
> >                                        <Tob...@un...>
> >
> cc
> >              Wednesday, 9
> >              November 2005
> Subject
> >              18:43                     Re: [photospace-developer]
> problems
> >                                        building index on
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Ahh, sorry missed the end of your message. You have checked permissions=
.
> >
> > The problem can only be file system related. Here's the source for
> > SearchIndex.update():
> >
> >   public void update(File updated) throws IOException
> >   {
> >     synchronized(this)
> >     {
> >       closeReader();
> >       FileUtils.deleteDirectory(index);
> >       if (!updated.renameTo(index))
> >         throw new IOException("Unable to move temporary index " +
> > updated + " to " + index);
> >     }
> >   }
> >
> > FileUtils is part of Jakarta Commons Lang and does a recursive delete.
> > File.renameTo() is part of the Java core. File.renameTo() is failing
> > either because the old index did not get deleted (in your case I
> > believe it has not even been created yet) or because it can't write to
> > /web/photospace/data/index.
> >
> > If you're up for it, I'd recommend making sure that tomcat can write
> > to that location. Just write a JSP that does File.renameTo() to that
> > location and see if it works.
> >
> > Alon
> >
> > On 11/9/05, alon salant <al...@sa...> wrote:
> > > The first thing I would look at are file system permissions. Can you
> > > confirm that the user under which tomcat is running has permission to
> > > write to /web/photospace/data/index?
> > >
> > > Alon
> > >
> > > On 11/9/05, Tobias SCHOESSLER <Tob...@un...> wrote:
> > > >
> > > > I am trying to run photospace on a linux server.
> > > >
> > > > when building the index I get this exception:
> > > >
> > > > 18:04:12,494 DEBUG [TP-Processor3]
> > > > org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
> > instance
> > > > 18:04:16,091 DEBUG [TP-Processor2]
> > > > org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
> > instance
> > > > 18:04:21,718 DEBUG [TP-Processor3]
> > > > org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
> > instance
> > > >  INFO [2005-11-09 18:04:21,736] [Thread-47] (IndexingJob.java:71) -
> > > > Indexing 1 files
> > > > 18:04:21,748 DEBUG [TP-Processor2]
> > > > org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
> > instance
> > > > ERROR [2005-11-09 18:04:21,857] [Thread-47] (IndexingJob.java:109) =
-
> > Error
> > > > indexing
> > > > java.io.IOException: Unable to move temporary index
> > > > /usr/local/tomcat2/temp/photospace-index to
> /web/photospace/data/index
> > > >         at photospace.search.SearchIndex.update(SearchIndex.java:59=
)
> > > >         at photospace.search.IndexerImpl.index(IndexerImpl.java:78)
> > > >         at photospace.search.IndexingJob.run(IndexingJob.java:99)
> > > >         at java.lang.Thread.run(Thread.java:595)
> > >
> >
> >
> >
>
>
> -------------------------------------------------------
> SF.Net email is sponsored by:
> Tame your development challenges with Apache's Geronimo App Server.
> Download
> it for free - -and be entered to win a 42" plasma tv or your very own
> Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
> _______________________________________________
> photospace-developer mailing list
> pho...@li...
> https://lists.sourceforge.net/lists/listinfo/photospace-developer
>
>
>
>
> -------------------------------------------------------
> SF.Net email is sponsored by:
> Tame your development challenges with Apache's Geronimo App Server. Downl=
oad
> it for free - -and be entered to win a 42" plasma tv or your very own
> Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
> _______________________________________________
> photospace-developer mailing list
> pho...@li...
> https://lists.sourceforge.net/lists/listinfo/photospace-developer
>