Re: [photospace-developer] problems building index on

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

ok, I got it fixed.

It comes down to this jdk bug

http://h21007.www2.hp.com/cmdspp/QuestionAnswer/1,1764,95
F27FD6-4941-11D6-B748-0003470C0153,00.html

renameTo seems to have problems renaming files across filesystems.

In my case we have tomcat and its temp dir on /usr/local and our web
deployment on something like /web on different partitions.

I patched SearchIndex.update(File updated) like this.

      public void update(File updated) throws IOException {
            synchronized (this) {
                  closeReader();
                  FileUtils.cleanDirectory(index);
                  FileUtils.copyDirectory(updated, index);
            }
      }

I had to upgrade the commons.io to 1.1 to support the directory copy.
Commons copyDirectory is probably less efficient than the filesystem's
rename implementation but this probably works on a broader range of
plattforms.

I tested windows 2000 and hp machine with some debian linux.

thanks alon salant
thanks to Philip S. Constantinou pointing out the filesystem issues with
renameTo.

______________________________________
Tobias Schoessler, Java Developer
Information Management Unit
Information Technology Service
United Nations Office on Drugs and Crime

Tel:  (+43-1) 26060-5173
Websites: www.unov.org, www.unodc.org
______________________________________
Impossible is not a fact, only an opinion

             alon salant                                                   
             <al...@sa...>                                             
             Sent by:                                                   To 
             photospace-develo         pho...@li... 
             per-admin@lists.s         rge.net                             
             ourceforge.net                                             cc 

                                                                   Subject 
             Wednesday, 9              Re: [photospace-developer] problems 
             November 2005             building index on                   
             21:20                                                         

Ah, didn't realize I wasn't replying to the list. The project has
actually not been very active lately - mostly b/c I have been working
on other things.

Did you try using File.renameTo() in a different context like a JSP?
Looking at the source for File.renameTo() there's not much going on.
There is a check against the SecurityManager which would throw an
exception if it failed (I think) and then a native call to the file
system:

    public boolean renameTo(File dest) {
             SecurityManager security = System.getSecurityManager();
             if (security != null) {
                 security.checkWrite(path);
                 security.checkWrite(dest.path);
             }
             return fs.rename(this, dest);
    }

It looks like fs.rename() is returning false. Why? Hmmm...

On 11/9/05, Tobias SCHOESSLER <Tob...@un...> wrote:
> Ok, I am digging into this further.
>
> I added debug logs to SearchIndex.java
>
>       public void update(File updated) throws IOException {
>             synchronized (this) {
>                   closeReader();
>                   log.info("****************** Can I write on "+index+" ?
> "+ index.canWrite() );
>                   FileUtils.deleteDirectory(index);
> //                FileUtils.cleanDirectory(index);
>                   log.info("****************** Cleaning "+index);
>                   log.info("****************** Does "+index+" still
exist?
> "+ index.exists() );
>                   log.info("****************** Can I write on "+updated+"
?
> "+ updated.canWrite() );
>                   log.info("****************** Can I write on "+index+" ?
> "+ index.canWrite() );
>
>                   if (!updated.renameTo(index))
>                         throw new IOException("Unable to move temporary
> index "
>                                     + updated + " to " + index);
>             }
>       }
>
>
> server log at context STARTUP - so context doesn't even start in this
case.
>
> INFO: Find registry server-registry.xml at classpath resource
> Nov 9, 2005 7:20:56 PM org.apache.catalina.startup.Catalina start
> INFO: Server startup in 34519 ms
> 19:20:59,799 DEBUG [TP-Processor3]
> org.apache.catalina.core.StandardWrapper:748 - Allocating non-STM
instance
> Nov 9, 2005 7:20:59 PM org.apache.catalina.core.ApplicationContext log
> INFO: HTMLManager: init: Associated with Deployer
> 'Catalina:type=Deployer,host=tomcat5.unodc.org'
> Nov 9, 2005 7:20:59 PM org.apache.catalina.core.ApplicationContext log
> INFO: HTMLManager: init: Global resources are available
> 19:20:59,824 DEBUG [TP-Processor3]
> org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
instance
> Nov 9, 2005 7:20:59 PM org.apache.catalina.core.ApplicationContext log
> INFO: HTMLManager: list: Listing contexts for virtual host
> 'tomcat5.unodc.org'
> 19:21:08,987 DEBUG [TP-Processor3]
> org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
instance
> 19:21:12,703 DEBUG [TP-Processor3]
> org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
instance
>  INFO [2005-11-09 19:21:12,723] [Thread-47] (IndexingJob.java:71) -
> Indexing ... 1 files
> 19:21:12,773 DEBUG [TP-Processor3]
> org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
instance
>  INFO [2005-11-09 19:21:12,841] [Thread-47] (SearchIndex.java:55) -
> ****************** Can I write on /web/photospace/data/index ? true
>  INFO [2005-11-09 19:21:12,849] [Thread-47] (SearchIndex.java:58) -
> ****************** Deleting /web/photospace/data/index
>  INFO [2005-11-09 19:21:12,849] [Thread-47] (SearchIndex.java:59) -
> ****************** Does /web/photospace/data/index still exist? false
>  INFO [2005-11-09 19:21:12,849] [Thread-47] (SearchIndex.java:60) -
> ****************** Can I write on
/usr/local/tomcat2/temp/photospace-index
> ? true
>  INFO [2005-11-09 19:21:12,849] [Thread-47] (SearchIndex.java:61) -
> ****************** Can I write on /web/photospace/data/index ? false
> ERROR [2005-11-09 19:21:12,856] [Thread-47] (IndexingJob.java:109) -
Error
> indexing
> java.io.IOException: Unable to move temporary index
> /usr/local/tomcat2/temp/photospace-index to /web/photospace/data/index
>         at photospace.search.SearchIndex.update(SearchIndex.java:64)
>         at photospace.search.IndexerImpl.index(IndexerImpl.java:78)
>         at photospace.search.IndexingJob.run(IndexingJob.java:99)
>         at java.lang.Thread.run(Thread.java:595)
> 19:21:42,910 DEBUG [TP-Processor2]
> org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
instance
> ERROR [2005-11-09 19:21:42,915] [TP-Processor2]
> (StandardWrapperValve.java:253) - Servlet.service() for servlet jsp threw
> exception
> java.lang.IllegalStateException: Search index has not been created yet
>         at photospace.search.SearchIndex.getReader(SearchIndex.java:26)
>         at photospace.search.Searcher.search(Searcher.java:233)
>         at photospace.search.Searcher.search(Searcher.java:178)
>         at photospace.search.Searcher.search(Searcher.java:163)
>         at photospace.web.tags.SearchTag.doStartTag(SearchTag.java:22)
>         at
>
org.apache.jsp.doc.index_002dstatus_jsp._jspx_meth_space_search_0(org.apache.jsp.doc.index_002dstatus_jsp:140)

>         at
>
org.apache.jsp.doc.index_002dstatus_jsp.access$1(org.apache.jsp.doc.index_002dstatus_jsp:128)

>         at
>
org.apache.jsp.doc.index_002dstatus_jsp$index_002dstatus_jspHelper.invoke1(org.apache.jsp.doc.index_002dstatus_jsp:511)

>         at
>
org.apache.jsp.doc.index_002dstatus_jsp$index_002dstatus_jspHelper.invoke(org.apache.jsp.doc.index_002dstatus_jsp:570)

>         at
>
org.apache.jsp.tag.web.main_002dlayout_tag.doTag(org.apache.jsp.tag.web.main_002dlayout_tag:281)

>         at
>
org.apache.jsp.doc.index_002dstatus_jsp._jspx_meth_tags_main$1layout_0(org.apache.jsp.doc.index_002dstatus_jsp:108)

>         at
>
org.apache.jsp.doc.index_002dstatus_jsp._jspService(org.apache.jsp.doc.index_002dstatus_jsp:82)

>         at
> org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
>         at
>
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:322)

>         at
> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:291)
>         at
> org.apache.jasper.servlet.JspServlet.service(JspServlet.java:241)
>
>
>  The debugs look fine and reasonable to me. Maybe the problem is that the
> Directory does not exists that it is trying to rename to.
>
> that why I tried the FileUtils.cleanDirectory(index); instead of
> FileUtils.deleteDirectory(index);
>
> in this case the context starts up, but still when clicking on build
index
> I get this:
>
> 19:33:16,816 DEBUG [TP-Processor1]
> org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
instance
> 19:33:17,082 DEBUG [TP-Processor1]
> org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
instance
> 19:33:17,093 DEBUG [TP-Processor1]
> org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
instance
> 19:33:17,102 DEBUG [TP-Processor1]
> org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
instance
> 19:33:17,111 DEBUG [TP-Processor1]
> org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
instance
> 19:33:19,781 DEBUG [TP-Processor1]
> org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
instance
>  INFO [2005-11-09 19:33:19,796] [Thread-66] (IndexingJob.java:71) -
> Indexing ... 1 files
> 19:33:19,853 DEBUG [TP-Processor1]
> org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
instance
>  INFO [2005-11-09 19:33:19,878] [Thread-66] (SearchIndex.java:55) -
> ****************** Can I write on /web/photospace/data/index ? true
>  INFO [2005-11-09 19:33:19,884] [Thread-66] (SearchIndex.java:58) -
> ****************** Cleaning /web/photospace/data/index
>  INFO [2005-11-09 19:33:19,885] [Thread-66] (SearchIndex.java:59) -
> ****************** Does /web/photospace/data/index still exist? true
>  INFO [2005-11-09 19:33:19,885] [Thread-66] (SearchIndex.java:60) -
> ****************** Can I write on
/usr/local/tomcat2/temp/photospace-index
> ? true
>  INFO [2005-11-09 19:33:19,885] [Thread-66] (SearchIndex.java:61) -
> ****************** Can I write on /web/photospace/data/index ? true
> ERROR [2005-11-09 19:33:19,888] [Thread-66] (IndexingJob.java:109) -
Error
> indexing
> java.io.IOException: Unable to move temporary index
> /usr/local/tomcat2/temp/photospace-index to /web/photospace/data/index
>         at photospace.search.SearchIndex.update(SearchIndex.java:64)
>         at photospace.search.IndexerImpl.index(IndexerImpl.java:78)
>         at photospace.search.IndexingJob.run(IndexingJob.java:99)
>         at java.lang.Thread.run(Thread.java:595)
>
>
> Again the debugs look fine to me, no clue why the renameTo fails though
...
> ?
>
>
> thanks for taking the time.
> Why don't you reply to the mailing list ? Looks so dead on sourceforge -
no
> posts, and yet its so active ?
>
>
>
> ______________________________________
> Tobias Schoessler, Java Developer
> Information Management Unit
> Information Technology Service
> United Nations Office on Drugs and Crime
>
> Tel:  (+43-1) 26060-5173
> Websites: www.unov.org, www.unodc.org
> ______________________________________
> Impossible is not a fact, only an opinion
>
>
>
>              alon salant
>              <al...@sa...>
>              Sent by:
To
>              as...@gm...         Tobias SCHOESSLER
>                                        <Tob...@un...>
>
cc
>              Wednesday, 9
>              November 2005
Subject
>              18:43                     Re: [photospace-developer]
problems
>                                        building index on
>
>
>
>
>
>
>
>
>
>
> Ahh, sorry missed the end of your message. You have checked permissions.
>
> The problem can only be file system related. Here's the source for
> SearchIndex.update():
>
>   public void update(File updated) throws IOException
>   {
>     synchronized(this)
>     {
>       closeReader();
>       FileUtils.deleteDirectory(index);
>       if (!updated.renameTo(index))
>         throw new IOException("Unable to move temporary index " +
> updated + " to " + index);
>     }
>   }
>
> FileUtils is part of Jakarta Commons Lang and does a recursive delete.
> File.renameTo() is part of the Java core. File.renameTo() is failing
> either because the old index did not get deleted (in your case I
> believe it has not even been created yet) or because it can't write to
> /web/photospace/data/index.
>
> If you're up for it, I'd recommend making sure that tomcat can write
> to that location. Just write a JSP that does File.renameTo() to that
> location and see if it works.
>
> Alon
>
> On 11/9/05, alon salant <al...@sa...> wrote:
> > The first thing I would look at are file system permissions. Can you
> > confirm that the user under which tomcat is running has permission to
> > write to /web/photospace/data/index?
> >
> > Alon
> >
> > On 11/9/05, Tobias SCHOESSLER <Tob...@un...> wrote:
> > >
> > > I am trying to run photospace on a linux server.
> > >
> > > when building the index I get this exception:
> > >
> > > 18:04:12,494 DEBUG [TP-Processor3]
> > > org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
> instance
> > > 18:04:16,091 DEBUG [TP-Processor2]
> > > org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
> instance
> > > 18:04:21,718 DEBUG [TP-Processor3]
> > > org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
> instance
> > >  INFO [2005-11-09 18:04:21,736] [Thread-47] (IndexingJob.java:71) -
> > > Indexing 1 files
> > > 18:04:21,748 DEBUG [TP-Processor2]
> > > org.apache.catalina.core.StandardWrapper:763 -   Returning non-STM
> instance
> > > ERROR [2005-11-09 18:04:21,857] [Thread-47] (IndexingJob.java:109) -
> Error
> > > indexing
> > > java.io.IOException: Unable to move temporary index
> > > /usr/local/tomcat2/temp/photospace-index to
/web/photospace/data/index
> > >         at photospace.search.SearchIndex.update(SearchIndex.java:59)
> > >         at photospace.search.IndexerImpl.index(IndexerImpl.java:78)
> > >         at photospace.search.IndexingJob.run(IndexingJob.java:99)
> > >         at java.lang.Thread.run(Thread.java:595)
> >
>
>
>

-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server.
Download
it for free - -and be entered to win a 42" plasma tv or your very own
Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
_______________________________________________
photospace-developer mailing list
pho...@li...
https://lists.sourceforge.net/lists/listinfo/photospace-developer