From: Demian K. <dem...@vi...> - 2013-05-29 15:26:38
|
The VuFind 2 master branch was recently upgraded to Jetty 8 -- have you not pulled in a while, or is there some Jetty 6 code still lurking in there somewhere? - Demian > -----Original Message----- > From: Tod Olson [mailto:to...@uc...] > Sent: Wednesday, May 29, 2013 11:21 AM > To: Demian Katz > Cc: Tod Olson; Mark Triggs; vuf...@li... Tech > Subject: Re: [VuFind-Tech] alphabetic browse indexing: not finding SQLite > library > > Okay, progress is being made. First thing is that VuFind still ships with > jetty 6, so this sort of invocation will do what I need: > > /usr/local/bin/java -server -Xms2048m -Xmx2048m \ > -XX:+UseParallelGC -XX:NewRatio=5 \ > -Dsolr.solr.home=$SOLR_HOME \ > -Djetty.logs=$JETTY_LOG \ > -Djetty.home=$JETTY_HOME \ > -cp $JETTY_HOME/start.jar:/usr/local/share/java/classes/sqlitejdbc- > native.jar \ > org.mortbay.start.Main $JETTY_HOME/etc/jetty.xml > > Also, it looks like the most recent jetty (version 9) will allow one to pass > in options on the command line to prepend and post-pend to the class path. So > if there is ever a need to upgrade jetty, the command-line invocation becomes > a bit more flexible. > > -Tod > > > On May 29, 2013, at 7:50 AM, Demian Katz <dem...@vi...> > wrote: > > > Yes, that does make sense. Hmm. I'm also not familiar enough with this > aspect of Java to recommend a best practice here.... > > > > - Demian > > > >> -----Original Message----- > >> From: Tod Olson [mailto:to...@uc...] > >> Sent: Wednesday, May 29, 2013 8:47 AM > >> To: Demian Katz > >> Cc: Tod Olson; Mark Triggs; vuf...@li... Tech > >> Subject: Re: [VuFind-Tech] alphabetic browse indexing: not finding SQLite > >> library > >> > >> Different error on the stack trace when calling the browse handler: > >> > >> java.lang.ClassNotFoundException: org.sqlite.JDBC > >> > >> So instead of finding the class but being unable to load the shared object > >> code, now it doesn't find any version of the class. Which I think is what > we > >> would expect, given that I'm still invoking with -jar. > >> > >> -Tod > >> > >> On May 29, 2013, at 7:34 AM, Demian Katz <dem...@vi...> > >> wrote: > >> > >>> What happens if you delete the Xerial driver from solr/lib? > >>> > >>> - Demian > >>> > >>>> -----Original Message----- > >>>> From: Tod Olson [mailto:to...@uc...] > >>>> Sent: Wednesday, May 29, 2013 6:45 AM > >>>> To: Demian Katz > >>>> Cc: Tod Olson; Mark Triggs; vuf...@li... Tech > >>>> Subject: Re: [VuFind-Tech] alphabetic browse indexing: not finding SQLite > >>>> library > >>>> > >>>> Well, I spoke too soon. Works fine for index-alphabetical.sh, but no > matter > >>>> how I jigger CLASSPATH or -cp, Jetty still loads the Xerial driver from > >>>> solr/lib. Maybe that's because Jetty is started using the -jar option: > >>>> > >>>> /usr/local/bin/java -server -Xms2048m -Xmx2048m \ > >>>> -XX:+UseParallelGC -XX:NewRatio=5 \ > >>>> -Dsolr.solr.home=/data/magma/vufind2/solr \ > >>>> -Djetty.logs=/data/magma/vufind2/solr/jetty/logs \ > >>>> -Djetty.home=/data/magma/vufind2/solr/jetty \ > >>>> -jar /data/magma/vufind2/solr/jetty/start.jar \ > >>>> /data/magma/vufind2/solr/jetty/etc/jetty.xml > >>>> > >>>> If I understand correctly, -jar means CLASSPATH is ignored. I *think* the > >>>> options are to: > >>>> 1. jam the Zentus driver into the jar file, > >>>> 2. mess around with the MANIFEST in jar file, or > >>>> 3. not use the -jar switch and explicitly name the main class. > >>>> > >>>> Since I'm looking for portability, I think that 1 & 2 are > contraindicated, > >>>> therefore 3 is the way to go. But we're now into an area of java I'm not > so > >>>> comfortable with, so any advice is appreciated. > >>>> > >>>> Best, > >>>> > >>>> -Tod > >>>> > >>>> On May 28, 2013, at 10:57 AM, Demian Katz <dem...@vi...> > >>>> wrote: > >>>> > >>>>> Thanks for sharing this! > >>>>> > >>>>> Might be worth adding a note to the alphabrowse wiki page for other > >> FreeBSD > >>>> users... > >>>>> > >>>>> - Demian > >>>>> > >>>>>> -----Original Message----- > >>>>>> From: Tod Olson [mailto:to...@uc...] > >>>>>> Sent: Tuesday, May 28, 2013 11:47 AM > >>>>>> To: Mark Triggs > >>>>>> Cc: vuf...@li... Tech > >>>>>> Subject: Re: [VuFind-Tech] alphabetic browse indexing: not finding > SQLite > >>>>>> library > >>>>>> > >>>>>> Well, this becomes more complicated. Here's the upshot: > >>>>>> > >>>>>> VuFind ships with the Xerial SQLite JDBC driver. It's an offshoot of > the > >>>>>> Zentus SQLite JDBC driver which bundles in the shared object code. > >>>>>> Unfortunately, I couldn't build a working native version for FreeBSD. > >>>>>> > >>>>>> FreeBSD has a port for the older Zentus SQLite JDBC driver, that's > where > >>>>>> sqlitejdbc-native.jar came from in the first place. My solution is to > >> hack > >>>> the > >>>>>> index-alphabetical.sh script to include the Zentus driver if it is > >>>> installed. > >>>>>> That lets me run on FreeBSD but not screw up deployment on Linux. > >>>>>> > >>>>>> I doubt this is of sufficiently broad interest to merge it into the > base > >>>> code. > >>>>>> > >>>>>> -Tod > >>>>>> > >>>>>> > >>>>>> On May 20, 2013, at 9:31 PM, Tod Olson <to...@uc...> wrote: > >>>>>> > >>>>>>> Right! sqlitejdbc-native.jar repeats 12 classes in the sqlite-jdbc- > >> 3.7.15- > >>>>>> SNAPSHOT.jar. So I'll try stacking the order when I get into the office > >> and > >>>>>> see how it goes. > >>>>>>> > >>>>>>> Thanks! > >>>>>>> > >>>>>>> -Tod > >>>>>>> > >>>>>>> > >>>>>>> On May 20, 2013, at 4:11 PM, Mark Triggs <ma...@di...> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> Hi Tod, > >>>>>>>> > >>>>>>>> What's the difference between sqlitejdbc-native.jar and > >>>>>>>> sqlite-jdbc-3.7.15-SNAPSHOT.jar here? Is there any overlap to the > >>>>>>>> classes those .jar files provide? > >>>>>>>> > >>>>>>>> Does it make a difference if you force the jar files to load in a > >>>>>>>> particular order? Any difference between: > >>>>>>>> > >>>>>>>> java -Dfile.encoding=UTF-8 -cp '../solr/lib/sqlitejdbc- > >>>>>> native.jar:../solr/lib/sqlite-jdbc-3.7.15-SNAPSHOT.jar:browse-indexing- > >>>> multi- > >>>>>> title.jar:../solr/lib/*' CreateBrowseSQLite sorted-title.tmp > >>>> title_browse.db > >>>>>>>> > >>>>>>>> versus: > >>>>>>>> > >>>>>>>> java -Dfile.encoding=UTF-8 -cp '../solr/lib/sqlite-jdbc-3.7.15- > >>>>>> SNAPSHOT.jar:../solr/lib/sqlitejdbc-native.jar:browse-indexing-multi- > >>>>>> title.jar:../solr/lib/*' CreateBrowseSQLite sorted-title.tmp > >>>> title_browse.db > >>>>>>>> > >>>>>>>> that? > >>>>>>>> > >>>>>>>> I'm just thinking that the '../solr/lib/*' wildcard probably adds > jars > >>>>>>>> to the classpath in inode order, and that's the sort of thing that > >> could > >>>>>>>> vary between machines. > >>>>>>>> > >>>>>>>> Cheers, > >>>>>>>> > >>>>>>>> Mark > >>>>>>>> > >>>>>>>> > >>>>>>>> Tod Olson <to...@uc...> writes: > >>>>>>>> > >>>>>>>>> vufind-tech, > >>>>>>>>> > >>>>>>>>> I have a problem with one machine that's unable to create the SQLite > >>>>>> indexes because the browse-indexer can't find the SQLite code. The > >> problem > >>>> is > >>>>>> unique to this host, other local are building the browse indexes just > >> fine. > >>>>>> The jar file being executed is a local version of the browse indexer, > and > >>>> it > >>>>>> worked just fine on this host a few weeks ago. Clearly something has > >>>> changed > >>>>>> on this host, but I've been unable to identify what, so I turn to the > >> list. > >>>>>>>>> > >>>>>>>>> Here's the failure message that we see on the one host: > >>>>>>>>> > >>>>>>>>> diamond% java -Dfile.encoding=UTF-8 -cp 'browse-indexing-multi- > >>>>>> title.jar:../solr/lib/*' CreateBrowseSQLite sorted-title.tmp > >>>> title_browse.db > >>>>>>>>> Exception in thread "main" java.sql.SQLException: Error opening > >>>> connection > >>>>>>>>> at org.sqlite.Conn.open(Conn.java:165) > >>>>>>>>> at org.sqlite.Conn.<init>(Conn.java:87) > >>>>>>>>> at org.sqlite.JDBC.createConnection(JDBC.java:113) > >>>>>>>>> at org.sqlite.JDBC.connect(JDBC.java:87) > >>>>>>>>> at java.sql.DriverManager.getConnection(DriverManager.java:620) > >>>>>>>>> at java.sql.DriverManager.getConnection(DriverManager.java:222) > >>>>>>>>> at CreateBrowseSQLite.create(CreateBrowseSQLite.java:128) > >>>>>>>>> at CreateBrowseSQLite.main(CreateBrowseSQLite.java:154) > >>>>>>>>> Caused by: java.lang.Exception: Error loading native library: > >>>>>> /org/sqlite/native/FreeBSD/amd64/libsqlitejdbc.so > >>>>>>>>> at > >>>>>> > >>>> > >> > org.sqlite.SQLiteJDBCLoader.loadSQLiteNativeLibrary(SQLiteJDBCLoader.java:241) > >>>>>>>>> at > >> org.sqlite.SQLiteJDBCLoader.initialize(SQLiteJDBCLoader.java:63) > >>>>>>>>> at org.sqlite.NativeDB.load(NativeDB.java:50) > >>>>>>>>> at org.sqlite.Conn.open(Conn.java:161) > >>>>>>>>> ... 7 more > >>>>>>>>> > >>>>>>>>> Can't find the SQLite native library code, that seems > straightforward. > >>>>>>>>> > >>>>>>>>> On successful host magma, truss shows that java finds the SQLite > JDBC > >>>> and > >>>>>> native object code here: > >>>>>>>>> > >>>>>>>>> /data/magma/vufind2/solr/lib/sqlitejdbc-native.jar > >>>>>>>>> /data/magma/vufind2/solr/lib/sqlite-jdbc-3.7.15-SNAPSHOT.jar > >>>>>>>>> /usr/local/lib/libsqlitejdbc.so > >>>>>>>>> /usr/local/lib/libsqlite3.so.8 > >>>>>>>>> > >>>>>>>>> These files all exist on the unsuccessful host, with the same > >> checksums. > >>>>>> Same for browse-indexing-multi-title.jar. So both hosts have the same > >>>> object > >>>>>> code. Java is at the same version, OpenJDK Runtime Environment (build > >>>>>> 1.6.0_32-b27). > >>>>>>>>> > >>>>>>>>> What _is_ different is that on the unsuccessful host, according to > >>>> truss: > >>>>>>>>> > >>>>>>>>> 1. java opens the sqlite jar files in the opposite order, and > >>>>>>>>> 2. _never_ stats for libsqlitejdbc.so. > >>>>>>>>> > >>>>>>>>> That second bit is what gets me. The successful host stats for the > >>>> shared > >>>>>> object file in the various java and system libraries, the unsuccessful > >> host > >>>>>> NEVER call stat() or access() or in any way looks for libsqlitejdbc.so. > >>>>>>>>> > >>>>>>>>> Looking at the dynamic linking on both systems, LD_LIBRARY_PATH is > not > >>>>>> set, setting it does not seem to make a difference, and both systems > list > >>>>>> /usr/local/lib in "search directories" /var/run/ld-elf.so.hints. So I > >>>> _think_ > >>>>>> that the load library configuration is _not_ to blame here. > >>>>>>>>> > >>>>>>>>> So what does that leave? Some conditional in the Java code? a config > >>>> file > >>>>>> I'm not thinking of? I'm almost tempted to walk through the command- > line > >>>>>> debugger, so any ideas would be most welcome. > >>>>>>>> > >>>>>>>> -- > >>>>>>>> Mark Triggs > >>>>>>>> <ma...@di...> > >>>>>>> > >>>>>> > >>>>>> > >>>>>> ----------------------------------------------------------------------- > -- > >> -- > >>>> --- > >>>>>> Try New Relic Now & We'll Send You this Cool Shirt > >>>>>> New Relic is the only SaaS-based application performance monitoring > >> service > >>>>>> that delivers powerful full stack analytics. Optimize and monitor your > >>>>>> browser, app, & servers with just a few lines of code. Try New Relic > >>>>>> and get this awesome Nerd Life shirt! > >> http://p.sf.net/sfu/newrelic_d2d_may > >>>>>> _______________________________________________ > >>>>>> Vufind-tech mailing list > >>>>>> Vuf...@li... > >>>>>> https://lists.sourceforge.net/lists/listinfo/vufind-tech > >>> > > |