xtf-devel — *** DO NOT SIGN UP HERE! *** Go to groups.google.com/group/xtf-devel - thanks.

[Xtf-devel] XTF consultants?

[Jamie Orchard-H. <ja...@da...>]

I'm looking for some expertise in XTF on a project I'm doing for a  
client and figured this would be a good place to start looking.

The project involves setting up XTF and writing the XSLT for full- 
text journals in XML (TEI.2) and PDF formats.

Telecommuting works just fine.

If anyone has interest, or can point me to someone else, please feel  
free to email me directly.

Thanks,
Jamie

[Xtf-devel] Switching list to google groups

[Martin H. <m1...@sn...>]

Hello XTF developers,

We are switching this email list to Google Groups. Why?

We've had non-stop problems for weeks with the email list service at
SourceForge. While in general SourceForge is a great service, they are
perhaps overcommitted staff-wise, and haven't been able to address our
issues. In particular, some of us cannot post anything to the list, and
others cannot access the archives. We have to change strategy to keep up
communication with and between you, the XTF users.

We looked around, and decided that Google Groups can offer us a higher
degree of reliability in the long term, and a working email list in the
short term.

I am signing each of you up for the new group automatically, so you need do
nothing. You should receive a message from the new group shortly. In the
future, please post XTF user related questions to:

*xtf...@go... <xtf...@go...>*
<xtf...@go...>

Archives of messages from the old lists will still be searchable (as such)
on SourceForge. Messages for the new lists should be searched using Google
Gorups.

Thank you for your patience, and for your interest in XTF.

Martin Haye
an XTF developer

[Xtf-devel] [ xtf-Support Requests-1548143 ] More tips

[SourceForge.net <no...@so...>]

Support Requests item #1548143, was opened at 2006-08-28 12:36
Message generated for change (Comment added) made by mhaye
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684779&aid=1548143&group_id=119724

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Martin Haye (mhaye)
Assigned to: Martin Haye (mhaye)
Summary: More tips

Initial Comment:
It would be nice to have a couple more sections in the
"tips & tricks" on:

(1) Setting up Oxygen to run XTF. Include specifying
xtf.jar so that extension functions like
fileutils:exists work right. Also, setting up transform
scenario, running debugger, switching from debug to
edit mode and back. Also, where to get input files for
later stages (e.g. XTF-marked dynaXML input,
resultFormatter input, etc.)

(2) Setting up Eclipse to run XTF (for playing with the
guts of the Java code). There are two ways I know of.
First, is how CDL work is done internally, running
Tomcat as a servlet container. The other way is
described by Peter Murray here:

http://drc-dev.ohiolink.edu/wiki/EclipseXTFHowTo

We should include his way as well, if he grants permission.


----------------------------------------------------------------------

>Comment By: Martin Haye (mhaye)
Date: 2006-10-06 16:35

Message:
Logged In: YES 
user_id=978215

Likewise, we should have a section on how to build XTF from
the source. This involves: (1) unzipping the source; (2)
installing ant; (3) running ant (either just to compile, or
to build a jar.)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684779&aid=1548143&group_id=119724

[Xtf-devel] [ xtf-Bugs-1571681 ] Wildcards and accented chars

[SourceForge.net <no...@so...>]

Bugs item #1571681, was opened at 2006-10-05 12:59
Message generated for change (Comment added) made by mhaye
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684778&aid=1571681&group_id=119724

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: textEngine
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 7
Submitted By: Martin Haye (mhaye)
Assigned to: Martin Haye (mhaye)
Summary: Wildcards and accented chars

Initial Comment:
Reported by Tamara Lopez:

I have run into a problem recently that I believe you
and I addressed
a while ago, but which seems to have reappeared.  The
problem comes
when I try to apply a wildcard to a term containing an
accented
character.  It results in the following stack trace.

I have replicated this in the default xtf interface, so
I don't
believe it is related to changes I have made in the
XSLT.  The query
term I am using is : f*qué (the final e has an accent
acute)

Unfortunately, looking through my email and the list, I
cannot find
how this was fixed before (if it was in fact the same
issue).  Do you
by any chance recall?  Thanks for any help you can give
with this
problem.

Thanks,
Tamara


Servlet Error: ClassCast
An unexpected servlet error has occurred.

Message:

org.apache.lucene.search.spans.SpanWildcardQuery

Stack Trace:

java.lang.ClassCastException:
org.apache.lucene.search.spans.SpanWildcardQuery
at org.cdlib.xtf.textEngine.SlopFixupRewriter.rewrite
(SlopFixupRewriter.java:146)
at org.apache.lucene.search.QueryRewriter.rewriteQuery
(QueryRewriter.java(Compiled Code))
at org.cdlib.xtf.textEngine.XtfQueryRewriter.rewriteQuery
(XtfQueryRewriter.java:47)
at
org.apache.lucene.search.QueryRewriter.rewrite(QueryRewriter.java:
374)
at org.cdlib.xtf.textEngine.SlopFixupRewriter.rewrite
(SlopFixupRewriter.java:138)
at org.apache.lucene.search.QueryRewriter.rewriteQuery
(QueryRewriter.java(Compiled Code))
at org.cdlib.xtf.textEngine.XtfQueryRewriter.rewriteQuery
(XtfQueryRewriter.java:47)
at
org.cdlib.xtf.textEngine.DefaultQueryProcessor.processRequest
(DefaultQueryProcessor.java:234)
at
org.cdlib.xtf.crossQuery.CrossQuery.apply(CrossQuery.java:198)
at
org.cdlib.xtf.crossQuery.CrossQuery.doGet(CrossQuery.java:149)

----------------------------------------------------------------------

>Comment By: Martin Haye (mhaye)
Date: 2006-10-05 13:02

Message:
Logged In: YES 
user_id=978215

AccentQueryRewriter wasn't properly retaining the XTF-ness
of XtfWildcardQuery. Just checked in a fix.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684778&aid=1571681&group_id=119724

[Xtf-devel] [ xtf-Bugs-1571681 ] Wildcards and accented chars

[SourceForge.net <no...@so...>]

Bugs item #1571681, was opened at 2006-10-05 12:59
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684778&aid=1571681&group_id=119724

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: textEngine
Group: None
Status: Open
Resolution: None
Priority: 7
Submitted By: Martin Haye (mhaye)
Assigned to: Martin Haye (mhaye)
Summary: Wildcards and accented chars

Initial Comment:
Reported by Tamara Lopez:

I have run into a problem recently that I believe you
and I addressed
a while ago, but which seems to have reappeared.  The
problem comes
when I try to apply a wildcard to a term containing an
accented
character.  It results in the following stack trace.

I have replicated this in the default xtf interface, so
I don't
believe it is related to changes I have made in the
XSLT.  The query
term I am using is : f*qué (the final e has an accent
acute)

Unfortunately, looking through my email and the list, I
cannot find
how this was fixed before (if it was in fact the same
issue).  Do you
by any chance recall?  Thanks for any help you can give
with this
problem.

Thanks,
Tamara


Servlet Error: ClassCast
An unexpected servlet error has occurred.

Message:

org.apache.lucene.search.spans.SpanWildcardQuery

Stack Trace:

java.lang.ClassCastException:
org.apache.lucene.search.spans.SpanWildcardQuery
at org.cdlib.xtf.textEngine.SlopFixupRewriter.rewrite
(SlopFixupRewriter.java:146)
at org.apache.lucene.search.QueryRewriter.rewriteQuery
(QueryRewriter.java(Compiled Code))
at org.cdlib.xtf.textEngine.XtfQueryRewriter.rewriteQuery
(XtfQueryRewriter.java:47)
at
org.apache.lucene.search.QueryRewriter.rewrite(QueryRewriter.java:
374)
at org.cdlib.xtf.textEngine.SlopFixupRewriter.rewrite
(SlopFixupRewriter.java:138)
at org.apache.lucene.search.QueryRewriter.rewriteQuery
(QueryRewriter.java(Compiled Code))
at org.cdlib.xtf.textEngine.XtfQueryRewriter.rewriteQuery
(XtfQueryRewriter.java:47)
at
org.cdlib.xtf.textEngine.DefaultQueryProcessor.processRequest
(DefaultQueryProcessor.java:234)
at
org.cdlib.xtf.crossQuery.CrossQuery.apply(CrossQuery.java:198)
at
org.cdlib.xtf.crossQuery.CrossQuery.doGet(CrossQuery.java:149)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684778&aid=1571681&group_id=119724

[Xtf-devel] [ xtf-Feature Requests-1306981 ] Search by modification date

[SourceForge.net <no...@so...>]

Feature Requests item #1306981, was opened at 2005-09-28 07:47
Message generated for change (Comment added) made by mhaye
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684781&aid=1306981&group_id=119724

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 7
Submitted By: Martin Haye (mhaye)
Assigned to: Martin Haye (mhaye)
Summary: Search by modification date

Initial Comment:
XTF currently stores the modification date and time of
each document in the index. It would be quite useful to
be able to search by this field. This may be
complicated by the very granular nature of these
searches, but perhaps it is feasible to do it using
newer Lucene classes which support fine-grained searches.

----------------------------------------------------------------------

>Comment By: Martin Haye (mhaye)
Date: 2006-10-04 13:26

Message:
Logged In: YES 
user_id=978215

Added two new methods to FileUtils. It now has three:

FileUtils.exists(String filePath)
   Returns true if the file is readable, else false.

FileUtils.lastModified(String filePath, String format)
   Gets last modified date of a file, and formats it
   according to rules in SimpleDateFormatter.
   A typical string is: yyyy-MM-dd:HH:mm:ss

FileUtils.length(String filePath)
   Gets the length in bytes of a file, or -1 if it
   doesn't exist.


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2006-10-04 12:18

Message:
Logged In: NO 

A feature like this is needed for OAI; here is the 
datestamp info from the OAI spec.  This is the datestamp 
format that date range searches will come in using. 
 
http://www.openarchives.org/OAI/openarchivesprotocol.html#Dates 
 
3.3 UTCdatetime 
 
Dates and times are uniformly encoded using ISO8601 and 
are expressed in UTC throughout the protocol. When time is 
included, the special UTC designator ("Z") must be used. 
UTC is implied for dates although no timezone designator 
is specified. For example, 1957-03-20T20:30:00Z is UTC 
8:30:00 PM on March 20th 1957. UTCdatetime is used in both 
protocol requests and protocol replies, in the way 
described in the following sections. 
 

----------------------------------------------------------------------

Comment By: Martin Haye (mhaye)
Date: 2006-10-04 12:10

Message:
Logged In: YES 
user_id=978215

Problem: since the date/time is stored as a long, it's
actually difficult to use it in a meaningful way.

A better solution would be to provide a method in FileUtils
that gets a file's date/time in "W3C profile of ISO" date
format. Basically YYYY-MM-DD:HH:MM:SS

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684781&aid=1306981&group_id=119724

[Xtf-devel] [ xtf-Feature Requests-1306981 ] Search by modification date

[SourceForge.net <no...@so...>]

Feature Requests item #1306981, was opened at 2005-09-28 07:47
Message generated for change (Comment added) made by nobody
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684781&aid=1306981&group_id=119724

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 7
Submitted By: Martin Haye (mhaye)
Assigned to: Martin Haye (mhaye)
Summary: Search by modification date

Initial Comment:
XTF currently stores the modification date and time of
each document in the index. It would be quite useful to
be able to search by this field. This may be
complicated by the very granular nature of these
searches, but perhaps it is feasible to do it using
newer Lucene classes which support fine-grained searches.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2006-10-04 12:18

Message:
Logged In: NO 

A feature like this is needed for OAI; here is the 
datestamp info from the OAI spec.  This is the datestamp 
format that date range searches will come in using. 
 
http://www.openarchives.org/OAI/openarchivesprotocol.html#Dates 
 
3.3 UTCdatetime 
 
Dates and times are uniformly encoded using ISO8601 and 
are expressed in UTC throughout the protocol. When time is 
included, the special UTC designator ("Z") must be used. 
UTC is implied for dates although no timezone designator 
is specified. For example, 1957-03-20T20:30:00Z is UTC 
8:30:00 PM on March 20th 1957. UTCdatetime is used in both 
protocol requests and protocol replies, in the way 
described in the following sections. 
 

----------------------------------------------------------------------

Comment By: Martin Haye (mhaye)
Date: 2006-10-04 12:10

Message:
Logged In: YES 
user_id=978215

Problem: since the date/time is stored as a long, it's
actually difficult to use it in a meaningful way.

A better solution would be to provide a method in FileUtils
that gets a file's date/time in "W3C profile of ISO" date
format. Basically YYYY-MM-DD:HH:MM:SS

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684781&aid=1306981&group_id=119724

[Xtf-devel] [ xtf-Feature Requests-1306981 ] Search by modification date

[SourceForge.net <no...@so...>]

Feature Requests item #1306981, was opened at 2005-09-28 07:47
Message generated for change (Comment added) made by mhaye
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684781&aid=1306981&group_id=119724

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
>Priority: 7
Submitted By: Martin Haye (mhaye)
Assigned to: Martin Haye (mhaye)
Summary: Search by modification date

Initial Comment:
XTF currently stores the modification date and time of
each document in the index. It would be quite useful to
be able to search by this field. This may be
complicated by the very granular nature of these
searches, but perhaps it is feasible to do it using
newer Lucene classes which support fine-grained searches.

----------------------------------------------------------------------

>Comment By: Martin Haye (mhaye)
Date: 2006-10-04 12:10

Message:
Logged In: YES 
user_id=978215

Problem: since the date/time is stored as a long, it's
actually difficult to use it in a meaningful way.

A better solution would be to provide a method in FileUtils
that gets a file's date/time in "W3C profile of ISO" date
format. Basically YYYY-MM-DD:HH:MM:SS

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684781&aid=1306981&group_id=119724

[Xtf-devel] [ xtf-Support Requests-1285435 ] Improve docs for XTF home path

[SourceForge.net <no...@so...>]

Support Requests item #1285435, was opened at 2005-09-08 15:25
Message generated for change (Comment added) made by mhaye
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684779&aid=1285435&group_id=119724

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Martin Haye (mhaye)
Assigned to: Martin Haye (mhaye)
Summary: Improve docs for XTF home path

Initial Comment:
There should be a documentation section talking about
the XTF base path. For servlets, it defaults to (for
instance) resin_home/webapps/xtf. For textIndexer, it
uses the XTF_HOME environment variable.

Also document use of the base-dir servlet container
attribute to point the servlets at a different base
directory.

----------------------------------------------------------------------

>Comment By: Martin Haye (mhaye)
Date: 2006-10-03 17:43

Message:
Logged In: YES 
user_id=978215

The Deployment Guide now contains a section talking about
the base path and how it's configured. Also, since many
people starting out forget to set XTF_HOME before running
the textIndexer, there's a note in that section to remind them.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684779&aid=1285435&group_id=119724

[Xtf-devel] [ xtf-Bugs-1570248 ] Diacritics characters

[SourceForge.net <no...@so...>]

Bugs item #1570248, was opened at 2006-10-03 13:29
Message generated for change (Comment added) made by mhaye
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684778&aid=1570248&group_id=119724

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: crossQuery
Group: None
Status: Closed
Resolution: Fixed
Priority: 5
Submitted By: Joao Lima (joaolima)
Assigned to: Martin Haye (mhaye)
Summary: Diacritics characters

Initial Comment:
Diacritics characters are replace by URL escape 
characters when change the "sort by" order, confusing 
the crossQuery servlet. 

To reproduce this bug, follow this steps: 

1. At http://content.cdlib.org/search?style=eschol,
type the word América

2. At result page, change the "sort by" order to 
title.

3. The servlet show the message
"Your search for am and e9rica in text, title, 
creator, description found 0 books."



----------------------------------------------------------------------

>Comment By: Martin Haye (mhaye)
Date: 2006-10-03 17:08

Message:
Logged In: YES 
user_id=978215

My last comment didn't make clear that I have checked in a
fix. If anybody needs the fix, they can access the new code
from CVS directly, wait for the next release, or ask me for
a custom xtf-jar build.

----------------------------------------------------------------------

Comment By: Martin Haye (mhaye)
Date: 2006-10-03 16:19

Message:
Logged In: YES 
user_id=978215

The servlet was not properly decoding the escaped UTF-8
encoding in the URL when creating the "http.url" parameter.
Turns out the HttpServletRequest methods do not perform
un-escaping (nor UTF-8 decoding) on the URL, so we have to
do it explicitly.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684778&aid=1570248&group_id=119724

[Xtf-devel] [ xtf-Support Requests-1397377 ] New servlet.dir parameter; renamed servlet.path & root.path

[SourceForge.net <no...@so...>]

Support Requests item #1397377, was opened at 2006-01-04 16:20
Message generated for change (Comment added) made by mhaye
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684779&aid=1397377&group_id=119724

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 7
Submitted By: Martin Haye (mhaye)
Assigned to: Martin Haye (mhaye)
Summary: New servlet.dir parameter; renamed servlet.path & root.path

Initial Comment:
Document new "servlet.dir" parameter passed to all
stylesheets. Also, renamed servlet.path to servlet.URL;
renamed root.path to root.URL.

Finally, document the fact that the servlet dir can be
overridden by specifying the "base-dir" initialization
parameter in the servlet *container* (e.g. Resin,
Tomcat, etc.)

----------------------------------------------------------------------

>Comment By: Martin Haye (mhaye)
Date: 2006-10-03 17:04

Message:
Logged In: YES 
user_id=978215

The Tag Reference now contains proper documentation for all
of these parameters, and improved docs for the HTTP header
parameters.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684779&aid=1397377&group_id=119724

[Xtf-devel] [ xtf-Bugs-1570248 ] Diacritics characters

[SourceForge.net <no...@so...>]

Bugs item #1570248, was opened at 2006-10-03 13:29
Message generated for change (Settings changed) made by mhaye
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684778&aid=1570248&group_id=119724

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: crossQuery
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Joao Lima (joaolima)
Assigned to: Martin Haye (mhaye)
Summary: Diacritics characters

Initial Comment:
Diacritics characters are replace by URL escape 
characters when change the "sort by" order, confusing 
the crossQuery servlet. 

To reproduce this bug, follow this steps: 

1. At http://content.cdlib.org/search?style=eschol,
type the word América

2. At result page, change the "sort by" order to 
title.

3. The servlet show the message
"Your search for am and e9rica in text, title, 
creator, description found 0 books."



----------------------------------------------------------------------

Comment By: Martin Haye (mhaye)
Date: 2006-10-03 16:19

Message:
Logged In: YES 
user_id=978215

The servlet was not properly decoding the escaped UTF-8
encoding in the URL when creating the "http.url" parameter.
Turns out the HttpServletRequest methods do not perform
un-escaping (nor UTF-8 decoding) on the URL, so we have to
do it explicitly.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684778&aid=1570248&group_id=119724

[Xtf-devel] [ xtf-Bugs-1570248 ] Diacritics characters

[SourceForge.net <no...@so...>]

Bugs item #1570248, was opened at 2006-10-03 13:29
Message generated for change (Comment added) made by mhaye
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684778&aid=1570248&group_id=119724

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: crossQuery
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Joao Lima (joaolima)
Assigned to: Martin Haye (mhaye)
Summary: Diacritics characters

Initial Comment:
Diacritics characters are replace by URL escape 
characters when change the "sort by" order, confusing 
the crossQuery servlet. 

To reproduce this bug, follow this steps: 

1. At http://content.cdlib.org/search?style=eschol,
type the word América

2. At result page, change the "sort by" order to 
title.

3. The servlet show the message
"Your search for am and e9rica in text, title, 
creator, description found 0 books."



----------------------------------------------------------------------

>Comment By: Martin Haye (mhaye)
Date: 2006-10-03 16:19

Message:
Logged In: YES 
user_id=978215

The servlet was not properly decoding the escaped UTF-8
encoding in the URL when creating the "http.url" parameter.
Turns out the HttpServletRequest methods do not perform
un-escaping (nor UTF-8 decoding) on the URL, so we have to
do it explicitly.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684778&aid=1570248&group_id=119724

[Xtf-devel] [ xtf-Bugs-1570262 ] sectionType in near query fails

[SourceForge.net <no...@so...>]

Bugs item #1570262, was opened at 2006-10-03 13:50
Message generated for change (Comment added) made by mhaye
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684778&aid=1570262&group_id=119724

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: textEngine
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 7
Submitted By: Martin Haye (mhaye)
Assigned to: Martin Haye (mhaye)
Summary: sectionType in near query fails

Initial Comment:
Observed by Lan Zhang:

I’ve got the sectionType working already, so we can
search within folder list, biographical sketch and
indexes. But it only works when I leave the proximity
field blank. It gives me an error message when I try to
change proximity to something else and restrict my
search within a section of the document. The error
message looks like this:

Servlet Error: QueryGen
An unexpected servlet error has occurred.

Message:

Expected: 'query', 'term', 'all', 'range', 'phrase',
'exact', 'near', 'orNear', 'and', 'or', 'not', or
'moreLike'; found 'sectionType'

----------------------------------------------------------------------

>Comment By: Martin Haye (mhaye)
Date: 2006-10-03 13:53

Message:
Logged In: YES 
user_id=978215

This was a problem in the Java QueryRequestParser... it
simply wasn't recognizing the sectionType query properly.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684778&aid=1570262&group_id=119724

[Xtf-devel] [ xtf-Bugs-1570262 ] sectionType in near query fails

[SourceForge.net <no...@so...>]

Bugs item #1570262, was opened at 2006-10-03 13:50
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684778&aid=1570262&group_id=119724

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: textEngine
Group: None
Status: Open
Resolution: None
Priority: 7
Submitted By: Martin Haye (mhaye)
Assigned to: Martin Haye (mhaye)
Summary: sectionType in near query fails

Initial Comment:
Observed by Lan Zhang:

I’ve got the sectionType working already, so we can
search within folder list, biographical sketch and
indexes. But it only works when I leave the proximity
field blank. It gives me an error message when I try to
change proximity to something else and restrict my
search within a section of the document. The error
message looks like this:

Servlet Error: QueryGen
An unexpected servlet error has occurred.

Message:

Expected: 'query', 'term', 'all', 'range', 'phrase',
'exact', 'near', 'orNear', 'and', 'or', 'not', or
'moreLike'; found 'sectionType'

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684778&aid=1570262&group_id=119724

[Xtf-devel] [ xtf-Bugs-1570248 ] Diacritics characters

[SourceForge.net <no...@so...>]

Bugs item #1570248, was opened at 2006-10-03 17:29
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684778&aid=1570248&group_id=119724

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: crossQuery
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Joao Lima (joaolima)
Assigned to: Martin Haye (mhaye)
Summary: Diacritics characters

Initial Comment:
Diacritics characters are replace by URL escape 
characters when change the "sort by" order, confusing 
the crossQuery servlet. 

To reproduce this bug, follow this steps: 

1. At http://content.cdlib.org/search?style=eschol,
type the word América

2. At result page, change the "sort by" order to 
title.

3. The servlet show the message
"Your search for am and e9rica in text, title, 
creator, description found 0 books."



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684778&aid=1570248&group_id=119724

[Xtf-devel] [ xtf-Support Requests-1397377 ] New servlet.dir parameter; renamed servlet.path & root.path

[SourceForge.net <no...@so...>]

Support Requests item #1397377, was opened at 2006-01-04 16:20
Message generated for change (Settings changed) made by mhaye
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684779&aid=1397377&group_id=119724

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 7
Submitted By: Martin Haye (mhaye)
Assigned to: Martin Haye (mhaye)
>Summary: New servlet.dir parameter; renamed servlet.path & root.path

Initial Comment:
Document new "servlet.dir" parameter passed to all
stylesheets. Also, renamed servlet.path to servlet.URL;
renamed root.path to root.URL.

Finally, document the fact that the servlet dir can be
overridden by specifying the "base-dir" initialization
parameter in the servlet *container* (e.g. Resin,
Tomcat, etc.)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684779&aid=1397377&group_id=119724

[Xtf-devel] [ xtf-Support Requests-1285208 ] Document MARC parsing

[SourceForge.net <no...@so...>]

Support Requests item #1285208, was opened at 2005-09-08 11:41
Message generated for change (Settings changed) made by mhaye
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684779&aid=1285208&group_id=119724

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Martin Haye (mhaye)
Assigned to: Martin Haye (mhaye)
>Summary: Document MARC parsing

Initial Comment:
Document experimental capability in XTF to parse and
index thousands or millions of records from a MARC file.

Basically, each record is transformed using marc4j into
MARCXML format. These are passed one at a time to the
prefilters. The first of these will typically, but not
always, be a MARC2MODS conversion stylesheet.

Also note that dynaXML will not work on these records.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684779&aid=1285208&group_id=119724

[Xtf-devel] [ xtf-Bugs-1494725 ] preview mode adds extra nodes

[SourceForge.net <no...@so...>]

Bugs item #1494725, was opened at 2006-05-24 20:47
Message generated for change (Comment added) made by mhaye
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684778&aid=1494725&group_id=119724

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: dynaXML
Group: None
>Status: Closed
>Resolution: Duplicate
Priority: 5
Submitted By: Nobody/Anonymous (nobody)
Assigned to: Martin Haye (mhaye)
Summary: preview mode adds extra nodes

Initial Comment:
preview mode adds extra nodes

http://texts.cdlib.org/view?docId=ead-preview&doc.view=entire_text&source=http://voro.cdlib.org:8081/test-oac/submission/berkeley/bancroft/m77_185_cubanc.xml

http://texts-dev.cdlib.org/view?docId=ead-preview&doc.view=entire_text&source=http://voro.cdlib.org:8081/test-oac/submission/berkeley/bancroft/m77_185_cubanc.xml

----------------------------------------------------------------------

>Comment By: Martin Haye (mhaye)
Date: 2006-09-29 17:02

Message:
Logged In: YES 
user_id=978215

It turns out that preview mode was actually doing the
correct thing, and that normal lazy tree mode was not. The
lazy tree creation procedure was removing whitespace where
it isn't legal to do so in XML. By contrast, preview mode
was preserving the original whitespace in the document.

Now that we've fixed XTF to preserve whitespace by default,
there is no longer a discrepancy between the two modes.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684778&aid=1494725&group_id=119724

[Xtf-devel] [ xtf-Bugs-1548258 ] EADs don't show hits in context

[SourceForge.net <no...@so...>]

Bugs item #1548258, was opened at 2006-08-28 17:22
Message generated for change (Comment added) made by mhaye
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684778&aid=1548258&group_id=119724

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: dynaXML
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Martin Haye (mhaye)
Assigned to: Martin Haye (mhaye)
Summary: EADs don't show hits in context

Initial Comment:
David Gewirtz points out that the EAD support in XTF is
quite incomplete, in that it doesn't show search hits
in context when the finding aid is viewed in dynaXML.
This should be fixed, to properly show off XTF's
capabilities.

----------------------------------------------------------------------

>Comment By: Martin Haye (mhaye)
Date: 2006-09-29 17:00

Message:
Logged In: YES 
user_id=978215

Fixed and initial response looks good.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684778&aid=1548258&group_id=119724

[Xtf-devel] [ xtf-Bugs-1568112 ] lazyTree key generation bug

[SourceForge.net <no...@so...>]

Bugs item #1568112, was opened at 2006-09-29 16:59
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684778&aid=1568112&group_id=119724

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: dynaXML
Group: None
Status: Open
Resolution: None
Priority: 7
Submitted By: Martin Haye (mhaye)
Assigned to: Martin Haye (mhaye)
Summary: lazyTree key generation bug

Initial Comment:
dynaXML is crashing when trying to create an XSL key on
the 'id' attribute. It complains that a non-dynamic key
is using dynamic nodes. However, I don't think any of
the dynamic nodes have IDs, so this is a mystery that
needs to be resolved.

Here is the discover's email on the topic:

 Tamara Lopez 	
to Marin
Aug 31

Hi Martin,

Sorry for the delay in my reply - Thursdays are crazy
for me this
semester!

I have no problem sending you the code, however, I
think I have
isolated the problem, so I will just describe it here:

In short, there is an additional key to the ones I
mentioned in my
previous post:

<xsl:key name="IDS" match="*[@id]" use="@id"/>

This key is used in the the <ref> template to locate
the node to
which the reference should be made:

<xsl:template match="ref">
      <xsl:choose>
            <xsl:when test="$enableMouseoverNotes =
'true'">
                  <span class="tooltip"
onmouseout="hideTip()">
                        <xsl:attribute name="onmouseover">
                              <xsl:text>doTooltip
(event,document.getElementById("</xsl:text><xsl:value-of
select="@target"/>
                             
<xsl:text>").innerHTML)</xsl:text>
                        </xsl:attribute>
                        <xsl:value-of
select="$noteIndicatorEditorial"/>
                  </span>
            </xsl:when>
            <xsl:otherwise>
                  <a>
                        <xsl:call-template name="makeURL">
                              <xsl:with-param name="url">
                                    <xsl:apply-templates
mode="xrefheader" select="key('IDS',@target)"/>
                              </xsl:with-param>
                              <xsl:with-param name="class">
                                    <xsl:choose>
                                          <xsl:when
test="@rend"><xsl:value-of select="@rend"/></xsl:when>
                                         
<xsl:otherwise><xsl:value-
of select="$class_ref"/></xsl:otherwise>
                                    </xsl:choose>
                              </xsl:with-param>
                        </xsl:call-template>
                        <xsl:apply-templates/>
                  </a>
            </xsl:otherwise>
      </xsl:choose>
</xsl:template>

Though it may not be as performant, I should be able to
access the
node to reference using Xpath instead of the key.  I
will investigate
that and let you know if I have additional problems  -
hopefully this
information will be somewhat useful to you, however, as
I suppose
others could create similar keys.

I would be interested in any information you can give
about why this
particular key/template combination is problematic.

Thanks as usual for your help.

Tamara

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684778&aid=1568112&group_id=119724

[Xtf-devel] [ xtf-Feature Requests-1568110 ] Finer control of subdirectory indexing

[SourceForge.net <no...@so...>]

Feature Requests item #1568110, was opened at 2006-09-29 16:56
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684781&aid=1568110&group_id=119724

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: textIndexer
Group: None
Status: Open
Resolution: None
Priority: 6
Submitted By: Martin Haye (mhaye)
Assigned to: Martin Haye (mhaye)
Summary: Finer control of subdirectory indexing

Initial Comment:
Currently the docSelector stylesheet does not have
enough control over which directories are processed. In
particular, if a data file is processed in a directory,
none of the subdirectories are even considered. This
should be placed under the control of the docSelector
rather than hard coded in the textIndexer.

Following are email discussions on the topic.

 Martin Haye 	
to Jakob, xtf-user
  Aug 30

Ah, I see. So docSelector could choose to traverse a
given subdirectory by outputting an <indexDir> tag for
it, or something like that. And then the
"if(anyProcessed)" condition could be removed, and
docSelector would be in full control.

One downside is that it will break compatibility with
old docSelectors, but I guess the best time to do that
is now, when people are upgrading docSelector anyway.

Anybody have an objection to this proposal?



--Martin

On 8/29/06, Jakob Saternus <ja...@uu...> wrote:


    Martin,

    Right now the output of docSelector.xsl consists of
a single
    <indexFiles> container tag and one <indexFile.../>
tag for each
    document file that needs to be indexed. This output
is based on
    directories that already have been traversed.. so
option (2)
    seems really strange to me.

    Why not remove the "if (anyProcessed)" condition
and let docSelector
    decide which directories/files should be skipped?

    Thanks,
    Jakob


    Tuesday, August 29, 2006, 9:44:49 PM, you wrote:

    > Jakob,

    > You make a good point. It's a bit arbitrary for
the Java code to be making
    > this decision, when it should be the docSelector.
I'm not sure quite how it
    > should be accomplished, but I have two ideas:

    > (1) XTF could pass an extra attribute for each
<directory> tag it passes to
    > the docSelector. The attribute would indicate
whether any files had been
    > processed in the parent directory, and the
default docSelector would simply
    > skip such directories, but you could then change
that behavior.

    > or (2) The docSelector could specify an extra
attribute on its output
    > <indexFIles> tag indicating whether
sub-directories should be traversed.

    > I'm leaning toward option 2, but am open to
anybody's preferences.

    > --Martin

    > On 8/29/06, Jakob Saternus < ja...@uu...> wrote:
    >>
    >>
    >> Hello,
    >>
    >> I've found something rather confusing about the
TextIndexer.
    >>
    >> This fragment is taken from
    >> org\cdlib\xtf\textIndexer\SrcTreeProcessor.java
    >>
    >> line 520
    >>   // If we found any files to process, the
convention is that
    >> subdirectories
    >>   // contain file related to the ones we
processed, and that they
    >> shouldn't
    >>   // be processed individually.
    >>   //
    >>
    >>    if( anyProcessed )
    >>         return;
    >>
    >>    // Didn't find any files to process. Try
sub-directories.
    >>
    >> Which means that the tree walker /never/ looks
into any subdirectories of
    >> those directories that contain any files that
already have been processed.
    >> This makes it impossible to index a structure
like this:
    >>
    >> /dir1/series1.xml
    >> /dir1/series1/subseries1.xml
    >> /dir1/series1/subseries2.xml
    >>
    >> ..and it should be possible to control such
behaviour in the
    >> documentSelector.xsl:
    >>
    >> <quote from="
    >>
http://xtf.sourceforge.net/WebDocs/HTML/XTF_Programming_Guide/XTFProgGuide.html#textIndexer_DocSelector_Prog
    >> ">
    >>     It is the responsibility of the Document
Selector XSLT code to output
    >> an XML fragment that identifies which of the
files in the directory should
    >> be indexed.
    >> </quote>
    >>
    >>
    >> /Jakob

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684781&aid=1568110&group_id=119724

[Xtf-devel] [ xtf-Feature Requests-1568108 ] Support HTTP 301 redirects

[SourceForge.net <no...@so...>]

Feature Requests item #1568108, was opened at 2006-09-29 16:52
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684781&aid=1568108&group_id=119724

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: General
Group: None
Status: Open
Resolution: None
Priority: 7
Submitted By: Martin Haye (mhaye)
Assigned to: Martin Haye (mhaye)
Summary: Support HTTP 301 redirects

Initial Comment:
Redirects are useful in many circumstances. A couple
examples: (1) forking off from dynaXML to an external
service that takes care of displaying documents; (2)
detecting whether JavaScript is enabled on a user's
first access, then redirecting to the proper page to
handle it.

Currently if a servlet needs to redirect the user,
JavaScript or a <meta> tag. Both of these have the
problem that they mess up the back button -- the user
no longer has the ability to return to the page they
came from, because the redirect activates again
immediately.

HTTP supports a better alternative. If the servlet
returns a 301 HTTP status code, the browser redirects
without affecting the back button.

XTF should provide a way for stylesheets to specify
that a redirect should occur. Perhaps the best place
for this would be in the queryParser/docReqParser
stylesheets, or maybe it would be in the
resultFormatter/docFormatter.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684781&aid=1568108&group_id=119724

[Xtf-devel] [ xtf-Feature Requests-1568105 ] Freeform boolean search

[SourceForge.net <no...@so...>]

Feature Requests item #1568105, was opened at 2006-09-29 16:48
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684781&aid=1568105&group_id=119724

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: crossQuery
Group: None
Status: Open
Resolution: None
Priority: 7
Submitted By: Martin Haye (mhaye)
Assigned to: Martin Haye (mhaye)
Summary: Freeform boolean search

Initial Comment:
A few different people now have requested that XTF
support a "freeform" search, where a user could type in
a boolean query on the various indexed fields. This
would be useful in an "Advanced Search" page for users
comfortable with learning some syntax, rather than
using a simpler, less powerful, HTML form.

This could be supported by adding facilities for
parsing Common Query Language (CQL) which has most of
the features that we would want for boolean queries.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=684781&aid=1568105&group_id=119724

[Xtf-devel] Loading records into XTF

[Jeff S. <JSu...@li...>]

Hello,

Please forgive this novice question - I'm looking to load some MARCXML =
records into our test XTF installation and would like to know the best =
(and/or quickest) method for doing this.  Any help you can provide =
(documentation or code samples) will be appreciated!

Thanks in advance,

Jeff

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

Jeff Suszczynski
Digital Initiatives Unit, Rush Rhees Library
University of Rochester
585.305.5279
585.275.1060