Menu
▾
▴
archive-access-cvs — CVS commits
You can subscribe to this list here.
| 2005 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(1) |
Aug
(10) |
Sep
(36) |
Oct
(339) |
Nov
(103) |
Dec
(152) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2006 |
Jan
(141) |
Feb
(102) |
Mar
(125) |
Apr
(203) |
May
(57) |
Jun
(30) |
Jul
(139) |
Aug
(46) |
Sep
(64) |
Oct
(105) |
Nov
(34) |
Dec
(162) |
| 2007 |
Jan
(81) |
Feb
(57) |
Mar
(141) |
Apr
(72) |
May
(9) |
Jun
(1) |
Jul
(144) |
Aug
(88) |
Sep
(40) |
Oct
(43) |
Nov
(34) |
Dec
(20) |
| 2008 |
Jan
(44) |
Feb
(45) |
Mar
(16) |
Apr
(36) |
May
(8) |
Jun
(77) |
Jul
(177) |
Aug
(66) |
Sep
(8) |
Oct
(33) |
Nov
(13) |
Dec
(37) |
| 2009 |
Jan
(2) |
Feb
(5) |
Mar
(8) |
Apr
|
May
(36) |
Jun
(19) |
Jul
(46) |
Aug
(8) |
Sep
(1) |
Oct
(66) |
Nov
(61) |
Dec
(10) |
| 2010 |
Jan
(13) |
Feb
(16) |
Mar
(38) |
Apr
(76) |
May
(47) |
Jun
(32) |
Jul
(35) |
Aug
(45) |
Sep
(20) |
Oct
(61) |
Nov
(24) |
Dec
(16) |
| 2011 |
Jan
(22) |
Feb
(34) |
Mar
(11) |
Apr
(8) |
May
(24) |
Jun
(23) |
Jul
(11) |
Aug
(42) |
Sep
(81) |
Oct
(48) |
Nov
(21) |
Dec
(20) |
| 2012 |
Jan
(30) |
Feb
(25) |
Mar
(4) |
Apr
(6) |
May
(1) |
Jun
(5) |
Jul
(5) |
Aug
(8) |
Sep
(6) |
Oct
(6) |
Nov
|
Dec
|
|
From: <bra...@us...> - 2008-04-17 20:55:14
|
Revision: 2255
http://archive-access.svn.sourceforge.net/archive-access/?rev=2255&view=rev
Author: bradtofel
Date: 2008-04-17 13:54:16 -0700 (Thu, 17 Apr 2008)
Log Message:
-----------
BUGFIX: (unreported) fixed SVN links within project site.
Modified Paths:
--------------
trunk/archive-access/projects/wayback/pom.xml
Modified: trunk/archive-access/projects/wayback/pom.xml
===================================================================
--- trunk/archive-access/projects/wayback/pom.xml 2008-04-17 20:52:50 UTC (rev 2254)
+++ trunk/archive-access/projects/wayback/pom.xml 2008-04-17 20:54:16 UTC (rev 2255)
@@ -130,11 +130,11 @@
<scm>
<connection>
- scm:svn:https://archive-access.svn.sourceforge.net/svnroot/archive-access/trunk/archive-access/projects/wayback
+ scm:svn:https://archive-access.svn.sourceforge.net/svnroot/archive-access/trunk/archive-access/projects/
</connection>
<tag>HEAD</tag>
<url>
- https://archive-access.svn.sourceforge.net/svnroot/archive-access/trunk/archive-access/projects/wayback
+ https://archive-access.svn.sourceforge.net/svnroot/archive-access/trunk/archive-access/projects/
</url>
</scm>
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <bra...@us...> - 2008-04-17 20:53:19
|
Revision: 2254
http://archive-access.svn.sourceforge.net/archive-access/?rev=2254&view=rev
Author: bradtofel
Date: 2008-04-17 13:52:50 -0700 (Thu, 17 Apr 2008)
Log Message:
-----------
DOCS: explicit mention of LocalARCResourceStore => LocalResourceStore implementation class change.
Updated bug tracking URL to webteam.archive.org/jira/
Added 1.2.1 release notes.
Modified Paths:
--------------
trunk/archive-access/projects/wayback/dist/src/site/site.xml
trunk/archive-access/projects/wayback/dist/src/site/xdoc/administrator_manual.xml
trunk/archive-access/projects/wayback/dist/src/site/xdoc/release_notes.xml
Modified: trunk/archive-access/projects/wayback/dist/src/site/site.xml
===================================================================
--- trunk/archive-access/projects/wayback/dist/src/site/site.xml 2008-04-17 20:39:00 UTC (rev 2253)
+++ trunk/archive-access/projects/wayback/dist/src/site/site.xml 2008-04-17 20:52:50 UTC (rev 2254)
@@ -35,7 +35,7 @@
<item name="FAQ" href="/faq.html"/>
<item name="API" href="./apidocs"/>
<item name="Browse/Submit a Bug"
- href="http://sourceforge.net/tracker/?group_id=118427&atid=681137"/>
+ href="http://webteam.archive.org/jira/secure/IssueNavigator.jspa?component=10031"/>
</menu>
<!--Its not possible to change the labels used in reports, not yet anyways.
Modified: trunk/archive-access/projects/wayback/dist/src/site/xdoc/administrator_manual.xml
===================================================================
--- trunk/archive-access/projects/wayback/dist/src/site/xdoc/administrator_manual.xml 2008-04-17 20:39:00 UTC (rev 2253)
+++ trunk/archive-access/projects/wayback/dist/src/site/xdoc/administrator_manual.xml 2008-04-17 20:52:50 UTC (rev 2254)
@@ -230,6 +230,12 @@
</li>
</ul>
</p>
+ <p>
+ <b>Note:</b> upgrading from Wayback 1.0 to 1.2 requires changing
+ ResourceStore implementations from <b>LocalARCResourceStore</b> to
+ <b>LocalResourceStore</b>. <b>LocalARCResourceStore</b> is now
+ deprecated.
+ </p>
</subsection>
Modified: trunk/archive-access/projects/wayback/dist/src/site/xdoc/release_notes.xml
===================================================================
--- trunk/archive-access/projects/wayback/dist/src/site/xdoc/release_notes.xml 2008-04-17 20:39:00 UTC (rev 2253)
+++ trunk/archive-access/projects/wayback/dist/src/site/xdoc/release_notes.xml 2008-04-17 20:52:50 UTC (rev 2254)
@@ -11,9 +11,68 @@
<section name="Releases">
<p>
Full listing of changes and bug fixes are not currently available prior
- to release 1.2.0.
+ to release 1.2.1.
</p>
</section>
+ <section name="Release 1.2.1">
+ <subsection name="Features">
+ <ul>
+ <li>
+ Now explicitly sets the <b>charset</b> component of replayed HTML
+ page <b>Content-Type</b> HTTP headers in Archival URL mode. This
+ overrides Tomcat's default behavior of explicitly setting this value
+ to Tomcat's <b>default</b> encoding character set, if a document
+ does not set it explicitly. The original <b>Content-Type</b> HTTP
+ header value is now returned as HTTP header
+ <b>X-Wayback-Orig-Content-Type</b>.
+ </li>
+ </ul>
+ </subsection>
+ <subsection name="Bug Fixes">
+ <ul>
+ <li>
+ added getter/setter for replay image, css, javascript, and html
+ error handling .jsps
+ </li>
+ <li>
+ now returns "closest" indicator on XML query results, fixing problem
+ with WAXToolbar/Proxy mode.(<i>ACC-11</i>)
+ </li>
+ <li>
+ <b>auto-indexer</b> now closes ARC/WARC files after indexing, fixing
+ out-of-filehandle problem(<i>ACC-12</i>)
+ </li>
+ <li>
+ <b>location-client</b> now syncs .warc and .warc.gz files with
+ locationDB, in addition to .arc and .arc.gz files.(<i>ACC-13</i>)
+ </li>
+ <li>
+ fixed problem which prevented captures archived after webapp was
+ deployed from being returned. Now captures up to the current moment
+ are returned. (<i>ACC-14</i>)
+ </li>
+ <li>
+ changed all .jsp files to return UTF-8(<i>ACC-18</i>)
+ </li>
+ <li>
+ now sending correct end Date to remote NutchWAX index.
+ (<i>ACC-20</i>)
+ </li>
+ <li>
+ fixed String OOB exception when attempting to rewrite some CSS text
+ (<i>ACC-17</i>)
+ </li>
+ <li>
+ now updates CSS "import 'URL';" and 'import "URL";' content.
+ Previously only updated "import url(URL);" content.
+ </li>
+ <li>
+ fixed Replay redirect loop when using RemoteResourceIndex
+ (<i>ACC-15</i>)
+ </li>
+ </ul>
+ </subsection>
+ </section>
<section name="Release 1.2.0">
<subsection name="Features">
<ul>
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <bra...@us...> - 2008-04-17 20:40:05
|
Revision: 2253
http://archive-access.svn.sourceforge.net/archive-access/?rev=2253&view=rev
Author: bradtofel
Date: 2008-04-17 13:39:00 -0700 (Thu, 17 Apr 2008)
Log Message:
-----------
COMMENT: fixed javadoc parameter.
Modified Paths:
--------------
trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/ReplayRenderer.java
Modified: trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/ReplayRenderer.java
===================================================================
--- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/ReplayRenderer.java 2008-04-17 20:18:56 UTC (rev 2252)
+++ trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/ReplayRenderer.java 2008-04-17 20:39:00 UTC (rev 2253)
@@ -54,7 +54,9 @@
* @param resource resource to replay
* @param uriConverter the URI converter to use to translate matching
* results into replayable URLs
- * @param results
+ * @param results all SearchResults that were returned from the
+ * ResourceIndex, probably including other capture
+ * dates of the same URL.
* @throws ServletException
* @throws IOException
* @throws WaybackException
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
Revision: 2252
http://archive-access.svn.sourceforge.net/archive-access/?rev=2252&view=rev
Author: bradtofel
Date: 2008-04-17 13:18:56 -0700 (Thu, 17 Apr 2008)
Log Message:
-----------
FEATURE: now explicitly set the character encoding of returned pages to whatever encoding we produced, to override Tomcat's default.
FEATURE: now send back original HTTP Content-Type header as X-Wayback-Orig-Content-Type.
Modified Paths:
--------------
trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/archivalurl/ArchivalUrlReplayRenderer.java
Modified: trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/archivalurl/ArchivalUrlReplayRenderer.java
===================================================================
--- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/archivalurl/ArchivalUrlReplayRenderer.java 2008-04-17 20:15:29 UTC (rev 2251)
+++ trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/archivalurl/ArchivalUrlReplayRenderer.java 2008-04-17 20:18:56 UTC (rev 2252)
@@ -112,6 +112,12 @@
// set the corrected length:
int bytes = page.getBytes().length;
headers.put(HTTP_LENGTH_HEADER, String.valueOf(bytes));
+ // Tomcat will always send a charset... It's trying to be smarter than
+ // we are. If the original page didn't include a "charset" as part of
+ // the "Content-Type" HTTP header, then Tomcat will use the default..
+ // who knows what that is, or what that will do to the page..
+ // let's try explicitly setting it to what we used:
+ httpResponse.setCharacterEncoding(page.getCharSet());
// send back the headers:
HttpHeaderOperation.sendHeaders(headers, httpResponse);
@@ -142,7 +148,9 @@
String u = UrlOperations.resolveUrl(baseUrl, value);
output.put(key, uriConverter.makeReplayURI(cd,u));
-
+ } else if(keyUp.startsWith(HTTP_CONTENT_TYPE_HEADER_UP)) {
+ output.put("X-Wayback-Orig-" + key,value);
+ output.put(key,value);
} else {
// others go out as-is:
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
Revision: 2251
http://archive-access.svn.sourceforge.net/archive-access/?rev=2251&view=rev
Author: bradtofel
Date: 2008-04-17 13:15:29 -0700 (Thu, 17 Apr 2008)
Log Message:
-----------
FEATURE: grabbing original encoding from page and resetting after generating inserted .jsp results.
Modified Paths:
--------------
trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/replay/StringHttpServletResponseWrapper.java
Modified: trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/replay/StringHttpServletResponseWrapper.java
===================================================================
--- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/replay/StringHttpServletResponseWrapper.java 2008-04-17 20:13:59 UTC (rev 2250)
+++ trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/replay/StringHttpServletResponseWrapper.java 2008-04-17 20:15:29 UTC (rev 2251)
@@ -39,12 +39,14 @@
public class StringHttpServletResponseWrapper extends HttpServletResponseWrapper {
private final static String WRAPPED_CHAR_ENCODING = "UTF-8";
private StringWriter sw = new StringWriter();
+ private String origEncoding = null;
/**
* @param response
*/
public StringHttpServletResponseWrapper(HttpServletResponse response) {
super(response);
+ origEncoding = getCharacterEncoding();
setCharacterEncoding(WRAPPED_CHAR_ENCODING);
}
public PrintWriter getWriter() {
@@ -54,6 +56,7 @@
* @return
*/
public String getStringResponse() {
+ setCharacterEncoding(origEncoding);
return sw.toString();
}
}
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
Revision: 2250
http://archive-access.svn.sourceforge.net/archive-access/?rev=2250&view=rev
Author: bradtofel
Date: 2008-04-17 13:13:59 -0700 (Thu, 17 Apr 2008)
Log Message:
-----------
FEATURE: added "Content-Type" static member.
Modified Paths:
--------------
trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/replay/HttpHeaderProcessor.java
Modified: trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/replay/HttpHeaderProcessor.java
===================================================================
--- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/replay/HttpHeaderProcessor.java 2008-04-16 06:57:20 UTC (rev 2249)
+++ trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/replay/HttpHeaderProcessor.java 2008-04-17 20:13:59 UTC (rev 2250)
@@ -48,6 +48,10 @@
public final static String HTTP_CONTENT_BASE_HEADER = "Content-Base";
public final static String HTTP_CONTENT_BASE_HEADER_UP =
HTTP_CONTENT_BASE_HEADER.toUpperCase();
+
+ public final static String HTTP_CONTENT_TYPE_HEADER = "Content-Type";
+ public final static String HTTP_CONTENT_TYPE_HEADER_UP =
+ HTTP_CONTENT_TYPE_HEADER.toUpperCase();
/**
* optionally add header key:value to output for later returning to client
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <al...@us...> - 2008-04-16 06:58:10
|
Revision: 2249
http://archive-access.svn.sourceforge.net/archive-access/?rev=2249&view=rev
Author: alexoz
Date: 2008-04-15 23:57:20 -0700 (Tue, 15 Apr 2008)
Log Message:
-----------
Added glossary link to the acronym "surt".
Modified Paths:
--------------
trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/list_rules.jsp
Modified: trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/list_rules.jsp
===================================================================
--- trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/list_rules.jsp 2008-04-16 05:47:16 UTC (rev 2248)
+++ trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/list_rules.jsp 2008-04-16 06:57:20 UTC (rev 2249)
@@ -1,7 +1,7 @@
<%@ include file="header.inc"%>
<form action="<c:url value="/admin"/>" method="GET" id="navForm">
-<label for="surtNavBox">SURT or URL:</label> <input size="50"
+<label for="surtNavBox"><a href="http://crawler.archive.org/articles/user_manual/glossary.html#surt"><acronym title="Sort-friendly URI Reordering Transform">SURT</acronym> or URL:</label> <input size="50"
name="surt" value="<c:out value="${surt}"/>" id="surtNavBox" /> <input
type="submit" value="Go!" /></form>
@@ -124,4 +124,4 @@
</tr>
</tbody>
</table>
-<%@ include file="footer.inc"%>
\ No newline at end of file
+<%@ include file="footer.inc"%>
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <al...@us...> - 2008-04-16 05:48:07
|
Revision: 2248
http://archive-access.svn.sourceforge.net/archive-access/?rev=2248&view=rev
Author: alexoz
Date: 2008-04-15 22:47:16 -0700 (Tue, 15 Apr 2008)
Log Message:
-----------
* AdminController.java, list_rules.jsp
Added form fields for public and private comments.
Modified Paths:
--------------
trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/webui/AdminController.java
trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/list_rules.jsp
Modified: trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/webui/AdminController.java
===================================================================
--- trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/webui/AdminController.java 2008-04-16 00:58:21 UTC (rev 2247)
+++ trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/webui/AdminController.java 2008-04-16 05:47:16 UTC (rev 2248)
@@ -195,6 +195,8 @@
rule.setRetrievalStart(parseDate(request.getParameter("retrievalStart")));
rule.setRetrievalEnd(parseDate(request.getParameter("retrievalEnd")));
rule.setSecondsSinceCapture(parseInteger(request.getParameter("secondsSinceCapture")));
+ rule.setPrivateComment(request.getParameter("privateComment"));
+ rule.setPublicComment(request.getParameter("publicComment"));
ruleDao.saveRule(rule);
return redirectToSurt(request, response, surt);
Modified: trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/list_rules.jsp
===================================================================
--- trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/list_rules.jsp 2008-04-16 00:58:21 UTC (rev 2247)
+++ trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/list_rules.jsp 2008-04-16 05:47:16 UTC (rev 2248)
@@ -68,6 +68,12 @@
<p><label for="policy">Policy:</label> <input name="policy"
id="policy" value="<c:out value="${rule.rule.policy}"/>" /></p>
+ <p><label for="privateComment">Private comment:</label><br />
+ <textarea name="privateComment" rows="4" cols="50"><c:out value="${rule.rule.privateComment}" /></textarea>
+ </p>
+ <p><label for="publicComment">Public comment:</label><br />
+ <textarea name="publicComment" rows="4" cols="50"><c:out value="${rule.rule.publicComment}" /></textarea>
+ </p>
<div class="priButtons">
<input type="submit" value="Save" name="saveRule" />
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <bra...@us...> - 2008-04-16 00:58:16
|
Revision: 2247
http://archive-access.svn.sourceforge.net/archive-access/?rev=2247&view=rev
Author: bradtofel
Date: 2008-04-15 17:58:21 -0700 (Tue, 15 Apr 2008)
Log Message:
-----------
Wayback 1.2.1 maintenance release
Added Paths:
-----------
branches/wayback-1_2_1/wayback/
Copied: branches/wayback-1_2_1/wayback (from rev 2246, trunk/archive-access/projects/wayback)
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <bra...@us...> - 2008-04-16 00:56:50
|
Revision: 2246
http://archive-access.svn.sourceforge.net/archive-access/?rev=2246&view=rev
Author: bradtofel
Date: 2008-04-15 17:56:56 -0700 (Tue, 15 Apr 2008)
Log Message:
-----------
Added Paths:
-----------
branches/wayback-1_2_1/
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <bra...@us...> - 2008-04-16 00:41:05
|
Revision: 2245
http://archive-access.svn.sourceforge.net/archive-access/?rev=2245&view=rev
Author: bradtofel
Date: 2008-04-15 17:41:09 -0700 (Tue, 15 Apr 2008)
Log Message:
-----------
RELEASE: 1.2.1 pom.xml updates.
Modified Paths:
--------------
trunk/archive-access/projects/wayback/dist/pom.xml
trunk/archive-access/projects/wayback/pom.xml
trunk/archive-access/projects/wayback/wayback-core/pom.xml
trunk/archive-access/projects/wayback/wayback-mapreduce/pom.xml
trunk/archive-access/projects/wayback/wayback-mapreduce-prereq/pom.xml
trunk/archive-access/projects/wayback/wayback-webapp/pom.xml
Modified: trunk/archive-access/projects/wayback/dist/pom.xml
===================================================================
--- trunk/archive-access/projects/wayback/dist/pom.xml 2008-04-16 00:40:18 UTC (rev 2244)
+++ trunk/archive-access/projects/wayback/dist/pom.xml 2008-04-16 00:41:09 UTC (rev 2245)
@@ -3,7 +3,7 @@
<parent>
<groupId>org.archive</groupId>
<artifactId>wayback</artifactId>
- <version>1.3.0-SNAPSHOT</version>
+ <version>1.2.1</version>
</parent>
<modelVersion>4.0.0</modelVersion>
@@ -54,13 +54,13 @@
<dependency>
<groupId>org.archive.wayback</groupId>
<artifactId>wayback-webapp</artifactId>
- <version>1.3.0-SNAPSHOT</version>
+ <version>1.2.1</version>
<type>war</type>
</dependency>
<dependency>
<groupId>org.archive.wayback</groupId>
<artifactId>wayback-mapreduce</artifactId>
- <version>1.3.0-SNAPSHOT</version>
+ <version>1.2.1</version>
</dependency>
</dependencies>
Modified: trunk/archive-access/projects/wayback/pom.xml
===================================================================
--- trunk/archive-access/projects/wayback/pom.xml 2008-04-16 00:40:18 UTC (rev 2244)
+++ trunk/archive-access/projects/wayback/pom.xml 2008-04-16 00:41:09 UTC (rev 2245)
@@ -16,7 +16,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>org.archive</groupId>
<artifactId>wayback</artifactId>
- <version>1.3.0-SNAPSHOT</version>
+ <version>1.2.1</version>
<packaging>pom</packaging>
<name>Wayback</name>
Modified: trunk/archive-access/projects/wayback/wayback-core/pom.xml
===================================================================
--- trunk/archive-access/projects/wayback/wayback-core/pom.xml 2008-04-16 00:40:18 UTC (rev 2244)
+++ trunk/archive-access/projects/wayback/wayback-core/pom.xml 2008-04-16 00:41:09 UTC (rev 2245)
@@ -17,7 +17,7 @@
<parent>
<groupId>org.archive</groupId>
<artifactId>wayback</artifactId>
- <version>1.3.0-SNAPSHOT</version>
+ <version>1.2.1</version>
</parent>
<groupId>org.archive.wayback</groupId>
<artifactId>wayback-core</artifactId>
Modified: trunk/archive-access/projects/wayback/wayback-mapreduce/pom.xml
===================================================================
--- trunk/archive-access/projects/wayback/wayback-mapreduce/pom.xml 2008-04-16 00:40:18 UTC (rev 2244)
+++ trunk/archive-access/projects/wayback/wayback-mapreduce/pom.xml 2008-04-16 00:41:09 UTC (rev 2245)
@@ -12,7 +12,7 @@
<parent>
<groupId>org.archive</groupId>
<artifactId>wayback</artifactId>
- <version>1.3.0-SNAPSHOT</version>
+ <version>1.2.1</version>
</parent>
<groupId>org.archive.wayback</groupId>
<artifactId>wayback-mapreduce</artifactId>
Modified: trunk/archive-access/projects/wayback/wayback-mapreduce-prereq/pom.xml
===================================================================
--- trunk/archive-access/projects/wayback/wayback-mapreduce-prereq/pom.xml 2008-04-16 00:40:18 UTC (rev 2244)
+++ trunk/archive-access/projects/wayback/wayback-mapreduce-prereq/pom.xml 2008-04-16 00:41:09 UTC (rev 2245)
@@ -10,7 +10,7 @@
<parent>
<groupId>org.archive</groupId>
<artifactId>wayback</artifactId>
- <version>1.3.0-SNAPSHOT</version>
+ <version>1.2.1</version>
</parent>
<groupId>org.archive.wayback</groupId>
<artifactId>wayback-mapreduce-prereq</artifactId>
Modified: trunk/archive-access/projects/wayback/wayback-webapp/pom.xml
===================================================================
--- trunk/archive-access/projects/wayback/wayback-webapp/pom.xml 2008-04-16 00:40:18 UTC (rev 2244)
+++ trunk/archive-access/projects/wayback/wayback-webapp/pom.xml 2008-04-16 00:41:09 UTC (rev 2245)
@@ -3,7 +3,7 @@
<parent>
<artifactId>wayback</artifactId>
<groupId>org.archive</groupId>
- <version>1.3.0-SNAPSHOT</version>
+ <version>1.2.1</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<groupId>org.archive.wayback</groupId>
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
Revision: 2244
http://archive-access.svn.sourceforge.net/archive-access/?rev=2244&view=rev
Author: bradtofel
Date: 2008-04-15 17:40:18 -0700 (Tue, 15 Apr 2008)
Log Message:
-----------
BUGFIX: ACC-15: RemoteResourceIndex was not using SelfRedirectFilter for replay requests, which caused redirect loops. Patch submitted by Alex Osborne.
Modified Paths:
--------------
trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/RemoteResourceIndex.java
Modified: trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/RemoteResourceIndex.java
===================================================================
--- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/RemoteResourceIndex.java 2008-04-16 00:37:46 UTC (rev 2243)
+++ trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/RemoteResourceIndex.java 2008-04-16 00:40:18 UTC (rev 2244)
@@ -33,6 +33,7 @@
import javax.xml.parsers.ParserConfigurationException;
import org.archive.wayback.ResourceIndex;
+import org.archive.wayback.UrlCanonicalizer;
import org.archive.wayback.WaybackConstants;
import org.archive.wayback.core.CaptureSearchResults;
import org.archive.wayback.core.SearchResult;
@@ -44,6 +45,10 @@
import org.archive.wayback.exception.ConfigurationException;
import org.archive.wayback.exception.ResourceIndexNotAvailableException;
import org.archive.wayback.exception.ResourceNotInArchiveException;
+import org.archive.wayback.resourceindex.filters.SelfRedirectFilter;
+import org.archive.wayback.util.ObjectFilter;
+import org.archive.wayback.util.ObjectFilterChain;
+import org.archive.wayback.util.url.AggressiveUrlCanonicalizer;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.Node;
@@ -75,6 +80,7 @@
private static final String WB_XML_ERROR_TAGNAME = "error";
private static final String WB_XML_ERROR_TITLE = "title";
private static final String WB_XML_ERROR_MESSAGE = "message";
+ private UrlCanonicalizer canonicalizer = new AggressiveUrlCanonicalizer();
@SuppressWarnings("unchecked")
private final ThreadLocal tl = new ThreadLocal() {
@@ -117,13 +123,15 @@
ResourceNotInArchiveException, BadQueryException,
AccessControlException {
- return urlToSearchResults(getRequestUrl(wbRequest));
+ return urlToSearchResults(getRequestUrl(wbRequest),
+ getSearchResultFilters(wbRequest));
}
- protected SearchResults urlToSearchResults(String requestUrl)
- throws ResourceIndexNotAvailableException,
- ResourceNotInArchiveException, BadQueryException,
- AccessControlException {
+ protected SearchResults urlToSearchResults(String requestUrl,
+ ObjectFilter<SearchResult> filter)
+ throws ResourceIndexNotAvailableException,
+ ResourceNotInArchiveException, BadQueryException,
+ AccessControlException {
Document document = null;
try {
@@ -141,7 +149,7 @@
}
checkDocumentForExceptions(document);
- return documentToSearchResults(document);
+ return documentToSearchResults(document, filter);
}
protected void checkDocumentForExceptions(Document document)
@@ -182,7 +190,27 @@
}
}
- protected SearchResults documentToSearchResults(Document document) {
+ protected ObjectFilter<SearchResult> getSearchResultFilters(
+ WaybackRequest wbRequest) {
+ String searchType = wbRequest.get(WaybackConstants.REQUEST_TYPE);
+ ObjectFilterChain<SearchResult> filters =
+ new ObjectFilterChain<SearchResult>();
+
+ if (searchType.equals(WaybackConstants.REQUEST_REPLAY_QUERY)
+ || searchType.equals(WaybackConstants.REQUEST_CLOSEST_QUERY)) {
+
+ SelfRedirectFilter selfRedirectFilter = new SelfRedirectFilter();
+ selfRedirectFilter.setCanonicalizer(canonicalizer);
+ filters.addFilter(selfRedirectFilter);
+ } else {
+ // no filters for now
+ filters = null;
+ }
+ return filters;
+ }
+
+ protected SearchResults documentToSearchResults(Document document,
+ ObjectFilter<SearchResult> filter) {
SearchResults results = null;
NodeList filters = getRequestFilters(document);
String resultsType = getResultsType(document);
@@ -203,7 +231,17 @@
for(int i = 0; i < xresults.getLength(); i++) {
Node xresult = xresults.item(i);
SearchResult result = searchElementToSearchResult(xresult);
- results.addSearchResult(result,true);
+
+ int ruling = ObjectFilter.FILTER_INCLUDE;
+ if (filter != null) {
+ ruling = filter.filterObject(result);
+ }
+
+ if (ruling == ObjectFilter.FILTER_ABORT) {
+ break;
+ } else if (ruling == ObjectFilter.FILTER_INCLUDE) {
+ results.addSearchResult(result, true);
+ }
}
return results;
}
@@ -291,4 +329,12 @@
public void shutdown() throws IOException {
// No-op
}
+
+ public UrlCanonicalizer getCanonicalizer() {
+ return canonicalizer;
+ }
+
+ public void setCanonicalizer(UrlCanonicalizer canonicalizer) {
+ this.canonicalizer = canonicalizer;
+ }
}
\ No newline at end of file
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <bra...@us...> - 2008-04-16 00:37:41
|
Revision: 2243
http://archive-access.svn.sourceforge.net/archive-access/?rev=2243&view=rev
Author: bradtofel
Date: 2008-04-15 17:37:46 -0700 (Tue, 15 Apr 2008)
Log Message:
-----------
TWEAK: Added oracle exclusion example, added NutchWAX example.
Modified Paths:
--------------
trunk/archive-access/projects/wayback/wayback-webapp/src/main/webapp/WEB-INF/wayback.xml
Modified: trunk/archive-access/projects/wayback/wayback-webapp/src/main/webapp/WEB-INF/wayback.xml
===================================================================
--- trunk/archive-access/projects/wayback/wayback-webapp/src/main/webapp/WEB-INF/wayback.xml 2008-04-16 00:36:51 UTC (rev 2242)
+++ trunk/archive-access/projects/wayback/wayback-webapp/src/main/webapp/WEB-INF/wayback.xml 2008-04-16 00:37:46 UTC (rev 2243)
@@ -67,7 +67,19 @@
<property name="webCache" ref="livewebcache" />
</bean>
-->
+<!--
+ The following bean is an example using the Access Control Oracle, thanks Alex Osborne.
+ Currently this is pretty undocumented, but here is a place to get started:
+ http://webteam.archive.org/confluence/display/wayback/Exclusions+API
+-->
+<!--
+ <bean id="excluder-factory-oracle" class="org.archive.wayback.accesscontrol.oracleclient.OracleExclusionFilterFactory">
+ <property name="oracleUrl" value="http://192.168.1.11:8080/oracle-0.0.1-SNAPSHOT/" />
+ <property name="accessGroup" value="ia_archiver" />
+ </bean>
+-->
+
<bean id="localbdbcollection" class="org.archive.wayback.webapp.WaybackCollection">
<property name="resourceStore">
@@ -164,10 +176,37 @@
<property name="searchUrlBase" value="http://indexhost:8080/index/xmlquery" />
</bean>
</property>
+
+
</bean>
-->
<!--
+ The following WaybackCollection bean template is an example for using a NutchWAX
+ full-text index with Wayback, using a RemoteResourceStore to access
+ replayed documents. You will need to change searchUrlBase to your local NutchWAX
+ installation. You may also need to ensure that the maxRecords on your RequestParser is
+ not greater than the maxRecords configured on the RemoteNutchResourceIndex.
+-->
+<!--
+ <bean id="remotenutchcollection" class="org.archive.wayback.webapp.WaybackCollection">
+
+ <property name="resourceStore">
+ <bean class="org.archive.wayback.resourcestore.Http11ResourceStore">
+ <property name="urlPrefix" value="http://localhost:8080/arcproxy/" />
+ </bean>
+ </property>
+
+ <property name="resourceIndex">
+ <bean class="org.archive.wayback.resourceindex.NutchResourceIndex" init-method="init">
+ <property name="searchUrlBase" value="http://webteam-ws.us.archive.org:8080/katrina/opensearch" />
+ <property name="maxRecords" value="100" />
+ </bean>
+ </property>
+ </bean>
+-->
+
+<!--
This is the only AccessPoint defined by default within this wayback.xml
Spring configuration file, providing an ArchivalURL Replay UI to the
"localbdbcollection" by providing ArchivalURL-specific implementations
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <bra...@us...> - 2008-04-16 00:36:45
|
Revision: 2242
http://archive-access.svn.sourceforge.net/archive-access/?rev=2242&view=rev
Author: bradtofel
Date: 2008-04-15 17:36:51 -0700 (Tue, 15 Apr 2008)
Log Message:
-----------
TWEAK: Commented out authorization example. quiets down tomcat's logs a little.
Modified Paths:
--------------
trunk/archive-access/projects/wayback/wayback-webapp/src/main/webapp/WEB-INF/web.xml
Modified: trunk/archive-access/projects/wayback/wayback-webapp/src/main/webapp/WEB-INF/web.xml
===================================================================
--- trunk/archive-access/projects/wayback/wayback-webapp/src/main/webapp/WEB-INF/web.xml 2008-04-15 23:58:06 UTC (rev 2241)
+++ trunk/archive-access/projects/wayback/wayback-webapp/src/main/webapp/WEB-INF/web.xml 2008-04-16 00:36:51 UTC (rev 2242)
@@ -5,7 +5,7 @@
xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee
http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd"
version="2.4">
-
+<!--
<security-constraint>
<web-resource-collection>
<web-resource-name>Secured-Wayback</web-resource-name>
@@ -20,7 +20,7 @@
<auth-method>BASIC</auth-method>
<realm-name>Secured-Wayback</realm-name>
</login-config>
-
+-->
<!-- General Installation information
-->
<context-param>
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <bra...@us...> - 2008-04-16 00:01:58
|
Revision: 2241
http://archive-access.svn.sourceforge.net/archive-access/?rev=2241&view=rev
Author: bradtofel
Date: 2008-04-15 16:58:06 -0700 (Tue, 15 Apr 2008)
Log Message:
-----------
TWEAK: made format more palletable to the validators, but this file is not intended for actual production use, just as a set of configuration examples. Also updated ResourceStore implementation classnames, hopefully to clear up any more confusion with the old classnames.
Modified Paths:
--------------
trunk/archive-access/projects/wayback/wayback-webapp/src/main/webapp/WEB-INF/wayback-templates.xml
Modified: trunk/archive-access/projects/wayback/wayback-webapp/src/main/webapp/WEB-INF/wayback-templates.xml
===================================================================
--- trunk/archive-access/projects/wayback/wayback-webapp/src/main/webapp/WEB-INF/wayback-templates.xml 2008-04-15 00:03:57 UTC (rev 2240)
+++ trunk/archive-access/projects/wayback/wayback-webapp/src/main/webapp/WEB-INF/wayback-templates.xml 2008-04-15 23:58:06 UTC (rev 2241)
@@ -1,3 +1,6 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE beans PUBLIC "-//SPRING//DTD BEAN//EN" "http://www.springframework.org/dtd/spring-beans.dtd">
+<beans>
<!-- SearchResultSource templates -->
<bean id="bdbsearchresultsource"
@@ -35,20 +38,18 @@
<property name="maxRecords" value="10000" />
</bean>
- <property name="remotenutchindex">
- <bean class="org.archive.wayback.resourceindex.NutchResourceIndex" init-method="init">
- <property name="searchUrlBase" value="http://webteam-ws.us.archive.org:8080/katrina/opensearch" />
- <property name="maxRecords" value="100" />
- </bean>
- </property>
+ <bean id="remotenutchindex" class="org.archive.wayback.resourceindex.NutchResourceIndex" init-method="init">
+ <property name="searchUrlBase" value="http://webteam-ws.us.archive.org:8080/katrina/opensearch" />
+ <property name="maxRecords" value="100" />
+ </bean>
<!-- ResourceStore templates -->
- <bean id="localstore" class="org.archive.wayback.resourcestore.LocalARCResourceStore">
- <property name="arcDir" value="/tmp/wayback/arcs/" />
+ <bean id="localstore" class="org.archive.wayback.resourcestore.LocalesourceStore">
+ <property name="dataDir" value="/tmp/wayback/arcs/" />
</bean>
- <bean id="remotestore" class="org.archive.wayback.resourcestore.HttpARCResourceStore">
+ <bean id="remotestore" class="org.archive.wayback.resourcestore.Http11ResourceStore">
<property name="urlPrefix" value="http://localhost:8080/arcproxy/" />
</bean>
@@ -140,4 +141,4 @@
</bean>
</property>
</bean>
-
\ No newline at end of file
+</beans>
\ No newline at end of file
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
Revision: 2240
http://archive-access.svn.sourceforge.net/archive-access/?rev=2240&view=rev
Author: bradtofel
Date: 2008-04-14 17:03:57 -0700 (Mon, 14 Apr 2008)
Log Message:
-----------
removed deprecated class.
Removed Paths:
-------------
trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourcestore/LocalARCResourceStore.java
Deleted: trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourcestore/LocalARCResourceStore.java
===================================================================
--- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourcestore/LocalARCResourceStore.java 2008-04-15 00:01:01 UTC (rev 2239)
+++ trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourcestore/LocalARCResourceStore.java 2008-04-15 00:03:57 UTC (rev 2240)
@@ -1,370 +0,0 @@
-/* LocalARCResourceStore
- *
- * $Id$
- *
- * Created on 5:14:45 PM Oct 12, 2006.
- *
- * Copyright (C) 2006 Internet Archive.
- *
- * This file is part of Wayback.
- *
- * Wayback is free software; you can redistribute it and/or modify
- * it under the terms of the GNU Lesser Public License as published by
- * the Free Software Foundation; either version 2.1 of the License, or
- * any later version.
- *
- * Wayback is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU Lesser Public License for more details.
- *
- * You should have received a copy of the GNU Lesser Public License
- * along with Wayback; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- */
-package org.archive.wayback.resourcestore;
-
-import java.io.File;
-import java.io.IOException;
-import java.net.MalformedURLException;
-import java.util.HashMap;
-import java.util.Iterator;
-import java.util.logging.Logger;
-
-//import org.apache.commons.httpclient.HttpException;
-import org.archive.io.ArchiveRecord;
-import org.archive.io.arc.ARCReader;
-import org.archive.io.arc.ARCReaderFactory;
-import org.archive.io.arc.ARCRecord;
-import org.archive.wayback.ResourceStore;
-import org.archive.wayback.WaybackConstants;
-import org.archive.wayback.core.Resource;
-import org.archive.wayback.core.SearchResult;
-//import org.archive.wayback.core.SearchResults;
-import org.archive.wayback.exception.ConfigurationException;
-import org.archive.wayback.exception.ResourceNotAvailableException;
-import org.archive.wayback.resourceindex.indexer.IndexClient;
-//import org.archive.wayback.util.CloseableIterator;
-import org.archive.wayback.util.DirMaker;
-
-/**
- * Implements ResourceStore using a local directory of ARC files.
- *
- * @author brad
- * @version $Date$, $Revision$
- */
-public class LocalARCResourceStore implements ResourceStore {
- private static final Logger LOGGER =
- Logger.getLogger(LocalARCResourceStore.class.getName());
-
- private final static int DEFAULT_RUN_INTERVAL_MS = 10000;
-
- private File arcDir = null;
- private File workDir = null;
- private File queuedDir = null;
- private int runInterval = DEFAULT_RUN_INTERVAL_MS;
- private IndexClient indexClient = null;
- private ArcIndexer indexer = new ArcIndexer();
-
- /**
- * Thread object of update thread -- also is flag indicating if the thread
- * has already been started -- static, and access to it is synchronized.
- */
- private Thread indexThread = null;
-
- /**
- * @throws ConfigurationException
- */
- public void init() throws ConfigurationException {
- if(arcDir == null) {
- throw new ConfigurationException("No arcDir set");
- }
- if(indexClient != null) {
- startAutoIndexThread();
- }
- }
-
- public Resource retrieveResource(SearchResult result) throws IOException,
- ResourceNotAvailableException {
-
- String arcName = resultToARCName(result);
- long offset = resultToARCOffset(result);
- if (!arcName.endsWith(ARCReader.DOT_COMPRESSED_ARC_FILE_EXTENSION)) {
- arcName += ARCReader.DOT_COMPRESSED_ARC_FILE_EXTENSION;
- }
- File arcFile = new File(arcName);
- if (!arcFile.isAbsolute()) {
- arcFile = new File(arcDir, arcName);
- }
- if (!arcFile.exists() || !arcFile.canRead()) {
-
- // TODO: this needs to be prettied up for end user consumption..
- throw new ResourceNotAvailableException("Cannot find ARC file ("
- + arcFile.getAbsolutePath() + ")");
- } else {
-
- ARCReader reader = ARCReaderFactory.get(arcFile);
-
- ArchiveRecord rec = reader.get(offset);
- // TODO: handle other types of ArchiveRecords...
- if(!(rec instanceof ARCRecord)) {
- throw new ResourceNotAvailableException("Bad ARCRecord format");
- }
- Resource r = new ArcResource((ARCRecord) rec, reader);
- return r;
- }
- }
-
- protected String resultToARCName(SearchResult result) {
- return result.get(WaybackConstants.RESULT_ARC_FILE);
- }
-
- protected long resultToARCOffset(SearchResult result) {
- return Long.parseLong(result.get(WaybackConstants.RESULT_OFFSET));
- }
-
- /**
- * Find any new ARC files and queue them for indexing.
- * @throws IOException
- */
- public void queueNewArcsForIndex() throws IOException {
-
- // build a HashMap of what has been queued already:
- HashMap<String,String> queued = new HashMap<String, String>();
- String entries[] = queuedDir.list();
- if(entries != null) {
- for (int i = 0; i < entries.length; i++) {
- queued.put(entries[i], "i");
- }
- }
- // now scan thru arcDir, and make a flag file for anything that was not
- // already there:
- String arcs[] = arcDir.list();
- if (arcs != null) {
- for (int i = 0; i < arcs.length; i++) {
- if (arcs[i].endsWith(".arc.gz") &&
- !queued.containsKey(arcs[i])) {
-
- File arc = new File(arcDir,arcs[i]);
- if(arc.isFile()) {
-
- File newQueuedFile = new File(queuedDir,arcs[i]);
- File newToBeIndexedFile = new File(workDir,arcs[i]);
- newToBeIndexedFile.createNewFile();
- newQueuedFile.createNewFile();
- }
- }
- }
- }
- }
-
- /**
- * Index up to 'max' ARC files queued for indexing, queueing the resulting
- * CDX files for merging with the BDBIndex.
- *
- * @param indexer
- * @param max maximum number to index in this method call, 0 for unlimited
- * @return int number of ARC files indexed
- * @throws MalformedURLException
- * @throws IOException
- */
- public int indexArcs(ArcIndexer indexer, int max)
- throws MalformedURLException, IOException {
-
- int numIndexed = 0;
- String toBeIndexed[] = workDir.list();
-
- if (toBeIndexed != null) {
- for (int i = 0; i < toBeIndexed.length; i++) {
- String base = toBeIndexed[i];
- File arcFile = new File(arcDir,base);
- File workFlagFile = new File(workDir,base);
- String cdxBase;
- if(base.endsWith(".arc.gz")) {
- cdxBase = base.substring(0,base.length() - 7);
- } else if(base.endsWith(".arc")) {
- cdxBase = base.substring(0,base.length() - 4);
- } else {
- cdxBase = base;
- }
-
- try {
-
- LOGGER.info("Indexing ARC " + arcFile.getAbsolutePath());
- Iterator<SearchResult> itr = indexer.iterator(arcFile);
-
- if(indexClient.addSearchResults(cdxBase, itr)) {
- if (!workFlagFile.delete()) {
- throw new IOException("Unable to delete "
- + workFlagFile.getAbsolutePath());
- }
- }
- numIndexed++;
- } catch (IOException e) {
- LOGGER.severe("FAILED index of " + arcFile.getAbsolutePath() +
- " cause: " + e.getLocalizedMessage());
- }
- if(max > 0 && (numIndexed >= max)) {
- break;
- }
- }
- }
- return numIndexed;
- }
-
- /**
- * Scan for new ARC files, and index any new files discovered.
- *
- * There are 3 main steps, which could be broken into separate threads:
- * 1) detect new ARCs
- * 2) create CDX files for each new ARC
- * 3) upload CDX files to target (or rename to local "incoming" directory)
- *
- * for now these are sequential.
- *
- * @return number of ARC files indexed
- */
- public int indexNewArcs() {
- int numIndexed = 0;
- try {
- queueNewArcsForIndex();
- } catch (IOException e) {
- e.printStackTrace();
- }
- try {
- numIndexed = indexArcs(indexer,10);
- } catch (MalformedURLException e) {
- // TODO Auto-generated catch block
- e.printStackTrace();
- } catch (IOException e) {
- // TODO Auto-generated catch block
- e.printStackTrace();
- }
- return numIndexed;
- }
-
- /**
- * start the AutoARCIndexThread thread, which will scan for new arcs, index
- * new arcs that appear, and merge indexed arcs (in CDX format) into the
- * BDBResourceIndex
- */
- private synchronized void startAutoIndexThread() {
- if (indexThread != null) {
- return;
- }
- indexThread = new AutoARCIndexThread(this,runInterval);
- indexThread.start();
- }
-
- /**
- * Thread that repeatedly calls indexNewArcs on the LocalARCResourceStore.
- */
- private class AutoARCIndexThread extends Thread {
-
- private LocalARCResourceStore store = null;
-
- private int runInterval;
-
- /**
- * @param store
- * @param runInterval
- */
- public AutoARCIndexThread(LocalARCResourceStore store,
- int runInterval) {
- super("AutoARCIndexThread");
- super.setDaemon(true);
- this.store = store;
- this.runInterval = runInterval;
- LOGGER.info("AutoARCIndexThread is alive.");
- }
-
- public void run() {
- int sleepInterval = runInterval;
- while (true) {
- try {
- int numIndexed = store.indexNewArcs();
- if (numIndexed == 0) {
- sleep(sleepInterval);
- sleepInterval += runInterval;
- } else {
- sleepInterval = runInterval;
- }
- } catch (InterruptedException e) {
- e.printStackTrace();
- }
- }
- }
- }
-
- /**
- * @return String path to workDir
- */
- public String getWorkDir() {
- return DirMaker.getAbsolutePath(workDir);
- }
- /**
- * @param workDir the workDir to set
- * @throws IOException
- */
- public void setWorkDir(String workDir) throws IOException {
- this.workDir = DirMaker.ensureDir(workDir);
- }
-
- /**
- * @return String path to queuedDir
- */
- public String getQueuedDir() {
- return DirMaker.getAbsolutePath(queuedDir);
- }
- /**
- * @param queuedDir the queuedDir to set
- * @throws IOException
- */
- public void setQueuedDir(String queuedDir) throws IOException {
- this.queuedDir = DirMaker.ensureDir(queuedDir);
- }
-
- /**
- * @return integer milliseconds between polls for new ARC content.
- */
- public int getRunInterval() {
- return runInterval;
- }
- /**
- * @param runInterval the runInterval to set
- */
- public void setRunInterval(int runInterval) {
- this.runInterval = runInterval;
- }
- /**
- * @return the arcDir
- */
- public String getArcDir() {
- return DirMaker.getAbsolutePath(arcDir);
- }
- /**
- * @param arcDir the arcDir to set
- * @throws IOException
- */
- public void setArcDir(String arcDir) throws IOException {
- this.arcDir = DirMaker.ensureDir(arcDir);
- }
-
- /**
- * @return the indexClient
- */
- public IndexClient getIndexClient() {
- return indexClient;
- }
-
- /**
- * @param indexClient the indexClient to set
- */
- public void setIndexClient(IndexClient indexClient) {
- this.indexClient = indexClient;
- }
-
- public void shutdown() throws IOException {
- // no-op... could shutdown threads...
- }
-}
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <bra...@us...> - 2008-04-15 00:01:03
|
Revision: 2239
http://archive-access.svn.sourceforge.net/archive-access/?rev=2239&view=rev
Author: bradtofel
Date: 2008-04-14 17:01:01 -0700 (Mon, 14 Apr 2008)
Log Message:
-----------
BUGFIX: was not updating
import 'URL';
import "URL";
correctly
Modified Paths:
--------------
trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/replay/TagMagix.java
trunk/archive-access/projects/wayback/wayback-core/src/test/java/org/archive/wayback/replay/TagMagixTest.java
Modified: trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/replay/TagMagix.java
===================================================================
--- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/replay/TagMagix.java 2008-04-11 04:25:37 UTC (rev 2238)
+++ trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/replay/TagMagix.java 2008-04-15 00:01:01 UTC (rev 2239)
@@ -75,12 +75,18 @@
// "url\\s*\\(\\s*(['\"]?.+?['\"]?)\\s*\\)";
private static String cssUrlPatString =
"url\\s*\\(\\s*([\\\\\"']*.+?[\\\\\"']*)\\s*\\)";
-
+
private static String cssImportPatString =
"@import\\s+" + cssUrlPatString;
+ private static String cssImportNoUrlPatString =
+ "@import\\s+([\"'].+?[\"'])";
+
private static Pattern cssImportPattern =
Pattern.compile(cssImportPatString);
+
+ private static Pattern cssImportNoUrlPattern =
+ Pattern.compile(cssImportNoUrlPatString);
private static Pattern cssUrlPattern = Pattern.compile(cssUrlPatString);
@@ -156,6 +162,7 @@
ResultURIConverter uriConverter, String captureDate,
String baseUrl) {
markupTagREURIC(page,uriConverter,captureDate,baseUrl,cssImportPattern);
+ markupTagREURIC(page,uriConverter,captureDate,baseUrl,cssImportNoUrlPattern);
}
public static void markupStyleUrls(StringBuilder page,
Modified: trunk/archive-access/projects/wayback/wayback-core/src/test/java/org/archive/wayback/replay/TagMagixTest.java
===================================================================
--- trunk/archive-access/projects/wayback/wayback-core/src/test/java/org/archive/wayback/replay/TagMagixTest.java 2008-04-11 04:25:37 UTC (rev 2238)
+++ trunk/archive-access/projects/wayback/wayback-core/src/test/java/org/archive/wayback/replay/TagMagixTest.java 2008-04-15 00:01:01 UTC (rev 2239)
@@ -327,8 +327,13 @@
"@import url(\r\n\"http://web.archive.org/wayback/2004/http://foo.com/f.css\"\n\r);",
"http://web.archive.org/wayback/","2004","http://foo.com/");
+ checkCSSMarkup("@import \"http://foo.com/f.css\";",
+ "@import \"http://web.archive.org/wayback/2004/http://foo.com/f.css\";",
+ "http://web.archive.org/wayback/","2004","http://foo.com/");
+ checkCSSMarkup("@import 'http://foo.com/f.css';",
+ "@import 'http://web.archive.org/wayback/2004/http://foo.com/f.css';",
+ "http://web.archive.org/wayback/","2004","http://foo.com/");
-
}
public void testStyleUrlMarkup() {
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <bra...@us...> - 2008-04-11 04:25:33
|
Revision: 2238
http://archive-access.svn.sourceforge.net/archive-access/?rev=2238&view=rev
Author: bradtofel
Date: 2008-04-10 21:25:37 -0700 (Thu, 10 Apr 2008)
Log Message:
-----------
ADDED: oracle access control reference. Upped Spring config to 2.5.1
Modified Paths:
--------------
trunk/archive-access/projects/wayback/wayback-core/pom.xml
Modified: trunk/archive-access/projects/wayback/wayback-core/pom.xml
===================================================================
--- trunk/archive-access/projects/wayback/wayback-core/pom.xml 2008-04-11 04:15:37 UTC (rev 2237)
+++ trunk/archive-access/projects/wayback/wayback-core/pom.xml 2008-04-11 04:25:37 UTC (rev 2238)
@@ -60,6 +60,11 @@
<version>2.0.1-SNAPSHOT</version>
</dependency>
<dependency>
+ <groupId>org.archive.access-control</groupId>
+ <artifactId>access-control</artifactId>
+ <version>0.0.1-SNAPSHOT</version>
+ </dependency>
+ <dependency>
<groupId>org.mozilla</groupId>
<artifactId>juniversalchardet</artifactId>
<version>1.0</version>
@@ -67,12 +72,12 @@
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
- <version>2.0.5</version>
+ <version>2.5.1</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-beans</artifactId>
- <version>2.0.5</version>
+ <version>2.5.1</version>
</dependency>
<!--
Doh... I'm not sure what package is configuring org.apache.commons-logging
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <bra...@us...> - 2008-04-11 04:15:31
|
Revision: 2237
http://archive-access.svn.sourceforge.net/archive-access/?rev=2237&view=rev
Author: bradtofel
Date: 2008-04-10 21:15:37 -0700 (Thu, 10 Apr 2008)
Log Message:
-----------
TWEAK: added 'pom.' to artifactID to silence warning
Modified Paths:
--------------
trunk/archive-access/projects/wayback/pom.xml
Modified: trunk/archive-access/projects/wayback/pom.xml
===================================================================
--- trunk/archive-access/projects/wayback/pom.xml 2008-04-11 04:14:48 UTC (rev 2236)
+++ trunk/archive-access/projects/wayback/pom.xml 2008-04-11 04:15:37 UTC (rev 2237)
@@ -187,7 +187,7 @@
<id>website</id>
<name>Website</name>
<!--Pass as command-line system property to maven-->
- <url>${website.url}/projects/${artifactId}</url>
+ <url>${website.url}/projects/${pom.artifactId}</url>
</site>
</distributionManagement>
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <bra...@us...> - 2008-04-11 04:14:41
|
Revision: 2236
http://archive-access.svn.sourceforge.net/archive-access/?rev=2236&view=rev
Author: bradtofel
Date: 2008-04-10 21:14:48 -0700 (Thu, 10 Apr 2008)
Log Message:
-----------
Fixed speling problems.
Modified Paths:
--------------
trunk/archive-access/projects/wayback/dist/src/site/xdoc/release_notes.xml
Modified: trunk/archive-access/projects/wayback/dist/src/site/xdoc/release_notes.xml
===================================================================
--- trunk/archive-access/projects/wayback/dist/src/site/xdoc/release_notes.xml 2008-04-11 04:10:27 UTC (rev 2235)
+++ trunk/archive-access/projects/wayback/dist/src/site/xdoc/release_notes.xml 2008-04-11 04:14:48 UTC (rev 2236)
@@ -41,7 +41,7 @@
from each file sequentially.
</li>
<li>
- extracted several replay features into separately jspInserts that
+ extracted several replay features into separate jspInserts that
can now be mixed and matched.
</li>
<li>
@@ -49,7 +49,7 @@
and in externally linked .css files.
</li>
<li>
- externalized commented embedded inside replayed HTML pages into
+ externalized comment embedded inside replayed HTML pages into
jspInsert: ArchiveComment.jsp.
</li>
<li>
@@ -82,7 +82,7 @@
</li>
<li>
command line tools now allow whitespace arguments, important for
- tools accepting delimter arguments.
+ tools accepting delimiter arguments.
</li>
<li>
replay URLs in query results now include non-standard ports, if
@@ -90,7 +90,7 @@
</li>
<li>
Timezone is now explicitly set to GMT/UTC, fixing a Calendar
- result partioning problem.
+ result partitioning problem.
</li>
<li>
uncaught character-encoding exceptions now handled, plus
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <bra...@us...> - 2008-04-11 04:10:22
|
Revision: 2235
http://archive-access.svn.sourceforge.net/archive-access/?rev=2235&view=rev
Author: bradtofel
Date: 2008-04-10 21:10:27 -0700 (Thu, 10 Apr 2008)
Log Message:
-----------
FEATURE: implemented destroy() method, which locates all AccessPoints in the Spring configuration, and shuts each of them down.
Modified Paths:
--------------
trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/webapp/RequestMapper.java
Modified: trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/webapp/RequestMapper.java
===================================================================
--- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/webapp/RequestMapper.java 2008-04-11 04:09:26 UTC (rev 2234)
+++ trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/webapp/RequestMapper.java 2008-04-11 04:10:27 UTC (rev 2235)
@@ -24,7 +24,11 @@
*/
package org.archive.wayback.webapp;
+import java.io.IOException;
import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Iterator;
+import java.util.Map;
import java.util.logging.Logger;
import javax.servlet.ServletContext;
@@ -147,8 +151,41 @@
/**
* clean up all WaybackContexts, which should release resources gracefully.
*/
+ @SuppressWarnings("unchecked")
public void destroy() {
LOGGER.info("shutting down contexts...");
- //TODO: shut everything down
+ Class accessPointClass;
+ try {
+ accessPointClass = Class.forName(ACCESS_POINT_CLASSNAME);
+ Map beanMap = factory.getBeansOfType(accessPointClass);
+ Iterator beanNameItr = beanMap.keySet().iterator();
+ Collection accessPoints = beanMap.values();
+ while(beanNameItr.hasNext()) {
+ String apName = (String) beanNameItr.next();
+ AccessPoint ap = (AccessPoint) beanMap.get(apName);
+ try {
+ LOGGER.info("Shutting down AccessPoint " + apName);
+ ap.shutdown();
+ LOGGER.info("Successfully shut down " + apName);
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+ }
+ for(Object o : accessPoints) {
+ if(o instanceof AccessPoint) {
+ AccessPoint ap = (AccessPoint) o;
+ try {
+ ap.shutdown();
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+ }
+ }
+
+ } catch (ClassNotFoundException e) {
+ // TODO Auto-generated catch block
+ e.printStackTrace();
+ }
+
}
}
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <bra...@us...> - 2008-04-11 04:09:20
|
Revision: 2234
http://archive-access.svn.sourceforge.net/archive-access/?rev=2234&view=rev
Author: bradtofel
Date: 2008-04-10 21:09:26 -0700 (Thu, 10 Apr 2008)
Log Message:
-----------
FEATURE: added shutdown() method, which shuts down the local collection.
Modified Paths:
--------------
trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/webapp/AccessPoint.java
Modified: trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/webapp/AccessPoint.java
===================================================================
--- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/webapp/AccessPoint.java 2008-04-11 04:08:21 UTC (rev 2233)
+++ trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/webapp/AccessPoint.java 2008-04-11 04:09:26 UTC (rev 2234)
@@ -348,6 +348,12 @@
}
}
+ public void shutdown() throws IOException {
+ if(collection != null) {
+ collection.shutdown();
+ }
+ }
+
private void logNotInArchive(WaybackException e, WaybackRequest r) {
if(e instanceof ResourceNotInArchiveException) {
String url = r.get(WaybackConstants.REQUEST_URL);
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <bra...@us...> - 2008-04-11 04:08:16
|
Revision: 2233
http://archive-access.svn.sourceforge.net/archive-access/?rev=2233&view=rev
Author: bradtofel
Date: 2008-04-10 21:08:21 -0700 (Thu, 10 Apr 2008)
Log Message:
-----------
FEATURE: added shutdown() method, which shuts down ResourceStore and ResourceIndex
Modified Paths:
--------------
trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/webapp/WaybackCollection.java
Modified: trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/webapp/WaybackCollection.java
===================================================================
--- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/webapp/WaybackCollection.java 2008-04-11 04:07:18 UTC (rev 2232)
+++ trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/webapp/WaybackCollection.java 2008-04-11 04:08:21 UTC (rev 2233)
@@ -24,6 +24,8 @@
*/
package org.archive.wayback.webapp;
+import java.io.IOException;
+
import org.archive.wayback.ResourceIndex;
import org.archive.wayback.ResourceStore;
import org.archive.wayback.exception.ConfigurationException;
@@ -38,6 +40,7 @@
public class WaybackCollection {
private ResourceStore resourceStore = null;
private ResourceIndex resourceIndex = null;
+ private boolean shutdownDone = false;
public ResourceStore getResourceStore() throws ConfigurationException {
if(resourceStore == null) {
throw new ConfigurationException("No resourceStore declared");
@@ -56,4 +59,16 @@
public void setResourceIndex(ResourceIndex resourceIndex) {
this.resourceIndex = resourceIndex;
}
+ public void shutdown() throws IOException {
+ if(shutdownDone) {
+ return;
+ }
+ if(resourceStore != null) {
+ resourceStore.shutdown();
+ }
+ if(resourceIndex != null) {
+ resourceIndex.shutdown();
+ }
+ shutdownDone = true;
+ }
}
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <bra...@us...> - 2008-04-11 04:07:13
|
Revision: 2232
http://archive-access.svn.sourceforge.net/archive-access/?rev=2232&view=rev
Author: bradtofel
Date: 2008-04-10 21:07:18 -0700 (Thu, 10 Apr 2008)
Log Message:
-----------
INTERFACE: added shutdown() to ResourceIndex interface, LocalResourceIndex is the only implementation that uses this at present, shutting down it's SearchResultSource
Modified Paths:
--------------
trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/ResourceIndex.java
trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/LocalResourceIndex.java
trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/RemoteResourceIndex.java
trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/distributed/AlphaPartitionedIndex.java
trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/distributed/RangeGroup.java
trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/distributed/RangeMember.java
Modified: trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/ResourceIndex.java
===================================================================
--- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/ResourceIndex.java 2008-04-11 04:06:12 UTC (rev 2231)
+++ trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/ResourceIndex.java 2008-04-11 04:07:18 UTC (rev 2232)
@@ -23,6 +23,8 @@
package org.archive.wayback;
+import java.io.IOException;
+
import org.archive.wayback.core.SearchResults;
import org.archive.wayback.core.WaybackRequest;
import org.archive.wayback.exception.AccessControlException;
@@ -53,4 +55,6 @@
throws ResourceIndexNotAvailableException,
ResourceNotInArchiveException, BadQueryException,
AccessControlException;
+
+ public void shutdown() throws IOException;
}
Modified: trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/LocalResourceIndex.java
===================================================================
--- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/LocalResourceIndex.java 2008-04-11 04:06:12 UTC (rev 2231)
+++ trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/LocalResourceIndex.java 2008-04-11 04:07:18 UTC (rev 2232)
@@ -464,4 +464,8 @@
public void setCanonicalizer(UrlCanonicalizer canonicalizer) {
this.canonicalizer = canonicalizer;
}
+
+ public void shutdown() throws IOException {
+ source.shutdown();
+ }
}
Modified: trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/RemoteResourceIndex.java
===================================================================
--- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/RemoteResourceIndex.java 2008-04-11 04:06:12 UTC (rev 2231)
+++ trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/RemoteResourceIndex.java 2008-04-11 04:07:18 UTC (rev 2232)
@@ -287,4 +287,8 @@
public void setSearchUrlBase(String searchUrlBase) {
this.searchUrlBase = searchUrlBase;
}
+
+ public void shutdown() throws IOException {
+ // No-op
+ }
}
\ No newline at end of file
Modified: trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/distributed/AlphaPartitionedIndex.java
===================================================================
--- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/distributed/AlphaPartitionedIndex.java 2008-04-11 04:06:12 UTC (rev 2231)
+++ trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/distributed/AlphaPartitionedIndex.java 2008-04-11 04:07:18 UTC (rev 2232)
@@ -248,4 +248,10 @@
public void setCanonicalizer(UrlCanonicalizer canonicalizer) {
this.canonicalizer = canonicalizer;
}
+
+ public void shutdown() throws IOException {
+ for(RangeGroup group : groups) {
+ group.shutdown();
+ }
+ }
}
Modified: trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/distributed/RangeGroup.java
===================================================================
--- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/distributed/RangeGroup.java 2008-04-11 04:06:12 UTC (rev 2231)
+++ trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/distributed/RangeGroup.java 2008-04-11 04:07:18 UTC (rev 2232)
@@ -24,6 +24,7 @@
*/
package org.archive.wayback.resourceindex.distributed;
+import java.io.IOException;
import java.util.Comparator;
import java.util.HashMap;
import java.util.Iterator;
@@ -169,4 +170,10 @@
}
};
}
+
+ public void shutdown() throws IOException {
+ for(RangeMember member : members.values()) {
+ member.shutdown();
+ }
+ }
}
Modified: trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/distributed/RangeMember.java
===================================================================
--- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/distributed/RangeMember.java 2008-04-11 04:06:12 UTC (rev 2231)
+++ trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/distributed/RangeMember.java 2008-04-11 04:07:18 UTC (rev 2232)
@@ -24,6 +24,8 @@
*/
package org.archive.wayback.resourceindex.distributed;
+import java.io.IOException;
+
import org.archive.wayback.ResourceIndex;
import org.archive.wayback.core.SearchResults;
import org.archive.wayback.core.WaybackRequest;
@@ -89,4 +91,7 @@
public void setUrlBase(String urlBase) {
index.setSearchUrlBase(urlBase);
}
+ public void shutdown() throws IOException {
+ index.shutdown();
+ }
}
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <bra...@us...> - 2008-04-11 04:06:09
|
Revision: 2231
http://archive-access.svn.sourceforge.net/archive-access/?rev=2231&view=rev
Author: bradtofel
Date: 2008-04-10 21:06:12 -0700 (Thu, 10 Apr 2008)
Log Message:
-----------
INTERFACE: added shutdown() method to SearchResultSource interface. BDBIndex is the only implementation which uses this at present.
Modified Paths:
--------------
trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/CompositeSearchResultSource.java
trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/SearchResultSource.java
trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/bdb/BDBIndex.java
trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/cdx/CDXIndex.java
Modified: trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/CompositeSearchResultSource.java
===================================================================
--- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/CompositeSearchResultSource.java 2008-04-11 04:05:05 UTC (rev 2230)
+++ trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/CompositeSearchResultSource.java 2008-04-11 04:06:12 UTC (rev 2231)
@@ -137,4 +137,10 @@
public List<SearchResultSource> getSources() {
return sources;
}
+
+ public void shutdown() throws IOException {
+ for(SearchResultSource source : sources) {
+ source.shutdown();
+ }
+ }
}
Modified: trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/SearchResultSource.java
===================================================================
--- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/SearchResultSource.java 2008-04-11 04:05:05 UTC (rev 2230)
+++ trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/SearchResultSource.java 2008-04-11 04:06:12 UTC (rev 2231)
@@ -62,4 +62,10 @@
* @throws IOException
*/
public void cleanup(CloseableIterator<SearchResult> c) throws IOException;
+
+ /**
+ * @param c
+ * @throws IOException
+ */
+ public void shutdown() throws IOException;
}
Modified: trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/bdb/BDBIndex.java
===================================================================
--- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/bdb/BDBIndex.java 2008-04-11 04:05:05 UTC (rev 2230)
+++ trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/bdb/BDBIndex.java 2008-04-11 04:06:12 UTC (rev 2231)
@@ -254,4 +254,12 @@
public void setUpdater(BDBIndexUpdater updater) {
this.updater = updater;
}
+
+ public void shutdown() throws IOException {
+ try {
+ shutdownDB();
+ } catch (DatabaseException e) {
+ throw new IOException(e.getMessage());
+ }
+ }
}
Modified: trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/cdx/CDXIndex.java
===================================================================
--- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/cdx/CDXIndex.java 2008-04-11 04:05:05 UTC (rev 2230)
+++ trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/resourceindex/cdx/CDXIndex.java 2008-04-11 04:06:12 UTC (rev 2231)
@@ -142,4 +142,8 @@
public void cleanup(CloseableIterator<SearchResult> c) throws IOException {
c.close();
}
+
+ public void shutdown() throws IOException {
+ // no-op
+ }
}
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
