You can subscribe to this list here.
2005 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(1) |
Aug
(10) |
Sep
(36) |
Oct
(339) |
Nov
(103) |
Dec
(152) |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2006 |
Jan
(141) |
Feb
(102) |
Mar
(125) |
Apr
(203) |
May
(57) |
Jun
(30) |
Jul
(139) |
Aug
(46) |
Sep
(64) |
Oct
(105) |
Nov
(34) |
Dec
(162) |
2007 |
Jan
(81) |
Feb
(57) |
Mar
(141) |
Apr
(72) |
May
(9) |
Jun
(1) |
Jul
(144) |
Aug
(88) |
Sep
(40) |
Oct
(43) |
Nov
(34) |
Dec
(20) |
2008 |
Jan
(44) |
Feb
(45) |
Mar
(16) |
Apr
(36) |
May
(8) |
Jun
(77) |
Jul
(177) |
Aug
(66) |
Sep
(8) |
Oct
(33) |
Nov
(13) |
Dec
(37) |
2009 |
Jan
(2) |
Feb
(5) |
Mar
(8) |
Apr
|
May
(36) |
Jun
(19) |
Jul
(46) |
Aug
(8) |
Sep
(1) |
Oct
(66) |
Nov
(61) |
Dec
(10) |
2010 |
Jan
(13) |
Feb
(16) |
Mar
(38) |
Apr
(76) |
May
(47) |
Jun
(32) |
Jul
(35) |
Aug
(45) |
Sep
(20) |
Oct
(61) |
Nov
(24) |
Dec
(16) |
2011 |
Jan
(22) |
Feb
(34) |
Mar
(11) |
Apr
(8) |
May
(24) |
Jun
(23) |
Jul
(11) |
Aug
(42) |
Sep
(81) |
Oct
(48) |
Nov
(21) |
Dec
(20) |
2012 |
Jan
(30) |
Feb
(25) |
Mar
(4) |
Apr
(6) |
May
(1) |
Jun
(5) |
Jul
(5) |
Aug
(8) |
Sep
(6) |
Oct
(6) |
Nov
|
Dec
|
From: <al...@us...> - 2008-02-27 11:11:45
|
Revision: 2205 http://archive-access.svn.sourceforge.net/archive-access/?rev=2205&view=rev Author: alexoz Date: 2008-02-27 03:10:51 -0800 (Wed, 27 Feb 2008) Log Message: ----------- Added mvn eclipse:eclipse settings files. Added Paths: ----------- trunk/archive-access/projects/access-control/access-control/.classpath trunk/archive-access/projects/access-control/access-control/.project trunk/archive-access/projects/access-control/access-control/.settings/ trunk/archive-access/projects/access-control/access-control/.settings/org.eclipse.jdt.core.prefs trunk/archive-access/projects/access-control/access-control/.settings/org.eclipse.wst.common.component trunk/archive-access/projects/access-control/access-control/.settings/org.eclipse.wst.common.project.facet.core.xml Property Changed: ---------------- trunk/archive-access/projects/access-control/access-control/ Property changes on: trunk/archive-access/projects/access-control/access-control ___________________________________________________________________ Name: svn:ignore + target Added: trunk/archive-access/projects/access-control/access-control/.classpath =================================================================== --- trunk/archive-access/projects/access-control/access-control/.classpath (rev 0) +++ trunk/archive-access/projects/access-control/access-control/.classpath 2008-02-27 11:10:51 UTC (rev 2205) @@ -0,0 +1,36 @@ +<classpath> + <classpathentry kind="src" path="src/main/java"/> + <classpathentry kind="src" path="src/main/resources" excluding="**/*.java"/> + <classpathentry kind="src" path="src/test/java" output="target/test-classes"/> + <classpathentry kind="output" path="target/classes"/> + <classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER"/> + <classpathentry kind="var" path="M2_REPO/commons-lang/commons-lang/2.3/commons-lang-2.3.jar"/> + <classpathentry kind="var" path="M2_REPO/org/dnsjava/dnsjava/2.0.3/dnsjava-2.0.3.jar"/> + <classpathentry kind="var" path="M2_REPO/it/unimi/dsi/mg4j/1.0.1/mg4j-1.0.1.jar"/> + <classpathentry kind="var" path="M2_REPO/org/archive/heritrix/commons/2.0.0-RC1/commons-2.0.0-RC1.jar"/> + <classpathentry kind="var" path="M2_REPO/org/codehaus/jettison/jettison/1.0-beta-1/jettison-1.0-beta-1.jar"/> + <classpathentry kind="var" path="M2_REPO/net/java/dev/jets3t/jets3t/0.5.0/jets3t-0.5.0.jar"/> + <classpathentry kind="var" path="M2_REPO/commons-cli/commons-cli/1.0/commons-cli-1.0.jar"/> + <classpathentry kind="var" path="M2_REPO/stax/stax-api/1.0.1/stax-api-1.0.1.jar"/> + <classpathentry kind="var" path="M2_REPO/junit/junit/4.4/junit-4.4.jar"/> + <classpathentry kind="var" path="M2_REPO/org/archive/overlays/archive-overlay-commons-httpclient/3.1/archive-overlay-commons-httpclient-3.1.jar"/> + <classpathentry kind="var" path="M2_REPO/org/archive/overlays/archive-overlay-commons-pool/1.3/archive-overlay-commons-pool-1.3.jar"/> + <classpathentry kind="var" path="M2_REPO/commons-io/commons-io/1.3.1/commons-io-1.3.1.jar"/> + <classpathentry kind="var" path="M2_REPO/poi/poi-scratchpad/2.5.1-final-20040804/poi-scratchpad-2.5.1-final-20040804.jar"/> + <classpathentry kind="var" path="M2_REPO/net/htmlparser/jericho/jericho-html/2.3/jericho-html-2.3.jar"/> + <classpathentry kind="var" path="M2_REPO/org/gnu/inet/libidn/0.6.5/libidn-0.6.5.jar"/> + <classpathentry kind="var" path="M2_REPO/commons-pool/commons-pool/1.3/commons-pool-1.3.jar"/> + <classpathentry kind="var" path="M2_REPO/com/anotherbigidea/javaswf/CVS-SNAPSHOT-1/javaswf-CVS-SNAPSHOT-1.jar"/> + <classpathentry kind="var" path="M2_REPO/commons-codec/commons-codec/1.3/commons-codec-1.3.jar"/> + <classpathentry kind="var" path="M2_REPO/commons-collections/commons-collections/3.1/commons-collections-3.1.jar"/> + <classpathentry kind="var" path="M2_REPO/fastutil/fastutil/5.0.7/fastutil-5.0.7.jar"/> + <classpathentry kind="var" path="M2_REPO/commons-logging/commons-logging/1.0.4/commons-logging-1.0.4.jar"/> + <classpathentry kind="var" path="M2_REPO/oro/oro/2.0.8/oro-2.0.8.jar"/> + <classpathentry kind="var" path="M2_REPO/poi/poi/2.5.1-final-20040804/poi-2.5.1-final-20040804.jar"/> + <classpathentry kind="var" path="M2_REPO/commons-dbcp/commons-dbcp/1.2.2/commons-dbcp-1.2.2.jar"/> + <classpathentry kind="var" path="M2_REPO/xpp3/xpp3_min/1.1.3.4.O/xpp3_min-1.1.3.4.O.jar"/> + <classpathentry kind="var" path="M2_REPO/com/lowagie/itext/1.3/itext-1.3.jar"/> + <classpathentry kind="var" path="M2_REPO/berkeleydb/je/3.2.44/je-3.2.44.jar"/> + <classpathentry kind="var" path="M2_REPO/commons-net/commons-net/1.4.1/commons-net-1.4.1.jar"/> + <classpathentry kind="var" path="M2_REPO/com/thoughtworks/xstream/xstream/1.2.2/xstream-1.2.2.jar"/> +</classpath> \ No newline at end of file Added: trunk/archive-access/projects/access-control/access-control/.project =================================================================== --- trunk/archive-access/projects/access-control/access-control/.project (rev 0) +++ trunk/archive-access/projects/access-control/access-control/.project 2008-02-27 11:10:51 UTC (rev 2205) @@ -0,0 +1,22 @@ +<projectDescription> + <name>access-control</name> + <comment></comment> + <projects/> + <buildSpec> + <buildCommand> + <name>org.eclipse.jdt.core.javabuilder</name> + </buildCommand> + <buildCommand> + <name>org.eclipse.wst.common.project.facet.core.builder</name> + </buildCommand> + <buildCommand> + <name>org.eclipse.wst.validation.validationbuilder</name> + </buildCommand> + </buildSpec> + <natures> + <nature>org.eclipse.wst.common.project.facet.core.nature</nature> + <nature>org.eclipse.jdt.core.javanature</nature> + <nature>org.eclipse.wst.common.modulecore.ModuleCoreNature</nature> + <nature>org.eclipse.jem.workbench.JavaEMFNature</nature> + </natures> +</projectDescription> \ No newline at end of file Added: trunk/archive-access/projects/access-control/access-control/.settings/org.eclipse.jdt.core.prefs =================================================================== --- trunk/archive-access/projects/access-control/access-control/.settings/org.eclipse.jdt.core.prefs (rev 0) +++ trunk/archive-access/projects/access-control/access-control/.settings/org.eclipse.jdt.core.prefs 2008-02-27 11:10:51 UTC (rev 2205) @@ -0,0 +1,5 @@ +#Wed Feb 27 20:55:56 EST 2008 +org.eclipse.jdt.core.compiler.codegen.targetPlatform=1.5 +eclipse.preferences.version=1 +org.eclipse.jdt.core.compiler.source=1.5 +org.eclipse.jdt.core.compiler.compliance=1.5 Added: trunk/archive-access/projects/access-control/access-control/.settings/org.eclipse.wst.common.component =================================================================== --- trunk/archive-access/projects/access-control/access-control/.settings/org.eclipse.wst.common.component (rev 0) +++ trunk/archive-access/projects/access-control/access-control/.settings/org.eclipse.wst.common.component 2008-02-27 11:10:51 UTC (rev 2205) @@ -0,0 +1,6 @@ +<project-modules id="moduleCoreId" project-version="1.5.0"> + <wb-module deploy-name="access-control"> + <wb-resource deploy-path="/" source-path="src/main/java"/> + <wb-resource deploy-path="/" source-path="src/main/resources"/> + </wb-module> +</project-modules> \ No newline at end of file Added: trunk/archive-access/projects/access-control/access-control/.settings/org.eclipse.wst.common.project.facet.core.xml =================================================================== --- trunk/archive-access/projects/access-control/access-control/.settings/org.eclipse.wst.common.project.facet.core.xml (rev 0) +++ trunk/archive-access/projects/access-control/access-control/.settings/org.eclipse.wst.common.project.facet.core.xml 2008-02-27 11:10:51 UTC (rev 2205) @@ -0,0 +1,6 @@ +<faceted-project> + <fixed facet="jst.java"/> + <fixed facet="jst.utility"/> + <installed facet="jst.utility" version="1.0"/> + <installed facet="jst.java" version="5.0"/> +</faceted-project> \ No newline at end of file This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <al...@us...> - 2008-02-27 10:23:10
|
Revision: 2204 http://archive-access.svn.sourceforge.net/archive-access/?rev=2204&view=rev Author: alexoz Date: 2008-02-27 02:21:34 -0800 (Wed, 27 Feb 2008) Log Message: ----------- Redirect oracle root to admin page. * index.html Replaced by index.jsp * index.jsp Created to redirect to admin?surt=( * jscalendar-1.0/index.html Deleted to prevent eclipse errors. * web.xml Use index.jsp as the welcome page instead of index.html. Modified Paths: -------------- trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/list_rules.jsp trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/web.xml Added Paths: ----------- trunk/archive-access/projects/access-control/oracle/src/main/webapp/index.jsp Removed Paths: ------------- trunk/archive-access/projects/access-control/oracle/src/main/webapp/index.html trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/index.html Modified: trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/list_rules.jsp =================================================================== --- trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/list_rules.jsp 2008-02-21 01:55:36 UTC (rev 2203) +++ trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/list_rules.jsp 2008-02-27 10:21:34 UTC (rev 2204) @@ -14,6 +14,14 @@ </ul> </div> +<div id="childSurts"> +<ul> + <c:forEach var="child" items="${childSurts}"> + <li><a href="<c:url value="/admin?surt=${surt}${child}" />"><c:out value="${child}" /></a></li> + </c:forEach> +</ul> +</div> + <table> <thead> <tr> Modified: trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/web.xml =================================================================== --- trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/web.xml 2008-02-21 01:55:36 UTC (rev 2203) +++ trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/web.xml 2008-02-27 10:21:34 UTC (rev 2204) @@ -69,7 +69,7 @@ </servlet-mapping> <welcome-file-list> - <welcome-file>index.html</welcome-file> + <welcome-file>index.jsp</welcome-file> </welcome-file-list> Deleted: trunk/archive-access/projects/access-control/oracle/src/main/webapp/index.html =================================================================== --- trunk/archive-access/projects/access-control/oracle/src/main/webapp/index.html 2008-02-21 01:55:36 UTC (rev 2203) +++ trunk/archive-access/projects/access-control/oracle/src/main/webapp/index.html 2008-02-27 10:21:34 UTC (rev 2204) @@ -1 +0,0 @@ -<h1>Hello world.</h1> \ No newline at end of file Added: trunk/archive-access/projects/access-control/oracle/src/main/webapp/index.jsp =================================================================== --- trunk/archive-access/projects/access-control/oracle/src/main/webapp/index.jsp (rev 0) +++ trunk/archive-access/projects/access-control/oracle/src/main/webapp/index.jsp 2008-02-27 10:21:34 UTC (rev 2204) @@ -0,0 +1,3 @@ +<% +response.sendRedirect("admin?surt=("); +%> \ No newline at end of file Deleted: trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/index.html =================================================================== --- trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/index.html 2008-02-21 01:55:36 UTC (rev 2203) +++ trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/index.html 2008-02-27 10:21:34 UTC (rev 2204) @@ -1,330 +0,0 @@ -<?xml version="1.0" encoding="utf-8"?> -<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> -<html xmlns="http://www.w3.org/1999/xhtml"> -<!-- $Id: index.html,v 1.15 2005/03/05 14:38:10 mishoo Exp $ --> - -<head> -<meta http-equiv="content-type" content="text/xml; charset=utf-8" /> -<title>The Coolest DHTML Calendar - Online Demo</title> -<link rel="stylesheet" type="text/css" media="all" href="skins/aqua/theme.css" title="Aqua" /> -<link rel="alternate stylesheet" type="text/css" media="all" href="calendar-blue.css" title="winter" /> -<link rel="alternate stylesheet" type="text/css" media="all" href="calendar-blue2.css" title="blue" /> -<link rel="alternate stylesheet" type="text/css" media="all" href="calendar-brown.css" title="summer" /> -<link rel="alternate stylesheet" type="text/css" media="all" href="calendar-green.css" title="green" /> -<link rel="alternate stylesheet" type="text/css" media="all" href="calendar-win2k-1.css" title="win2k-1" /> -<link rel="alternate stylesheet" type="text/css" media="all" href="calendar-win2k-2.css" title="win2k-2" /> -<link rel="alternate stylesheet" type="text/css" media="all" href="calendar-win2k-cold-1.css" title="win2k-cold-1" /> -<link rel="alternate stylesheet" type="text/css" media="all" href="calendar-win2k-cold-2.css" title="win2k-cold-2" /> -<link rel="alternate stylesheet" type="text/css" media="all" href="calendar-system.css" title="system" /> - -<!-- import the calendar script --> -<script type="text/javascript" src="calendar.js"></script> - -<!-- import the language module --> -<script type="text/javascript" src="lang/calendar-en.js"></script> - -<!-- other languages might be available in the lang directory; please check -your distribution archive. --> - -<!-- helper script that uses the calendar --> -<script type="text/javascript"> - -var oldLink = null; -// code to change the active stylesheet -function setActiveStyleSheet(link, title) { - var i, a, main; - for(i=0; (a = document.getElementsByTagName("link")[i]); i++) { - if(a.getAttribute("rel").indexOf("style") != -1 && a.getAttribute("title")) { - a.disabled = true; - if(a.getAttribute("title") == title) a.disabled = false; - } - } - if (oldLink) oldLink.style.fontWeight = 'normal'; - oldLink = link; - link.style.fontWeight = 'bold'; - return false; -} - -// This function gets called when the end-user clicks on some date. -function selected(cal, date) { - cal.sel.value = date; // just update the date in the input field. - if (cal.dateClicked && (cal.sel.id == "sel1" || cal.sel.id == "sel3")) - // if we add this call we close the calendar on single-click. - // just to exemplify both cases, we are using this only for the 1st - // and the 3rd field, while 2nd and 4th will still require double-click. - cal.callCloseHandler(); -} - -// And this gets called when the end-user clicks on the _selected_ date, -// or clicks on the "Close" button. It just hides the calendar without -// destroying it. -function closeHandler(cal) { - cal.hide(); // hide the calendar -// cal.destroy(); - _dynarch_popupCalendar = null; -} - -// This function shows the calendar under the element having the given id. -// It takes care of catching "mousedown" signals on document and hiding the -// calendar if the click was outside. -function showCalendar(id, format, showsTime, showsOtherMonths) { - var el = document.getElementById(id); - if (_dynarch_popupCalendar != null) { - // we already have some calendar created - _dynarch_popupCalendar.hide(); // so we hide it first. - } else { - // first-time call, create the calendar. - var cal = new Calendar(1, null, selected, closeHandler); - // uncomment the following line to hide the week numbers - // cal.weekNumbers = false; - if (typeof showsTime == "string") { - cal.showsTime = true; - cal.time24 = (showsTime == "24"); - } - if (showsOtherMonths) { - cal.showsOtherMonths = true; - } - _dynarch_popupCalendar = cal; // remember it in the global var - cal.setRange(1900, 2070); // min/max year allowed. - cal.create(); - } - _dynarch_popupCalendar.setDateFormat(format); // set the specified date format - _dynarch_popupCalendar.parseDate(el.value); // try to parse the text in field - _dynarch_popupCalendar.sel = el; // inform it what input field we use - - // the reference element that we pass to showAtElement is the button that - // triggers the calendar. In this example we align the calendar bottom-right - // to the button. - _dynarch_popupCalendar.showAtElement(el.nextSibling, "Br"); // show the calendar - - return false; -} - -var MINUTE = 60 * 1000; -var HOUR = 60 * MINUTE; -var DAY = 24 * HOUR; -var WEEK = 7 * DAY; - -// If this handler returns true then the "date" given as -// parameter will be disabled. In this example we enable -// only days within a range of 10 days from the current -// date. -// You can use the functions date.getFullYear() -- returns the year -// as 4 digit number, date.getMonth() -- returns the month as 0..11, -// and date.getDate() -- returns the date of the month as 1..31, to -// make heavy calculations here. However, beware that this function -// should be very fast, as it is called for each day in a month when -// the calendar is (re)constructed. -function isDisabled(date) { - var today = new Date(); - return (Math.abs(date.getTime() - today.getTime()) / DAY) > 10; -} - -function flatSelected(cal, date) { - var el = document.getElementById("preview"); - el.innerHTML = date; -} - -function showFlatCalendar() { - var parent = document.getElementById("display"); - - // construct a calendar giving only the "selected" handler. - var cal = new Calendar(0, null, flatSelected); - - // hide week numbers - cal.weekNumbers = false; - - // We want some dates to be disabled; see function isDisabled above - cal.setDisabledHandler(isDisabled); - cal.setDateFormat("%A, %B %e"); - - // this call must be the last as it might use data initialized above; if - // we specify a parent, as opposite to the "showCalendar" function above, - // then we create a flat calendar -- not popup. Hidden, though, but... - cal.create(parent); - - // ... we can show it here. - cal.show(); -} -</script> - -<style type="text/css"> -.ex { font-weight: bold; background: #fed; color: #080 } -.help { color: #080; font-style: italic; } -body { background: #fea; font: 10pt tahoma,verdana,sans-serif; } -table { font: 13px verdana,tahoma,sans-serif; } -a { color: #00f; } -a:visited { color: #00f; } -a:hover { color: #f00; background: #fefaf0; } -a:active { color: #08f; } -.key { border: 1px solid #000; background: #fff; color: #008; -padding: 0px 5px; cursor: default; font-size: 80%; } -</style> - -</head> -<body onload="showFlatCalendar()"> - -<h2><a href="http://www.dynarch.com/projects/calendar/" -title="Visit the project website">jscalendar</a>-1.0 -"It is happening again"</h2> - -<p> -<div style="float: right; border: 1px solid #b87; padding: 2px; font-size: 90%; background: #ffb;"> -Theme:<br /> -<a href="#" id="defaultTheme" onclick="return setActiveStyleSheet(this, 'Aqua');">Aqua</a> -| -<a href="#" onclick="return setActiveStyleSheet(this, 'winter');">winter</a> -| -<a href="#" onclick="return setActiveStyleSheet(this, 'blue');">blue</a> -| -<a href="#" onclick="return setActiveStyleSheet(this, 'summer');">summer</a> -| -<a href="#" onclick="return setActiveStyleSheet(this, 'green');">green</a> -<br /> -<a href="#" onclick="return setActiveStyleSheet(this, 'win2k-1');">win2k-1</a> -| -<a href="#" onclick="return setActiveStyleSheet(this, 'win2k-2');">win2k-2</a> -| -<a href="#" onclick="return setActiveStyleSheet(this, 'win2k-cold-1');">win2k-cold-1</a> -| -<a href="#" onclick="return setActiveStyleSheet(this, 'win2k-cold-2');">win2k-cold-2</a> -<br /> -<a href="#" onclick="return setActiveStyleSheet(this, 'system');">system</a> -<script type="text/javascript"> -setActiveStyleSheet(document.getElementById("defaultTheme"), "Aqua"); -</script> -</div> -<a href="release-notes.html">Release notes</a>. -<br /> -Set it up in minutes: - <a href="simple-1.html">popup calendar</a>, - <a href="simple-2.html">flat calendar</a>. -Other samples: - <a href="simple-3.html">special days</a>, - <a href="dayinfo.html">day info</a>, - <a href="multiple-dates.html">multiple dates selection</a> -<br /> -Documentation: - <a href="doc/html/reference.html">HTML</a>, - <a href="doc/reference.pdf">PDF</a>. -<br /> -</p> - -<div style="padding-left:20px; font-size: 90%; font-style: italic;"> - -</div> - -<table style="width: 100%"> -<tr valign="top"> -<td style="background: #ffa; padding: 5px; border: 1px solid #995;"> - -<form action="#"> -<div style="background: #995; color: #ffa; font-weight: bold; padding: 2px;"> -Popup examples -</div> - -<br /> - -<b>Date #1:</b> <input type="text" name="date1" id="sel1" size="30" -><input type="reset" value=" ... " -onclick="return showCalendar('sel1', '%Y-%m-%d [%W] %H:%M', '24', true);"> %Y-%m-%d [%W] %H:%M -- single -click<br /> - -<b>Date #2:</b> <input type="text" name="date2" id="sel2" size="30" -><input type="reset" value=" ... " -onclick="return showCalendar('sel2', '%a, %b %e, %Y [%I:%M %p]', '12');"> %a, %b %e, %Y [%I:%M %p] --- double click - -<br /><br /> -<!-- -if you remove this comment and leave the following HTML code -you will see a horrible effect, in all supported browsers (IE and Mozilla). ---> -<SELECT multiple size="4" name="component-select"> - <OPTION selected value="Component_1_a">Component_1</OPTION> - <OPTION selected value="Component_1_b">Component_2</OPTION> - <OPTION>Component_3</OPTION> - <OPTION>Component_4</OPTION> - <OPTION>Component_5</OPTION> - <OPTION>Component_6</OPTION> - <OPTION>Component_7</OPTION> -</SELECT> -this select should hide when the calendar is above it. -<br /><br /> - -<b>Date #3:</b> <input type="text" name="date3" id="sel3" size="30" -><input type="reset" value=" ... " -onclick="return showCalendar('sel3', '%d/%m/%Y');"> %d/%m/%Y --- single click -<br /> - -<b>Date #4:</b> <input type="text" name="date4" id="sel4" size="30" -><input type="reset" value=" ... " -onclick="return showCalendar('sel4', '%A, %B %e, %Y');"> %A, %B %e, %Y -- -double click - -</form> - -<p>This is release <b>1.0</b>. Works on MSIE/Win 5.0 or better (really), -Opera 7+, Mozilla, Firefox, Netscape 6.x, 7.0 and all other Gecko-s, -Konqueror and Safari.</p> - -<h4>Keyboard navigation</h4> - -<p>Starting with version 0.9.2, you can also use the keyboard to select -dates (only for popup calendars; does <em>not</em> work with Opera -7 or Konqueror/Safari). The following keys are available:</p> - -<ul> - - <li><span class="key">←</span> , <span class="key">→</span> , - <span class="key">↑</span> , <span class="key">↓</span> -- select date</li> - <li><span class="key">CTRL</span> + <span class="key">←</span> , - <span class="key">→</span> -- select month</li> - <li><span class="key">CTRL</span> + <span class="key">↑</span> , - <span class="key">↓</span> -- select year</li> - <li><span class="key">SPACE</span> -- go to <em>today</em> date</li> - <li><span class="key">ENTER</span> -- accept the currently selected date</li> - <li><span class="key">ESC</span> -- cancel selection</li> - -</ul> - - </td> - - <td style="padding: 5px; margin: 5px; border: 1px solid #984; background: #ed9; width: 19em;"> - - <div style="background: #984; color: #fea; font-weight: bold; padding: 2px; text-align: center"> - Flat calendar - </div> - - <p style="width: 12em"><small>A non-popup version will appear below as soon - as the page is loaded. Note that it doesn't show the week number.</small></p> - - <!-- the calendar will be inserted here --> - <div id="display" style="float: right; clear: both;"></div> - <div id="preview" style="font-size: 80%; text-align: center; padding: 2px"> </div> - - <p style="clear: both;"><small> - The example above uses the <code>setDisabledHandler()</code> member function - to setup a handler that would only enable days withing a range of 10 days, - forward or backward, from the current date. - </small></p> - - </div> - - </td> - - </tr> - </table> - -<hr /><address> -© <a href="http://www.dynarch.com/">dynarch.com</a> 2002-2005 <br /> -Author: <a href="http://www.bazon.net/mishoo/">Mihai -Bazon</a><br /> Distributed under the <a -href="http://www.gnu.org/licenses/lgpl.html">GNU LGPL</a>.</address> - -<p style="font-size: smaller">If you use this script on a public page we -would love it if you would <a href="http://www.dynarch.com/contact.html">let us -know</a>.</p> - -</body></html> This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <bra...@us...> - 2008-02-21 01:55:33
|
Revision: 2203 http://archive-access.svn.sourceforge.net/archive-access/?rev=2203&view=rev Author: bradtofel Date: 2008-02-20 17:55:36 -0800 (Wed, 20 Feb 2008) Log Message: ----------- BUGFIX: (ACC-11) now associates last requested timestamp for the current user with query as well as replay requests, allowing the closest indicator to be set properly on XML query requests. Modified Paths: -------------- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/proxy/ProxyReplayRequestParser.java trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/proxy/ProxyRequestParser.java Modified: trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/proxy/ProxyReplayRequestParser.java =================================================================== --- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/proxy/ProxyReplayRequestParser.java 2008-02-20 23:50:37 UTC (rev 2202) +++ trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/proxy/ProxyReplayRequestParser.java 2008-02-21 01:55:36 UTC (rev 2203) @@ -31,7 +31,6 @@ import org.apache.commons.httpclient.URIException; import org.archive.util.InetAddressUtil; import org.archive.wayback.WaybackConstants; -import org.archive.wayback.core.Timestamp; import org.archive.wayback.core.WaybackRequest; import org.archive.wayback.exception.BadQueryException; import org.archive.wayback.requestparser.BaseRequestParser; @@ -96,15 +95,6 @@ wbRequest.put(WaybackConstants.REQUEST_TYPE, WaybackConstants.REQUEST_REPLAY_QUERY); - // Get the id from the request. If no id, use the ip-address instead. - // Then get the timestamp (or rather datestr) matching this id. - String id = httpRequest.getHeader("Proxy-Id"); - if (id == null) - id = httpRequest.getRemoteAddr(); - wbRequest.put(WaybackConstants.REQUEST_EXACT_DATE, Timestamp - .getTimestampForId(httpRequest.getContextPath(), id)); - wbRequest.fixup(httpRequest); - return wbRequest; } public List<String> getLocalhostNames() { Modified: trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/proxy/ProxyRequestParser.java =================================================================== --- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/proxy/ProxyRequestParser.java 2008-02-20 23:50:37 UTC (rev 2202) +++ trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/proxy/ProxyRequestParser.java 2008-02-21 01:55:36 UTC (rev 2203) @@ -26,10 +26,17 @@ import java.util.List; +import javax.servlet.http.HttpServletRequest; + import org.archive.wayback.RequestParser; +import org.archive.wayback.WaybackConstants; +import org.archive.wayback.core.Timestamp; +import org.archive.wayback.core.WaybackRequest; +import org.archive.wayback.exception.BadQueryException; import org.archive.wayback.requestparser.CompositeRequestParser; import org.archive.wayback.requestparser.FormRequestParser; import org.archive.wayback.requestparser.OpenSearchRequestParser; +import org.archive.wayback.webapp.AccessPoint; /** * @@ -54,4 +61,20 @@ public void setLocalhostNames(List<String> localhostNames) { prrp.setLocalhostNames(localhostNames); } + public WaybackRequest parse(HttpServletRequest httpRequest, + AccessPoint wbContext) throws BadQueryException { + + WaybackRequest wbRequest = super.parse(httpRequest, wbContext); + if (wbRequest != null) { + // Get the id from the request. If no id, use the ip-address instead. + // Then get the timestamp (or rather datestr) matching this id. + String id = httpRequest.getHeader("Proxy-Id"); + if (id == null) + id = httpRequest.getRemoteAddr(); + wbRequest.put(WaybackConstants.REQUEST_EXACT_DATE, Timestamp + .getTimestampForId(httpRequest.getContextPath(), id)); + wbRequest.fixup(httpRequest); + } + return wbRequest; + } } This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
Revision: 2202 http://archive-access.svn.sourceforge.net/archive-access/?rev=2202&view=rev Author: bradtofel Date: 2008-02-20 15:50:37 -0800 (Wed, 20 Feb 2008) Log Message: ----------- FEATURE: added getters/setters for the various error .jsp handlers. Modified Paths: -------------- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/replay/BaseReplayDispatcher.java Modified: trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/replay/BaseReplayDispatcher.java =================================================================== --- trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/replay/BaseReplayDispatcher.java 2008-02-16 01:09:41 UTC (rev 2201) +++ trunk/archive-access/projects/wayback/wayback-core/src/main/java/org/archive/wayback/replay/BaseReplayDispatcher.java 2008-02-20 23:50:37 UTC (rev 2202) @@ -175,4 +175,36 @@ renderException(httpRequest, httpResponse, wbRequest, e); } } + + public String getErrorJsp() { + return errorJsp; + } + + public void setErrorJsp(String errorJsp) { + this.errorJsp = errorJsp; + } + + public String getImageErrorJsp() { + return imageErrorJsp; + } + + public void setImageErrorJsp(String imageErrorJsp) { + this.imageErrorJsp = imageErrorJsp; + } + + public String getJavascriptErrorJsp() { + return javascriptErrorJsp; + } + + public void setJavascriptErrorJsp(String javascriptErrorJsp) { + this.javascriptErrorJsp = javascriptErrorJsp; + } + + public String getCssErrorJsp() { + return cssErrorJsp; + } + + public void setCssErrorJsp(String cssErrorJsp) { + this.cssErrorJsp = cssErrorJsp; + } } This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <al...@us...> - 2008-02-16 01:09:42
|
Revision: 2201 http://archive-access.svn.sourceforge.net/archive-access/?rev=2201&view=rev Author: alexoz Date: 2008-02-15 17:09:41 -0800 (Fri, 15 Feb 2008) Log Message: ----------- Added in-progress access control administration UI. * NewSurtTokenizer.java, NewSurtTokenizerTest.java Added an iterator-based SURT tokenizer that breaks a SURT into pieces and returns each piece one by one. It also leaves commas at the end of hostname components. * jscalendar-1.0 Added javascript calendar library (LGPL). Modified Paths: -------------- trunk/archive-access/projects/access-control/oracle/pom.xml trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/exclusions-oracle-servlet.xml trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/urlrewrite.xml trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/web.xml trunk/archive-access/projects/access-control/pom.xml Added Paths: ----------- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/surt/NewSurtTokenizer.java trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/surt/NewSurtTokenizerTest.java trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/webui/ trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/webui/AdminController.java trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/webui/DisplayRule.java trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/webui/SurtNode.java trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/ trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/footer.inc trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/header.inc trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/index.jsp trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/list_rules.jsp trunk/archive-access/projects/access-control/oracle/src/main/webapp/css/ trunk/archive-access/projects/access-control/oracle/src/main/webapp/css/admin.css trunk/archive-access/projects/access-control/oracle/src/main/webapp/js/ trunk/archive-access/projects/access-control/oracle/src/main/webapp/js/calendar.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/ trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/ChangeLog trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/README trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/bugtest-hidden-selects.html trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/calendar-blue.css trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/calendar-blue2.css trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/calendar-brown.css trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/calendar-green.css trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/calendar-setup.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/calendar-setup_stripped.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/calendar-system.css trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/calendar-tas.css trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/calendar-win2k-1.css trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/calendar-win2k-2.css trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/calendar-win2k-cold-1.css trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/calendar-win2k-cold-2.css trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/calendar.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/calendar.php trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/calendar_stripped.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/dayinfo.html trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/doc/ trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/doc/html/ trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/doc/html/field-button.jpg trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/doc/html/reference-Z-S.css trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/doc/html/reference.css trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/doc/html/reference.html trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/doc/reference.pdf trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/img.gif trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/index.html trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/ trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-af.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-al.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-bg.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-big5-utf8.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-big5.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-br.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-ca.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-cs-utf8.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-cs-win.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-da.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-de.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-du.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-el.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-en.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-es.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-fi.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-fr.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-he-utf8.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-hr-utf8.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-hr.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-hu.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-it.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-jp.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-ko-utf8.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-ko.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-lt-utf8.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-lt.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-lv.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-nl.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-no.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-pl-utf8.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-pl.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-pt.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-ro.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-ru.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-ru_win_.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-si.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-sk.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-sp.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-sv.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-tr.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/calendar-zh.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/lang/cn_utf8.js trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/menuarrow.gif trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/menuarrow2.gif trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/multiple-dates.html trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/release-notes.html trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/simple-1.html trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/simple-2.html trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/simple-3.html trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/skins/ trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/skins/aqua/ trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/skins/aqua/active-bg.gif trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/skins/aqua/dark-bg.gif trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/skins/aqua/hover-bg.gif trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/skins/aqua/menuarrow.gif trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/skins/aqua/normal-bg.gif trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/skins/aqua/rowhover-bg.gif trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/skins/aqua/status-bg.gif trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/skins/aqua/theme.css trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/skins/aqua/title-bg.gif trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/skins/aqua/today-bg.gif trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/test-position.html trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/test.php trunk/archive-access/projects/access-control/oracle/src/test/java/org/archive/accesscontrol/webui/ trunk/archive-access/projects/access-control/oracle/src/test/java/org/archive/accesscontrol/webui/SurtNodeTest.java Added: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/surt/NewSurtTokenizer.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/surt/NewSurtTokenizer.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/surt/NewSurtTokenizer.java 2008-02-16 01:09:41 UTC (rev 2201) @@ -0,0 +1,138 @@ +package org.archive.surt; + +import java.util.ArrayList; +import java.util.Iterator; +import java.util.List; +import java.util.regex.Pattern; + +/** + * The new SURT tokenizer breaks a SURT up into tokens. + * + * For example "(org,archive,www,)/path/file.html?query#anchor" is broken up into: + * + * ["(" + * "org," + * "archive," + * "www," + * ")/" + * "path/" + * "file.html" + * "?query" + * "#anchor"] + * + * @author aosborne + * + */ +public class NewSurtTokenizer implements Iterable<String> { + private String surt; + private int endOfAuthority; + private int endOfPath; + private int surtLength; + + public NewSurtTokenizer(String surt) { + super(); + this.surt = surt; + surtLength = surt.length(); + endOfAuthority = surt.indexOf(')'); + if (endOfAuthority == -1) { + endOfAuthority = surtLength; + } + + int hash = surt.indexOf('#'); + int question = surt.indexOf('?'); + if (hash == -1) { + endOfPath = question; + } else if (question == -1) { + endOfPath = hash; + } else { + endOfPath = hash < question ? hash : question; + } + if (endOfPath == -1) { + endOfPath = surtLength; + } + + } + + private class NewSurtTokenizerIterator implements Iterator<String> { + int pos = 0; + + public boolean hasNext() { + return pos < surtLength; + } + + private int nextPieceEnd() { + // ROOT: "(..." + if (pos == 0) { + return 1; // "(" + } + // Host components: "foo,..." + if (pos < endOfAuthority || endOfAuthority == -1) { + int endOfHostComponent = surt.indexOf(',', pos); + if (endOfHostComponent == -1) { + return surtLength; + } else { + return endOfHostComponent + 1; + } + } + + // Host index: ")/..." + if (pos == endOfAuthority) { + return pos + 2; + } + + // Path segments: "directory/" + if (pos < endOfPath || endOfPath == -1) { + int endOfPathSegment = surt.indexOf('/', pos); + if (endOfPathSegment < endOfPath && endOfPathSegment != -1) { + return endOfPathSegment + 1; + } else if (endOfPath != -1) { // file: "hello.html" + return endOfPath; + } else { + return surtLength; + } + } + + // Query string + if (surt.charAt(pos) == '?') { + int endOfQuery = surt.indexOf('#'); + if (endOfQuery != -1) { + return endOfQuery; + } else { + return surtLength; + } + } + + // Anchor "#boo" + return surtLength; + } + + public String next() { + int pieceEnd = nextPieceEnd(); + String piece = surt.substring(pos, pieceEnd); + pos = pieceEnd; + return piece; + } + + public void remove() { + // TODO Auto-generated method stub + + } + + } + + public Iterator<String> iterator() { + return new NewSurtTokenizerIterator(); + } + + public List<String> toList() { + List<String> list = new ArrayList<String>(); + for (String piece: this) { + list.add(piece); + } + return list; + } + public String[] toArray() { + return (String[]) toList().toArray(); + } + +} Added: trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/surt/NewSurtTokenizerTest.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/surt/NewSurtTokenizerTest.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/surt/NewSurtTokenizerTest.java 2008-02-16 01:09:41 UTC (rev 2201) @@ -0,0 +1,165 @@ +package org.archive.surt; + +import java.util.Iterator; + +import junit.framework.TestCase; + +public class NewSurtTokenizerTest extends TestCase { + + public void testRoot() { + NewSurtTokenizer tok = new NewSurtTokenizer("("); + Iterator<String> it = tok.iterator(); + + assertEquals("(", it.next()); + assertFalse(it.hasNext()); + + } + + public void testOneSeg() { + NewSurtTokenizer tok = new NewSurtTokenizer("(org"); + Iterator<String> it = tok.iterator(); + + assertEquals("(", it.next()); + assertEquals("org", it.next()); + assertFalse(it.hasNext()); + } + + public void testOneSegComma() { + NewSurtTokenizer tok = new NewSurtTokenizer("(org,"); + Iterator<String> it = tok.iterator(); + + assertEquals("(", it.next()); + assertEquals("org,", it.next()); + assertFalse(it.hasNext()); + } + + public void testFewSegs() { + NewSurtTokenizer tok = new NewSurtTokenizer("(org,archive,www"); + Iterator<String> it = tok.iterator(); + + assertEquals("(", it.next()); + assertEquals("org,", it.next()); + assertEquals("archive,", it.next()); + assertEquals("www", it.next()); + assertFalse(it.hasNext()); + } + + public void testFewSegsComma() { + NewSurtTokenizer tok = new NewSurtTokenizer("(org,archive,www,"); + Iterator<String> it = tok.iterator(); + + assertEquals("(", it.next()); + assertEquals("org,", it.next()); + assertEquals("archive,", it.next()); + assertEquals("www,", it.next()); + assertFalse(it.hasNext()); + } + + public void testIndex() { + NewSurtTokenizer tok = new NewSurtTokenizer("(org,archive,www,)/"); + Iterator<String> it = tok.iterator(); + + assertEquals("(", it.next()); + assertEquals("org,", it.next()); + assertEquals("archive,", it.next()); + assertEquals("www,", it.next()); + assertEquals(")/", it.next()); + assertFalse(it.hasNext()); + } + + public void testPage() { + NewSurtTokenizer tok = new NewSurtTokenizer("(org,archive,www,)/about.html"); + Iterator<String> it = tok.iterator(); + + assertEquals("(", it.next()); + assertEquals("org,", it.next()); + assertEquals("archive,", it.next()); + assertEquals("www,", it.next()); + assertEquals(")/", it.next()); + assertEquals("about.html", it.next()); + assertFalse(it.hasNext()); + } + + public void testPath() { + NewSurtTokenizer tok = new NewSurtTokenizer("(org,archive,www,)/one/two/"); + Iterator<String> it = tok.iterator(); + + assertEquals("(", it.next()); + assertEquals("org,", it.next()); + assertEquals("archive,", it.next()); + assertEquals("www,", it.next()); + assertEquals(")/", it.next()); + assertEquals("one/", it.next()); + assertEquals("two/", it.next()); + assertFalse(it.hasNext()); + } + + + public void testPathPage() { + NewSurtTokenizer tok = new NewSurtTokenizer("(org,archive,www,)/one/two/about.html"); + Iterator<String> it = tok.iterator(); + + assertEquals("(", it.next()); + assertEquals("org,", it.next()); + assertEquals("archive,", it.next()); + assertEquals("www,", it.next()); + assertEquals(")/", it.next()); + assertEquals("one/", it.next()); + assertEquals("two/", it.next()); + assertEquals("about.html", it.next()); + assertFalse(it.hasNext()); + } + + + public void testQuery() { + NewSurtTokenizer tok = new NewSurtTokenizer("(org,archive,www,)/one/two/about.html?yo=hey&hi"); + Iterator<String> it = tok.iterator(); + + assertEquals("(", it.next()); + assertEquals("org,", it.next()); + assertEquals("archive,", it.next()); + assertEquals("www,", it.next()); + assertEquals(")/", it.next()); + assertEquals("one/", it.next()); + assertEquals("two/", it.next()); + assertEquals("about.html", it.next()); + assertEquals("?yo=hey&hi", it.next()); + assertFalse(it.hasNext()); + } + + + public void testAnchor() { + NewSurtTokenizer tok = new NewSurtTokenizer("(org,archive,www,)/one/two/about.html#fishing/,)fish(?moo"); + Iterator<String> it = tok.iterator(); + + assertEquals("(", it.next()); + assertEquals("org,", it.next()); + assertEquals("archive,", it.next()); + assertEquals("www,", it.next()); + assertEquals(")/", it.next()); + assertEquals("one/", it.next()); + assertEquals("two/", it.next()); + assertEquals("about.html", it.next()); + assertEquals("#fishing/,)fish(?moo", it.next()); + assertFalse(it.hasNext()); + } + + public void testQueryAnchor() { + NewSurtTokenizer tok = new NewSurtTokenizer("(org,archive,www,)/fishes/pinky.html?moo=yes&bar=12#423"); + Iterator<String> it = tok.iterator(); + + assertEquals("(", it.next()); + assertEquals("org,", it.next()); + assertEquals("archive,", it.next()); + assertEquals("www,", it.next()); + assertEquals(")/", it.next()); + assertEquals("fishes/", it.next()); + assertEquals("pinky.html", it.next()); + assertEquals("?moo=yes&bar=12", it.next()); + assertEquals("#423", it.next()); + assertFalse(it.hasNext()); + + } + + +} Modified: trunk/archive-access/projects/access-control/oracle/pom.xml =================================================================== --- trunk/archive-access/projects/access-control/oracle/pom.xml 2008-02-13 00:50:02 UTC (rev 2200) +++ trunk/archive-access/projects/access-control/oracle/pom.xml 2008-02-16 01:09:41 UTC (rev 2201) @@ -1,18 +1,16 @@ <?xml version="1.0" encoding="UTF-8"?><project> + <parent> + <artifactId>access-control</artifactId> + <groupId>org.archive</groupId> + <version>0.0.1-SNAPSHOT</version> + </parent> <modelVersion>4.0.0</modelVersion> <groupId>org.archive.access-control</groupId> <artifactId>oracle</artifactId> <packaging>war</packaging> + <name>Access-Control: Oracle Webapp</name> <version>0.0.1-SNAPSHOT</version> <description></description> - <name>Access-Control: Oracle Webapp</name> - - <parent> - <groupId>org.archive</groupId> - <artifactId>access-control</artifactId> - <version>0.0.1-SNAPSHOT</version> - </parent> - <build> <extensions> <extension> @@ -73,15 +71,20 @@ <dependencies> <dependency> <groupId>javax.servlet</groupId> + <artifactId>jstl</artifactId> + <version>1.1.2</version> + </dependency> + <dependency> + <groupId>javax.servlet</groupId> <artifactId>servlet-api</artifactId> <version>2.4</version> </dependency> - <dependency> - <groupId>org.archive.access-control</groupId> - <artifactId>access-control</artifactId> - <version>0.0.1-SNAPSHOT</version> - </dependency> <dependency> + <groupId>org.archive.access-control</groupId> + <artifactId>access-control</artifactId> + <version>0.0.1-SNAPSHOT</version> + </dependency> + <dependency> <groupId>org.springframework</groupId> <artifactId>spring-webmvc</artifactId> <version>2.5.1</version> @@ -112,10 +115,10 @@ <version>8.2-504.jdbc3</version> </dependency> <dependency> - <groupId>hsqldb</groupId> - <artifactId>hsqldb</artifactId> - <version>1.8.0.7</version> - </dependency> + <groupId>hsqldb</groupId> + <artifactId>hsqldb</artifactId> + <version>1.8.0.7</version> + </dependency> <dependency> <groupId>com.thoughtworks.xstream</groupId> <artifactId>xstream</artifactId> @@ -146,5 +149,15 @@ <artifactId>commons-dbcp</artifactId> <version>1.2.2</version> </dependency> + <dependency> + <groupId>org.apache.velocity</groupId> + <artifactId>velocity</artifactId> + <version>1.5</version> + </dependency> + <dependency> + <groupId>taglibs</groupId> + <artifactId>standard</artifactId> + <version>1.1.2</version> + </dependency> </dependencies> -</project> +</project> \ No newline at end of file Added: trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/webui/AdminController.java =================================================================== --- trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/webui/AdminController.java (rev 0) +++ trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/webui/AdminController.java 2008-02-16 01:09:41 UTC (rev 2201) @@ -0,0 +1,134 @@ +package org.archive.accesscontrol.webui; + +import java.net.URLEncoder; +import java.text.DateFormat; +import java.text.ParseException; +import java.text.SimpleDateFormat; +import java.util.ArrayList; +import java.util.Collections; +import java.util.Date; +import java.util.HashMap; +import java.util.LinkedList; +import java.util.List; +import java.util.Map; +import java.util.TimeZone; + +import javax.servlet.http.HttpServletRequest; +import javax.servlet.http.HttpServletResponse; + +import org.apache.commons.httpclient.URIException; +import org.archive.accesscontrol.model.HibernateRuleDao; +import org.archive.accesscontrol.model.Rule; +import org.archive.accesscontrol.model.RuleSet; +import org.archive.surt.SURTTokenizer; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.web.servlet.ModelAndView; +import org.springframework.web.servlet.mvc.AbstractController; + +public class AdminController extends AbstractController { + private HibernateRuleDao ruleDao; + private static final DateFormat dateFormatter = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"); + static { + dateFormatter.setTimeZone(TimeZone.getTimeZone("GMT")); + } + + @Autowired + public AdminController(HibernateRuleDao ruleDao) { + this.ruleDao = ruleDao; + } + + protected ModelAndView ruleList(String surt, HttpServletRequest request, + HttpServletResponse response) throws Exception { + Long editingRuleId = null; + if (request.getParameter("edit") != null) { + try { + editingRuleId = Long.decode(request.getParameter("edit")); + } catch (NumberFormatException e) { + } + } + return ruleList(surt, editingRuleId, request, response); + } + + protected ModelAndView ruleList(String surt, Long editingRuleId, HttpServletRequest request, + HttpServletResponse response) throws Exception { + Map<String, Object> model = new HashMap<String, Object>(); + RuleSet rules = ruleDao.getRuleTree(surt); + ArrayList<DisplayRule> ruleList = new ArrayList<DisplayRule>(); + + for (Rule rule: rules) { + int comparison = rule.getSurt().compareTo(surt); + if (comparison <= 0) { + DisplayRule displayRule = new DisplayRule(rule, comparison != 0); + displayRule.setEditing(rule.getId().equals(editingRuleId)); + ruleList.add(displayRule); + } else { + // lowerRules.add(rule); + } + } + Collections.sort(ruleList); + + model.put("rules", ruleList); + model.put("surt", surt); + model.put("encodedSurt", URLEncoder.encode(surt, "utf-8")); + model.put("breadcrumbs", SurtNode.nodesFromSurt(surt)); + model.put("editingRuleId", request.getParameter("edit")); + return new ModelAndView("list_rules", model); + } + + @Override + protected ModelAndView handleRequestInternal(HttpServletRequest request, + HttpServletResponse response) throws Exception { + if (request.getParameter("saveRule") != null) { + return saveRule(request, response); + } + + + String surt = (String) request.getAttribute("id"); + if (surt == null) { + surt = request.getParameter("surt"); + } + + if (surt != null) { + return ruleList(surt, request, response); + } + + return new ModelAndView("index"); + } + + private ModelAndView saveRule(HttpServletRequest request, + HttpServletResponse response) throws Exception { + String surt = request.getParameter("surt"); + Long ruleId = Long.decode(request.getParameter("edit")); + Rule rule = ruleDao.getRule(ruleId); + rule.setSurt(surt); + rule.setPolicy(request.getParameter("policy")); + rule.setWho(request.getParameter("who")); + rule.setCaptureStart(parseDate(request.getParameter("captureStart"))); + rule.setCaptureEnd(parseDate(request.getParameter("captureEnd"))); + rule.setRetrievalStart(parseDate(request.getParameter("retrievalStart"))); + rule.setRetrievalEnd(parseDate(request.getParameter("retrievalEnd"))); + rule.setSecondsSinceCapture(parseInteger(request.getParameter("secondsSinceCapture"))); + ruleDao.saveRule(rule); + + response.setHeader("Location", request.getContextPath() + "/admin?surt=" + URLEncoder.encode(surt, "UTF-8")); + response.setStatus(302); + return null; + } + + private Date parseDate(String s) { + try { + return dateFormatter.parse(s); + } catch (ParseException e) { + return null; + } + } + + private Integer parseInteger(String s) { + try { + return Integer.decode(s); + } catch (NumberFormatException e) { + return null; + } + } + +} Added: trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/webui/DisplayRule.java =================================================================== --- trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/webui/DisplayRule.java (rev 0) +++ trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/webui/DisplayRule.java 2008-02-16 01:09:41 UTC (rev 2201) @@ -0,0 +1,56 @@ +package org.archive.accesscontrol.webui; + +import java.io.UnsupportedEncodingException; +import java.net.URLEncoder; + +import org.archive.accesscontrol.model.Rule; + +/** + * Wrapper for Rule that holds extra fields and methods useful for rendering a rule. + * @author ato + * + */ +public class DisplayRule implements Comparable<DisplayRule> { + private Rule rule; + private boolean inherited; + private boolean editing; + + public DisplayRule(Rule rule, boolean inherited) { + super(); + this.rule = rule; + this.inherited = inherited; + } + + public Rule getRule() { + return rule; + } + + public void setRule(Rule rule) { + this.rule = rule; + } + + public int compareTo(DisplayRule o) { + return getRule().compareTo(o.getRule()); + } + + public boolean isInherited() { + return inherited; + } + + public void setInherited(boolean inherited) { + this.inherited = inherited; + } + + public boolean isEditing() { + return editing; + } + + public void setEditing(boolean editing) { + this.editing = editing; + } + + public String getEncodedSurt() throws UnsupportedEncodingException { + return URLEncoder.encode(rule.getSurt(), "utf-8"); + } + +} Added: trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/webui/SurtNode.java =================================================================== --- trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/webui/SurtNode.java (rev 0) +++ trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/webui/SurtNode.java 2008-02-16 01:09:41 UTC (rev 2201) @@ -0,0 +1,51 @@ +package org.archive.accesscontrol.webui; + +import java.util.ArrayList; +import java.util.List; + +import org.archive.surt.NewSurtTokenizer; + +/** + * A node in the SURT tree. + * + * @author aosborne + * + */ +public class SurtNode { + private String name; + private String surt; + + public SurtNode(String name, String surt) { + super(); + this.name = name; + this.surt = surt; + } + public String getName() { + return name; + } + public String getSurt() { + return surt; + } + + /** + * Return a list of the elements in a given SURT. + * + * For example for "(org,archive," we return: + * + * [new SurtNode("(", "("), + * new SurtNode("org,", "(org"), + * new SurtNode("archive,", "archive,")] + * + * @param surt + * @return + */ + public static List<SurtNode> nodesFromSurt(String surt) { + List<SurtNode> list = new ArrayList<SurtNode>(); + String running = ""; + for (String token: new NewSurtTokenizer(surt)) { + running += token; + list.add(new SurtNode(token, running)); + } + return list; + } +} Modified: trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/exclusions-oracle-servlet.xml =================================================================== --- trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/exclusions-oracle-servlet.xml 2008-02-13 00:50:02 UTC (rev 2200) +++ trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/exclusions-oracle-servlet.xml 2008-02-16 01:09:41 UTC (rev 2201) @@ -16,10 +16,13 @@ class="org.springframework.web.servlet.handler.BeanNameUrlHandlerMapping" /> <bean name="/rules" - class="org.archive.accesscontrol.oracle.RulesController"> - + class="org.archive.accesscontrol.oracle.RulesController"> </bean> + <bean name="/admin" + class="org.archive.accesscontrol.webui.AdminController"> + </bean> + <!-- MultipartResolver for parsing file uploads, implementation for Commons FileUpload --> <!-- <bean id="multipartResolver" class="org.springframework.web.multipart.commons.CommonsMultipartResolver"/> @@ -59,6 +62,7 @@ <bean id="viewResolver" class="org.springframework.web.servlet.view.InternalResourceViewResolver"> + <property name="viewClass" value="org.springframework.web.servlet.view.JstlView"/> <property name="prefix" value="/WEB-INF/views/" /> <property name="suffix" value=".jsp" /> </bean> Modified: trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/urlrewrite.xml =================================================================== --- trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/urlrewrite.xml 2008-02-13 00:50:02 UTC (rev 2200) +++ trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/urlrewrite.xml 2008-02-16 01:09:41 UTC (rev 2201) @@ -12,7 +12,7 @@ attribute. This allows AutoFormatView to render the appropriate type of output. </note> - <from>^/(.*)\.(\w+)$</from> + <from>^/(rules.*)\.(\w+)$</from> <set name="format">$2</set> <to last="false">/$1</to> </rule> @@ -28,4 +28,11 @@ <set name="id">$1</set> <to>/rules</to> </rule> + + <rule> + <from>^/admin/(.*)$</from> + <set name="id">$1</set> + <to>/admin</to> + </rule> + </urlrewrite> Added: trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/footer.inc =================================================================== --- trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/footer.inc (rev 0) +++ trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/footer.inc 2008-02-16 01:09:41 UTC (rev 2201) @@ -0,0 +1,2 @@ +</body> +</html> \ No newline at end of file Added: trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/header.inc =================================================================== --- trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/header.inc (rev 0) +++ trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/header.inc 2008-02-16 01:09:41 UTC (rev 2201) @@ -0,0 +1,22 @@ +<?xml version="1.0" encoding="UTF-8"?> +<%@ page contentType="text/html" %> +<%@ page pageEncoding="utf-8" %> +<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %> +<%@ taglib prefix="fmt" uri="http://java.sun.com/jsp/jstl/fmt" %> +<!DOCTYPE html + PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" + "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> +<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> +<head> + <title>Archive Access Control Admin</title> + <link rel="stylesheet" type="text/css" href="<c:url value="/css/admin.css"/>" /> + + <style type="text/css">@import url(<c:url value="/jscalendar-1.0/calendar-win2k-1.css"/>);</style> + <script type="text/javascript" src="<c:url value="/jscalendar-1.0/calendar.js"/>"></script> + <script type="text/javascript" src="<c:url value="/jscalendar-1.0/lang/calendar-en.js"/>"></script> + <script type="text/javascript" src="<c:url value="/jscalendar-1.0/calendar-setup.js"/>"></script> + <script type="text/javascript" src="<c:url value="/js/calendar.js"/>"></script> + +</head> +<body> + <h1>Archive Access Control</h1> \ No newline at end of file Added: trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/index.jsp =================================================================== --- trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/index.jsp (rev 0) +++ trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/index.jsp 2008-02-16 01:09:41 UTC (rev 2201) @@ -0,0 +1,17 @@ +<%@ include file="header.inc"%> + +<form action="<c:url value="/admin"/>" method="GET"><input size="50" name="surt" + value="(org,archive,www,)/foo" /> <input type="submit" value="Go!" /> +</form> + +<h2 class="breadcrumb"><a href="admin?surt=(org">(org</a>,<a + href="admin?surt=(org,archive">archive</a></h2> + +<ul> + <li><a href="admin?surt=(org,archive,audio">audio</a></li> + <li>web</li> + <li>webteam</li> + <li>www</li> +</ul> + +<%@ include file="footer.inc"%> Added: trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/list_rules.jsp =================================================================== --- trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/list_rules.jsp (rev 0) +++ trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/views/list_rules.jsp 2008-02-16 01:09:41 UTC (rev 2201) @@ -0,0 +1,103 @@ +<%@ include file="header.inc"%> + +<form action="<c:url value="/admin"/>" method="GET" id="navForm"> +<label for="surtNavBox">SURT or URL:</label> <input size="50" + name="surt" value="<c:out value="${surt}"/>" id="surtNavBox" /> <input + type="submit" value="Go!" /></form> + +<div id="breadcrumbsContainer"> +<ul id="breadcrumbs"> + <c:forEach var="node" items="${breadcrumbs}"> + <li><a href="<c:url value="/admin?surt=${node.surt}" />" + title="<c:out value="${node.surt}" />"><c:out value="${node.name}" /></a></li> + </c:forEach> +</ul> +</div> + +<table> + <thead> + <tr> + <th>SURT</th> + <th>Capture date</th> + <th>Retrieval date</th> + <th>Group</th> + <th>Policy</th> + <th></th> + </tr> + </thead> + <tbody> + <c:forEach var="rule" items="${rules }"> + <c:choose> + <c:when test="${rule.editing }"> + <tr class="rule editing"> + <td colspan="6"> + <form action="<c:url value="/admin"/>" method="post"><input + type="hidden" name="edit" + value="<c:out value="${rule.rule.id}"/>" /> + <fieldset> + <p><label for="surt">SURT:</label> <input name="surt" + id="surt" value="<c:out value="${rule.rule.surt}"/>" /></p> + + <p><label for="who">Group:</label> <input name="who" id="who" + value="<c:out value="${rule.rule.who }"/>" /></p> + + <p><label for="captureStart">Captured from</label> <input + name="captureStart" id="captureStart" + value="<fmt:formatDate value="${rule.rule.captureStart }" type="both" pattern="yyyy-MM-dd HH:mm:ss" />" /> + to <input name="captureEnd" id="captureEnd" + value="<fmt:formatDate value="${rule.rule.captureEnd }" type="both" pattern="yyyy-MM-dd HH:mm:ss" />" /></p> + + <p><label for="captureStart">Retrieved from</label> <input + name="retrievalStart" id="retrievalStart" + value="<fmt:formatDate value="${rule.rule.retrievalStart }" type="both" pattern="yyyy-MM-dd HH:mm:ss"/>" /> + to <input name="retrievalEnd" id="retrievalEnd" + value="<fmt:formatDate value="${rule.rule.retrievalEnd }" type="both" pattern="yyyy-MM-dd HH:mm:ss"/>" /></p> + + <p><label for="secondsSinceCapture">Seconds since + capture (embargo):</label> <input name="secondsSinceCapture" + id="secondsSinceCapture" + value="<c:out value="${rule.rule.secondsSinceCapture }"/>" /></p> + + <p><label for="policy">Policy:</label> <input name="policy" + id="policy" value="<c:out value="${rule.rule.policy}"/>" /></p> + <input type="submit" value="Save" name="saveRule" /></fieldset> + </form> + <script type="text/javascript"> + setupDateRangeCalendars("captureStart", "captureEnd"); + setupDateRangeCalendars("retrievalStart", "retrievalEnd"); + </script></td> + </tr> + </c:when> + <c:otherwise> + <tr class="rule<c:if test="${rule.inherited }"> inherited</c:if>" + id="rule_<c:out value="${rule.rule.id}"/>"> + <td><c:out value="${rule.rule.surt}" /></td> + <td><span class="date" + title="<fmt:formatDate value="${rule.rule.captureStart }" type="both" pattern="yyyy-MM-dd HH:mm:ss" />"><fmt:formatDate + value="${rule.rule.captureStart }" type="both" + pattern="yyyy-MM-dd" /></span> to <span class="date" + title="<fmt:formatDate value="${rule.rule.captureEnd }" type="both" pattern="yyyy-MM-dd HH:mm:ss" />"><fmt:formatDate + value="${rule.rule.captureEnd }" type="both" pattern="yyyy-MM-dd" /></span></td> + <td><span class="date" + title="<fmt:formatDate value="${rule.rule.retrievalStart }" type="both" pattern="yyyy-MM-dd HH:mm:ss" />"><fmt:formatDate + value="${rule.rule.retrievalStart }" type="both" + pattern="yyyy-MM-dd" /></span> to <span class="date" + title="<fmt:formatDate value="${rule.rule.retrievalEnd }" type="both" pattern="yyyy-MM-dd HH:mm:ss" />"><fmt:formatDate + value="${rule.rule.retrievalEnd }" type="both" + pattern="yyyy-MM-dd" /></span></td> + <td><c:out value="${rule.rule.who}" /></td> + <td><c:out value="${rule.rule.policy}" /></td> + <td><a + href="<c:url value="/admin?surt=${rule.encodedSurt}&edit=${rule.rule.id}"/>">Edit</a></td> + </tr> + </c:otherwise> + </c:choose> + </c:forEach> + <tr> + <td colspan="6" class="newrule"><a + href="<c:url value="/admin?surt=${encodedSurt}&edit=new"/>">Add + new rule</a></td> + </tr> + </tbody> +</table> +<%@ include file="footer.inc"%> \ No newline at end of file Modified: trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/web.xml =================================================================== --- trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/web.xml 2008-02-13 00:50:02 UTC (rev 2200) +++ trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/web.xml 2008-02-16 01:09:41 UTC (rev 2201) @@ -61,10 +61,13 @@ <servlet-mapping> <servlet-name>exclusions-oracle</servlet-name> - <url-pattern>/</url-pattern> + <url-pattern>/rules</url-pattern> </servlet-mapping> + <servlet-mapping> + <servlet-name>exclusions-oracle</servlet-name> + <url-pattern>/admin</url-pattern> + </servlet-mapping> - <welcome-file-list> <welcome-file>index.html</welcome-file> </welcome-file-list> Added: trunk/archive-access/projects/access-control/oracle/src/main/webapp/css/admin.css =================================================================== --- trunk/archive-access/projects/access-control/oracle/src/main/webapp/css/admin.css (rev 0) +++ trunk/archive-access/projects/access-control/oracle/src/main/webapp/css/admin.css 2008-02-16 01:09:41 UTC (rev 2201) @@ -0,0 +1,65 @@ +.inherited { + color: #999; +} + +td { + background: #eee; + padding: 4px; +} + +span.date { + color: #006; + font-weight: bold; +} + +th { + text-align: left; +} + +td.newrule { + text-align: right; +} + +h1 { + padding-left: 40px; + background-image: url(../images/greylock.gif); + background-repeat: no-repeat; +} + +ul#breadcrumbs { + margin: 1px; + padding: 0; + font-family: Monaco, Courier, monospace; + white-space: nowrap; +} + +div#breadcrumbsContainer { + border-bottom: solid 1px #999; + margin-bottom: 2em; + padding-top: 2em; + padding-left: 1em; + +} + +ul#breadcrumbs li { + display: inline; + list-style-type: none; +} + +ul#breadcrumbs a { + background: #eef; + text-decoration: none; + padding: 2px 2px; + border-left: 1px solid #999; + border-right: 1px solid #999; + border-top: 1px solid #999; + border-bottom: 1px solid #eef; + +} + +ul#breadcrumbs a:hover { + background: #ccf; +} +table { + clear: both; +} \ No newline at end of file Added: trunk/archive-access/projects/access-control/oracle/src/main/webapp/js/calendar.js =================================================================== --- trunk/archive-access/projects/access-control/oracle/src/main/webapp/js/calendar.js (rev 0) +++ trunk/archive-access/projects/access-control/oracle/src/main/webapp/js/calendar.js 2008-02-16 01:09:41 UTC (rev 2201) @@ -0,0 +1,33 @@ +function ensureSmaller(smaller,larger,change) { + if(smaller.value != "" && larger.value != "") { + if(smaller.value > larger.value) { + if(smaller == change) { + smaller.value = larger.value; + } else { + larger.value = smaller.value; + } + } + } +} +function mySetupCal(id,fn) { + var dateFormatString = "%Y-%m-%d %H:%M:%S"; + Calendar.setup( + { + inputField : id, + ifFormat : dateFormatString, + showsTime : true, + timeFormat : "24", + onUpdate : fn, + }); +} + +function setupDateRangeCalendars(startField, endField) { + var capStart = document.getElementById(startField); + var capEnd = document.getElementById(endField); + + function ensureCapStart(cal) { ensureSmaller(capStart,capEnd,capEnd) } + function ensureCapEnd(cal) { ensureSmaller(capStart,capEnd,capStart) } + + mySetupCal(startField,ensureCapStart); + mySetupCal(endField,ensureCapEnd); +} \ No newline at end of file Added: trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/ChangeLog =================================================================== --- trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/ChangeLog (rev 0) +++ trunk/archive-access/projects/access-control/oracle/src/main/webapp/jscalendar-1.0/ChangeLog 2008-02-16 01:09:41 UTC (rev 2201) @@ -0,0 +1,761 @@ +2005-03-07 Mihai Bazon <mih...@ya...> + + * skins/aqua/theme.css: *** empty log message *** + + * release-notes.html: updated release notes + + * calendar-setup.js: + use a better approach to initialize the calendar--don't call _init twice, + it's the most time consuming function in the calendar. Instead, determine + the date beforehand if possible and pass it to the calendar at constructor. + + * calendar.js: + avoid keyboard operation when 'multiple dates' is set (very buggy for now) + + * calendar.js: + fixed keyboard handling problems: now it works fine when "showsOtherMonths" + is passed; it also seems to be fine with disabled dates (won't normally + allow selection)--however this area is still likely to be buggy, i.e. in a + month that has all the dates disabled. + + * calendar.js: + some trivial performance improvements in the _init function + Added Date.parseDate (old Calendar.prototype.parseDate now calls this one) + +2005-03-05 Mihai Bazon <mih...@ya...> + + * release-notes.html: updated release notes + + * dayinfo.html: *** empty log message *** + + * calendar-setup.js: + bugfix--update an inputField even if flat calendar is selected + + * calendar.js: + fixed bugs in parseDate function (if for some reason the input string is + totally broken, then check numbers for NaN and use values from the current + date instead) + + * make-release.pl: copy the skins subdirectory and all skins + + * index.html: added Aqua skin + + * skins/aqua/active-bg.gif, skins/aqua/dark-bg.gif, skins/aqua/hover-bg.gif, skins/aqua/menuarrow.gif, skins/aqua/normal-bg.gif, skins/aqua/rowhover-bg.gif, skins/aqua/status-bg.gif, skins/aqua/theme.css, skins/aqua/title-bg.gif, skins/aqua/today-bg.gif: + in the future, skins will go to this directory, each in a separate subdir; for now there's only Aqua, an excellent new skin + + * calendar.js: workaround IE bug, needed in the Aqua theme + don't hide select elements unless browser is IE or Opera + + * lang/calendar-bg.js, lang/calendar-big5-utf8.js, lang/calendar-big5.js, lang/calendar-br.js, lang/calendar-ca.js, lang/calendar-cs-utf8.js, lang/calendar-cs-win.js, lang/calendar-da.js, lang/calendar-de.js, lang/calendar-el.js, lang/calendar-en.js, lang/calendar-es.js, lang/calendar-fi.js, lang/calendar-fr.js, lang/calendar-he-utf8.js, lang/calendar-hu.js, lang/calendar-it.js, lang/calendar-ko-utf8.js, lang/calendar-ko.js, lang/calendar-lt-utf8.js, lang/calendar-lt.js, lang/calendar-lv.js, lang/calendar-nl.js, lang/calendar-no.js, lang/calendar-pl-utf8.js, lang/calendar-pl.js, lang/calendar-pt.js, lang/calendar-ro.js, lang/calendar-ru.js, lang/calendar-ru_win_.js, lang/calendar-si.js, lang/calendar-sk.js, lang/calendar-sp.js, lang/calendar-sv.js, lang/calendar-zh.js, lang/cn_utf8.js: + updated urls, copyright notices + + * doc/reference.tex: updated documentation + + * calendar.js, index.html: + renamed the global variable to _dynarch_popupCalendar to avoid name clashes + + * multiple-dates.html: start with an empty array + + * calendar.js: + fixed bugs in the time selector (12:XX pm was wrongfully understood as 12:XX am) + + * calendar.js: + using innerHTML instead of text nodes; works better in Safari and also makes + a smaller, cleaner code + +2005-03-04 Mihai Bazon <mih...@ya...> + + * calendar.js: + fixed a performance regression that occurred after adding support for multiple dates + fixed the time selection bug (now it keeps time correctly) + clicking today will close the calendar if "today" is already selected + + * lang/cn_utf8.js: new translation + +2005-02-17 Mihai Bazon <mih...@ya...> + + * lang/calendar-ar-utf8.zip: Added arabic translation + +2004-10-19 Mihai Bazon <mih...@ya...> + + * lang/calendar-zh.js: updated + +2004-09-20 Mihai Bazon <mih...@ya...> + + * lang/calendar-no.js: updated (Daniel Holmen) + +2004-09-20 Mihai Bazon <mih...@ya...> + + * lang/calendar-no.js: updated (Daniel Holmen) + +2004-08-11 Mihai Bazon <mih...@ya...> + + * lang/calendar-nl.js: updated language file (thanks to Arjen Duursma) + + * lang/calendar-sp.js: updated (thanks to Rafael Velasco) + +2004-07-21 Mihai Bazon <mih...@ya...> + + * lang/calendar-br.js: updated + + * calendar-setup.js: fixed bug (dateText) + +2004-07-21 Mihai Bazon <mih...@ya...> + + * lang/calendar-br.js: updated + + * calendar-setup.js: fixed bug (dateText) + +2004-07-04 Mihai Bazon <mih...@ya...> + + * lang/calendar-lv.js: + added LV translation (thanks to Juris Valdovskis) + +2004-06-25 Mihai Bazon <mih...@ya...> + + * calendar.js: + fixed bug in IE (el.calendar.tooltips is null or not an object) + +2004-06-24 Mihai Bazon <mih...@ya...> + + * doc/reference.tex: fixed latex compilation + + * index.html: linking other sample files + + * calendar-setup.js, calendar.js, dayinfo.html: + ability to display day info (dateText parameter) + sample file + +2004-06-23 Mihai Bazon <mih...@ya...> + + * doc/reference.tex, lang/calendar-bg.js, lang/calendar-br.js, lang/calendar-ca.js, lang/calendar-en.js, lang/calendar-es.js, lang/calendar-fr.js, lang/calendar-it.js, lang/calendar-ko-utf8.js, lang/calendar-ko.js, lang/calendar-nl.js, lang/calendar-sv.js, README, calendar.js, index.html: + email address changed + +2004-06-14 Mihai Bazon <mih...@ya...> + + * lang/calendar-cs-utf8.js, lang/calendar-cs-win.js: + updated translations + + * calendar-system.css: added z-index to drop downs + + * lang/calendar-en.js: + first day of week can now be part of the language file + + * lang/calendar-es.js: + updated language file (thanks to Servilio Afre Puentes) + + * calendar-blue2.css, calendar-brown.css, calendar-green.css, calendar-tas.css, calendar-win2k-1.css, calendar-win2k-2.css, calendar-win2k-cold-1.css, calendar-win2k-cold-2.css, calendar-blue.css: + added z-index property to drop downs (fixes bug) + +2004-06-13 Mihai Bazon <mih...@ya...> + + * calendar-setup.js: fixed bug (apply showOthers to flat calendars too) + +2004-06-06 Mihai Bazon <mih...@ya...> + + * calendar-setup.js: + firstDay defaults to "null", in which case the value in the language file + will be used + + * calendar.js: + firstDayOfWeek can now default to a value specified in the language definition file + + * index.html: first day of week is now numeric + +2004-06-02 Mihai Bazon <mih...@ya...> + + * calendar.js: added date tooltip function + +2004-05-28 Mihai Bazon <mih...@ya...> + + * lang/calendar-br.js: updated (thanks to Marcos Pont) + + * calendar-setup.js: fixed small bug + +2004-05-01 Mihai Bazon <mih...@ya...> + + * calendar-setup.js: returns the calendar object + +2004-04-28 Mihai Bazon <mih...@ya...> + + * calendar-setup.js: + patch to read the date value from the inputField, according to ifFormat (if + both are passed), for flat calendars. (thanks Colin T. Hill) + +2004-04-20 Mihai Bazon <mih...@ya...> + + * calendar-setup.js, calendar.js, multiple-dates.html: + added support for multiple dates selection + + * lang/calendar-nl.js: + updated Dutch translation, thanks to Jeroen Wolsink + + * lang/calendar-big5-utf8.js, lang/calendar-big5.js: + Traditional Chinese language (thanks GaryFu) + +2004-03-26 Mihai Bazon <mih...@ya...> + + * lang/calendar-fr.js, lang/calendar-pt.js: updated + + * lang/calendar-ru_win_.js, lang/calendar-ru.js: + updated, thanks to Sly Golovanov + +2004-03-25 Mihai Bazon <mih...@ya...> + + * lang/calendar-fr.js: updated (thanks to David Duret) + +2004-03-24 Mihai Bazon <mih...@ya...> + + * lang/calendar-da.js: updated (thanks to Michael Thingmand Henriksen) + +2004-03-21 Mihai Bazon <mih...@ya...> + + * lang/calendar-ca.js: updated (thanks to David Valls) + +2004-03-17 Mihai Bazon <mih...@ya...> + + * lang/calendar-de.js: updated to UTF8 (thanks to Jack (tR)) + +2004-03-09 Mihai Bazon <mih...@ya...> + + * lang/calendar-bg.js: Bulgarian translation + +2004-03-08 Mihai Bazon <mih...@ya...> + + * lang/calendar-he-utf8.js: Hebrew translation (thanks to Idan Sofer) + + * lang/calendar-hu.js: updated (thanks to Istvan Karaszi) + +2004-02-27 Mihai Bazon <mih...@ya...> + + * lang/calendar-it.js: updated (thanks to Fabio Di Bernardini) + +2004-02-25 Mihai Bazon <mih...@ya...> + + * calendar.js: fix for Safari (thanks to Olivier Chirouze / XPWeb) + +2004-02-22 Mihai Bazon <mih...@ya...> + + * lang/calendar-al.js: Albanian language file + +2004-02-17 Mihai Bazon <mih...@ya...> + + * lang/calendar-fr.js: fixed + + * lang/calendar-fr.js: + FR translation updated (thanks to SIMON Alexandre) + + * lang/calendar-es.js: ES translation updated, thanks to David Gonzales + +2004-02-10 Mihai Bazon <mih...@ya...> + + * lang/calendar-pt.js: + updated Portugese translation, thanks to Elcio Ferreira + +2004-02-09 Mihai Bazon <mih...@ya...> + + * TODO: updated + +2004-02-06 Mihai Bazon <mih...@ya...> + + * README: describe the PHP files + + * make-release.pl: includes php files + + * make-release.pl: ChangeLog included in the distribution (if found) + + * calendar.js, doc/reference.tex, index.html: switched to version 0.9.6 + + * doc/Calendar.setup.tex, doc/reference.tex: updated documentation + + * release-notes.html: updated release notes + + * calendar.js: Fixed bug: Feb/29 and year change now keeps Feb in view + + * calendar.js: fixed the "ESC" problem (call the close handler) + + * calendar.js: fixed day of year range (1 to 366 instead of 0 to 365) + + * calendar.js: fixed week number calculations + + * doc/reference.tex: fixed (date input format) + + * calendar.php: removed comment + + * calendar-blue.css, calendar-blue2.css, calendar-brown.css, calendar-green.css, calendar-system.css, calendar-tas.css, calendar-win2k-1.css, calendar-win2k-2.css, calendar-win2k-cold-1.css, calendar-win2k-cold-2.css, calendar.js: + workaround for IE bug (you can't normally specify through CSS the style for + an element having two classes or more; we had to change a classname) + + * calendar-blue.css, calendar-blue2.css, calendar-brown.css, calendar-green.css, calendar-system.css, calendar-tas.css, calendar-win2k-1.css, calendar-win2k-2.css, calendar-win2k-cold-1.css, calendar-win2k-cold-2.css: + smaller fonts on days that are in neighbor months + +2004-02-04 Mihai Bazon <mih...@ya...> + + * index.html: first demo shows the "showOtherMonths" capability + + * calendar-setup.js: support new parameters in the calendar. + added: firstDay, showOthers, cache. + + * calendar-blue.css, calendar-blue2.css, calendar-brown.css, calendar-green.css, calendar-system.css, calendar-win2k-1.css, calendar-win2k-2.css, calendar-win2k-cold-1.css, calendar-win2k-cold-2.css, calendar.js, lang/calendar-en.js, lang/calendar-ro.js: + new parameters: firstDayOfWeek, showsOtherMonths; removed mondayFirst. + This adds support for setting any day to be the first day of week (by just + clicking the day name in the display); also, if showsOtherMonths is enabled + then dates belonging to adjacent months that are in the current view will be + displayed and the calendar will have a fixed height. + + all themes updated. + + * test.ph... [truncated message content] |
From: <al...@us...> - 2008-02-13 00:50:03
|
Revision: 2200 http://archive-access.svn.sourceforge.net/archive-access/?rev=2200&view=rev Author: alexoz Date: 2008-02-12 16:50:02 -0800 (Tue, 12 Feb 2008) Log Message: ----------- Catch 404s, timeouts etc when fetching robots.txt separately and throw RobotsUnvailableException. Modified Paths: -------------- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/CachingRobotClient.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/HttpRobotClient.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/RobotClient.java Modified: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/CachingRobotClient.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/CachingRobotClient.java 2008-02-11 21:29:21 UTC (rev 2199) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/CachingRobotClient.java 2008-02-13 00:50:02 UTC (rev 2200) @@ -4,16 +4,20 @@ import java.util.ArrayList; import java.util.Collection; import java.util.List; +import java.util.logging.Logger; import org.archive.accesscontrol.LruCache; +import org.archive.accesscontrol.RobotsUnavailableException; /** - * The CachingRobotClient wraps another RobotClient and caches requests. + * The CchingRobotClient wraps another RobotClient and caches requests. * * @author aosborne * */ public class CachingRobotClient extends RobotClient { + private static final Logger LOGGER = Logger.getLogger( + CachingRobotClient.class.getName()); protected LruCache<String, RobotRules> cache = new LruCache<String, RobotRules>(); protected RobotClient client; private static final int PREPARE_THREAD_COUNT = 15; @@ -36,7 +40,7 @@ @Override public RobotRules getRulesForUrl(String url, String userAgent) - throws IOException { + throws IOException, RobotsUnavailableException { String robotsUrl = robotsUrlForUrl(url); RobotRules rules; @@ -78,6 +82,9 @@ getRulesForUrl(url, userAgent); } catch (IOException e) { e.printStackTrace(); + } catch (RobotsUnavailableException e) { + // TODO Auto-generated catch block + e.printStackTrace(); } } } Modified: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/HttpRobotClient.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/HttpRobotClient.java 2008-02-11 21:29:21 UTC (rev 2199) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/HttpRobotClient.java 2008-02-13 00:50:02 UTC (rev 2200) @@ -1,12 +1,19 @@ package org.archive.accesscontrol.robotstxt; import java.io.IOException; +import java.net.ConnectException; +import java.net.NoRouteToHostException; +import java.net.UnknownHostException; import java.util.Collection; +import java.util.logging.Logger; +import org.apache.commons.httpclient.ConnectTimeoutException; import org.apache.commons.httpclient.HttpClient; +import org.apache.commons.httpclient.HttpException; import org.apache.commons.httpclient.HttpMethod; import org.apache.commons.httpclient.MultiThreadedHttpConnectionManager; import org.apache.commons.httpclient.methods.GetMethod; +import org.archive.accesscontrol.RobotsUnavailableException; /** * HttpRobotClient allows fetching of robots.txt rules over HTTP. @@ -15,6 +22,8 @@ * */ public class HttpRobotClient extends RobotClient { + private static final Logger LOGGER = Logger.getLogger( + RobotClient.class.getName()); protected HttpClient http = new HttpClient( new MultiThreadedHttpConnectionManager()); @@ -22,11 +31,32 @@ return http; } - public RobotRules getRulesForUrl(String url, String userAgent) throws IOException { + public RobotRules getRulesForUrl(String url, String userAgent) throws IOException, RobotsUnavailableException { String robotsUrl = robotsUrlForUrl(url); HttpMethod method = new GetMethod(robotsUrl); method.addRequestHeader("User-Agent", userAgent); - http.executeMethod(method); + try { + int code = http.executeMethod(method); + // TODO: Constant 200 + if (code != 200) { + throw new RobotsUnavailableException(robotsUrl); + } + } catch (HttpException e) { + e.printStackTrace(); + throw new RobotsUnavailableException(robotsUrl); + } catch (UnknownHostException e) { + LOGGER.info("Unknown host for URL " + robotsUrl); + throw new RobotsUnavailableException(robotsUrl); + } catch (ConnectTimeoutException e) { + LOGGER.info("Connection Timeout for URL " + robotsUrl); + throw new RobotsUnavailableException(robotsUrl); + } catch (NoRouteToHostException e) { + LOGGER.info("No route to host for URL " + robotsUrl); + throw new RobotsUnavailableException(robotsUrl); + } catch (ConnectException e) { + LOGGER.info("ConnectException URL " + robotsUrl); + throw new RobotsUnavailableException(robotsUrl); + } RobotRules rules = new RobotRules(); rules.parse(method.getResponseBodyAsStream()); return rules; Modified: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/RobotClient.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/RobotClient.java 2008-02-11 21:29:21 UTC (rev 2199) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/RobotClient.java 2008-02-13 00:50:02 UTC (rev 2200) @@ -4,6 +4,7 @@ import java.util.Collection; import org.apache.commons.httpclient.URIException; +import org.archive.accesscontrol.RobotsUnavailableException; import org.archive.net.LaxURI; /** @@ -21,9 +22,10 @@ * @param userAgent * @return * @throws IOException + * @throws RobotsUnavailableException */ public boolean isRobotPermitted(String url, String userAgent) - throws IOException { + throws IOException, RobotsUnavailableException { RobotRules rules = getRulesForUrl(url, userAgent); return !rules.blocksPathForUA(new LaxURI(url, false).getPath(), userAgent); @@ -35,10 +37,11 @@ * @param url * @param userAgent * @return - * @throws IOException + * @throws IOException a local problem occurred when attempting to fetch the robots.txt + * @throws RobotsUnavailableException a remote problem, we found no robots.txt or the server is down. */ public abstract RobotRules getRulesForUrl(String url, String userAgent) - throws IOException; + throws IOException, RobotsUnavailableException; public static String robotsUrlForUrl(String url) throws URIException { LaxURI uri = new LaxURI(url, false); This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <al...@us...> - 2008-02-11 21:29:17
|
Revision: 2199 http://archive-access.svn.sourceforge.net/archive-access/?rev=2199&view=rev Author: alexoz Date: 2008-02-11 13:29:21 -0800 (Mon, 11 Feb 2008) Log Message: ----------- * AccessControlClient.java, CachingRobotClient.java, HttpRobotClient.java, RobotClient.java Added a method to enable using a proxy for robots.txt fetches. Modified Paths: -------------- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/AccessControlClient.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/CachingRobotClient.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/HttpRobotClient.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/RobotClient.java Modified: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/AccessControlClient.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/AccessControlClient.java 2008-02-11 21:23:12 UTC (rev 2198) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/AccessControlClient.java 2008-02-11 21:29:21 UTC (rev 2199) @@ -176,5 +176,14 @@ public void setRobotPreparationEnabled(boolean robotPreparationEnabled) { this.robotPreparationEnabled = robotPreparationEnabled; } + + /** + * Use a proxy server when fetching robots.txt data. + * @param host + * @param port + */ + public void setRobotProxy(String host, int port) { + robotClient.setRobotProxy(host, port); + } } Modified: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/CachingRobotClient.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/CachingRobotClient.java 2008-02-11 21:23:12 UTC (rev 2198) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/CachingRobotClient.java 2008-02-11 21:29:21 UTC (rev 2199) @@ -102,4 +102,9 @@ } } + @Override + public void setRobotProxy(String host, int port) { + client.setRobotProxy(host, port); + } + } Modified: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/HttpRobotClient.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/HttpRobotClient.java 2008-02-11 21:23:12 UTC (rev 2198) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/HttpRobotClient.java 2008-02-11 21:29:21 UTC (rev 2199) @@ -36,4 +36,9 @@ public void prepare(Collection<String> urls, String userAgent) { // no-op } + + @Override + public void setRobotProxy(String host, int port) { + http.getHostConfiguration().setProxy(host, port); + } } Modified: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/RobotClient.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/RobotClient.java 2008-02-11 21:23:12 UTC (rev 2198) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/RobotClient.java 2008-02-11 21:29:21 UTC (rev 2199) @@ -55,4 +55,11 @@ * This may be a no-op. */ public abstract void prepare(Collection<String> urls, String userAgent); + + /** + * Use a proxy server when fetching robots.txt data. + * @param host + * @param port + */ + public abstract void setRobotProxy(String host, int port); } This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <al...@us...> - 2008-02-11 21:23:22
|
Revision: 2198 http://archive-access.svn.sourceforge.net/archive-access/?rev=2198&view=rev Author: alexoz Date: 2008-02-11 13:23:12 -0800 (Mon, 11 Feb 2008) Log Message: ----------- Added robots.txt fetch and rule evaluation. * AccessControlException.java, RobotsUnavailableExcpetion.java, RuleOracleUnavailableException.java Created exceptions for when the oracle or robots.txt are unreachable. * CachingRuleDao.java, HttpRuleDao.java, RuleDao.java Adjusted to throw the new exceptions. * CachingRobotClient.java, HttpRobotClient.java, RobotClient.java, RobotRules.java Created caching robots.txt client based on RobotRules class from wayback. * AccessControlClient.java Added (optional) robots.txt lookups and prepare calls. Modified Paths: -------------- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/AccessControlClient.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/CachingRuleDao.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/HttpRuleDao.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/RuleDao.java trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/AccessControlClientTest.java Added Paths: ----------- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/AccessControlException.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/RobotsUnavailableException.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/RuleOracleUnavailableException.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/CachingRobotClient.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/HttpRobotClient.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/RobotClient.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/RobotRules.java trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/robotstxt/ trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/robotstxt/HttpRobotClientTest.java Modified: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/AccessControlClient.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/AccessControlClient.java 2008-02-08 22:40:50 UTC (rev 2197) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/AccessControlClient.java 2008-02-11 21:23:12 UTC (rev 2198) @@ -1,11 +1,14 @@ package org.archive.accesscontrol; +import java.io.IOException; import java.util.ArrayList; import java.util.Collection; import java.util.Date; import org.archive.accesscontrol.model.Rule; import org.archive.accesscontrol.model.RuleSet; +import org.archive.accesscontrol.robotstxt.CachingRobotClient; +import org.archive.accesscontrol.robotstxt.RobotClient; import org.archive.net.PublicSuffixes; import org.archive.util.ArchiveUtils; import org.archive.util.SURT; @@ -21,10 +24,15 @@ */ public class AccessControlClient { protected RuleDao ruleDao; + protected RobotClient robotClient; + private boolean robotLookupsEnabled = true; + private boolean robotPreparationEnabled = true; + private String robotUserAgent = "wayback-access-control"; - public AccessControlClient(RuleDao ruleDao) { + public AccessControlClient(RuleDao ruleDao, RobotClient robotClient) { super(); this.ruleDao = ruleDao; + this.robotClient = robotClient; } /** @@ -35,7 +43,7 @@ * "http://localhost:8080/exclusions-oracle/" */ public AccessControlClient(String oracleUrl) { - this(new CachingRuleDao(oracleUrl)); + this(new CachingRuleDao(oracleUrl), new CachingRobotClient()); } /** @@ -51,10 +59,24 @@ * Group name of the user accessing the document. * @return Access-control policy that should be enforced. eg "robots", * "block" or "allow". + * @throws RobotsUnavailableException + * @throws RuleOracleUnavailableException */ public String getPolicy(String url, Date captureDate, Date retrievalDate, - String who) { + String who) throws RobotsUnavailableException, RuleOracleUnavailableException { Rule matchingRule = getRule(url, captureDate, retrievalDate, who); + + if (robotLookupsEnabled && matchingRule != null && "robots".equals(matchingRule.getPolicy())) { + try { + if (robotClient.isRobotPermitted(url, robotUserAgent)) { + return "allow"; + } else { + return "block"; + } + } catch (IOException e) { + throw new RobotsUnavailableException(e); + } + } return matchingRule.getPolicy(); } @@ -70,9 +92,10 @@ * @param who * Group name of the user accessing the document. * @return + * @throws RuleOracleUnavailableException */ public Rule getRule(String url, Date captureDate, Date retrievalDate, - String who) { + String who) throws RuleOracleUnavailableException { url = ArchiveUtils.addImpliedHttpIfNecessary(url); String surt = SURT.fromURI(url); String publicSuffix = PublicSuffixes @@ -103,6 +126,10 @@ .reduceSurtToTopmostAssigned(getSurtAuthority(surt))); } ruleDao.prepare(publicSuffixes); + + if (robotPreparationEnabled) { + robotClient.prepare(urls, robotUserAgent); + } } protected String getSurtAuthority(String surt) { @@ -124,4 +151,30 @@ return surt; } } + + + public String getRobotUserAgent() { + return robotUserAgent; + } + + public void setRobotUserAgent(String robotUserAgent) { + this.robotUserAgent = robotUserAgent; + } + + public boolean isRobotLookupsEnabled() { + return robotLookupsEnabled; + } + + public void setRobotLookupsEnabled(boolean robotLookupsEnabled) { + this.robotLookupsEnabled = robotLookupsEnabled; + } + + public boolean isRobotPreparationEnabled() { + return robotPreparationEnabled; + } + + public void setRobotPreparationEnabled(boolean robotPreparationEnabled) { + this.robotPreparationEnabled = robotPreparationEnabled; + } + } Added: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/AccessControlException.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/AccessControlException.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/AccessControlException.java 2008-02-11 21:23:12 UTC (rev 2198) @@ -0,0 +1,30 @@ +package org.archive.accesscontrol; + +public class AccessControlException extends Exception { + + /** + * + */ + private static final long serialVersionUID = 4300180270651774259L; + + public AccessControlException() { + super(); + // TODO Auto-generated constructor stub + } + + public AccessControlException(String arg0, Throwable arg1) { + super(arg0, arg1); + // TODO Auto-generated constructor stub + } + + public AccessControlException(String arg0) { + super(arg0); + // TODO Auto-generated constructor stub + } + + public AccessControlException(Throwable arg0) { + super(arg0); + // TODO Auto-generated constructor stub + } + +} Modified: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/CachingRuleDao.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/CachingRuleDao.java 2008-02-08 22:40:50 UTC (rev 2197) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/CachingRuleDao.java 2008-02-11 21:23:12 UTC (rev 2198) @@ -36,7 +36,7 @@ this.ruleDao = ruleDao; } - public RuleSet getRuleTree(String surt) { + public RuleSet getRuleTree(String surt) throws RuleOracleUnavailableException { RuleSet rules; synchronized (cache) { rules = cache.get(surt); @@ -65,7 +65,12 @@ break; surt = surts.remove(0); } - getRuleTree(surt); + try { + getRuleTree(surt); + } catch (RuleOracleUnavailableException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } } } } Modified: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/HttpRuleDao.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/HttpRuleDao.java 2008-02-08 22:40:50 UTC (rev 2197) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/HttpRuleDao.java 2008-02-11 21:23:12 UTC (rev 2198) @@ -35,9 +35,10 @@ } /** + * @throws RuleOracleUnavailableException * @see RuleDao#getRuleTree(String) */ - public RuleSet getRuleTree(String surt) { + public RuleSet getRuleTree(String surt) throws RuleOracleUnavailableException { HttpMethod method = new GetMethod(oracleUrl + "/rules/tree/" + surt); RuleSet rules; @@ -47,8 +48,7 @@ System.out.println(response); rules = (RuleSet) xstream.fromXML(method.getResponseBodyAsStream()); } catch (IOException e) { - e.printStackTrace(); - return null; + throw new RuleOracleUnavailableException(e); } method.releaseConnection(); return rules; Added: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/RobotsUnavailableException.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/RobotsUnavailableException.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/RobotsUnavailableException.java 2008-02-11 21:23:12 UTC (rev 2198) @@ -0,0 +1,30 @@ +package org.archive.accesscontrol; + +public class RobotsUnavailableException extends AccessControlException { + + /** + * + */ + private static final long serialVersionUID = -6268896797166951256L; + + public RobotsUnavailableException() { + super(); + // TODO Auto-generated constructor stub + } + + public RobotsUnavailableException(String arg0, Throwable arg1) { + super(arg0, arg1); + // TODO Auto-generated constructor stub + } + + public RobotsUnavailableException(String arg0) { + super(arg0); + // TODO Auto-generated constructor stub + } + + public RobotsUnavailableException(Throwable arg0) { + super(arg0); + // TODO Auto-generated constructor stub + } + +} Modified: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/RuleDao.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/RuleDao.java 2008-02-08 22:40:50 UTC (rev 2197) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/RuleDao.java 2008-02-11 21:23:12 UTC (rev 2198) @@ -25,9 +25,10 @@ * * @param surt * @return + * @throws RuleOracleUnavailableException * @throws URIException */ - public RuleSet getRuleTree(String surt); + public RuleSet getRuleTree(String surt) throws RuleOracleUnavailableException; /** * This method allows a RuleDao to prepare for lookups from a given set of Added: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/RuleOracleUnavailableException.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/RuleOracleUnavailableException.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/RuleOracleUnavailableException.java 2008-02-11 21:23:12 UTC (rev 2198) @@ -0,0 +1,30 @@ +package org.archive.accesscontrol; + +public class RuleOracleUnavailableException extends AccessControlException { + + /** + * + */ + private static final long serialVersionUID = 8574598479427378024L; + + public RuleOracleUnavailableException() { + super(); + // TODO Auto-generated constructor stub + } + + public RuleOracleUnavailableException(String arg0, Throwable arg1) { + super(arg0, arg1); + // TODO Auto-generated constructor stub + } + + public RuleOracleUnavailableException(String arg0) { + super(arg0); + // TODO Auto-generated constructor stub + } + + public RuleOracleUnavailableException(Throwable arg0) { + super(arg0); + // TODO Auto-generated constructor stub + } + +} Added: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/CachingRobotClient.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/CachingRobotClient.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/CachingRobotClient.java 2008-02-11 21:23:12 UTC (rev 2198) @@ -0,0 +1,105 @@ +package org.archive.accesscontrol.robotstxt; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.Collection; +import java.util.List; + +import org.archive.accesscontrol.LruCache; + +/** + * The CachingRobotClient wraps another RobotClient and caches requests. + * + * @author aosborne + * + */ +public class CachingRobotClient extends RobotClient { + protected LruCache<String, RobotRules> cache = new LruCache<String, RobotRules>(); + protected RobotClient client; + private static final int PREPARE_THREAD_COUNT = 15; + + public RobotClient getClient() { + return client; + } + + public void setClient(RobotClient client) { + this.client = client; + } + + public CachingRobotClient() { + this.client = new HttpRobotClient(); + } + + public CachingRobotClient(RobotClient client) { + this.client = client; + } + + @Override + public RobotRules getRulesForUrl(String url, String userAgent) + throws IOException { + String robotsUrl = robotsUrlForUrl(url); + RobotRules rules; + + synchronized(cache) { + rules = cache.get(robotsUrl); + } + if (rules == null) { + rules = client.getRulesForUrl(url, userAgent); + + synchronized(cache) { + cache.put(robotsUrl, rules); + } + } + return rules; + } + + public LruCache<String, RobotRules> getCache() { + return cache; + } + + class FetchThread extends Thread { + private List<String> urls; + private String userAgent; + + public FetchThread(List<String> urls, String userAgent) { + this.urls = urls; + this.userAgent = userAgent; + } + + public void run() { + while (true) { + String url; + synchronized (urls) { + if (urls.isEmpty()) + break; + url = urls.remove(0); + } + try { + getRulesForUrl(url, userAgent); + } catch (IOException e) { + e.printStackTrace(); + } + } + } + } + + /** + * Prepare the cache to lookup info for a given set of urls. The fetches + * happen in parallel so this also makes a good option for speeding up bulk lookups. + */ + public void prepare(Collection<String> urls, String userAgent) { + List<String> safeUrls = new ArrayList<String>(urls); + FetchThread threads[] = new FetchThread[PREPARE_THREAD_COUNT ]; + for (int i = 0; i < PREPARE_THREAD_COUNT ; i++) { + threads[i] = new FetchThread(safeUrls, userAgent); + threads[i].start(); + } + for (int i = 0; i < PREPARE_THREAD_COUNT ; i++) { + try { + threads[i].join(); + } catch (InterruptedException e) { + } + } + } + +} Added: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/HttpRobotClient.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/HttpRobotClient.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/HttpRobotClient.java 2008-02-11 21:23:12 UTC (rev 2198) @@ -0,0 +1,39 @@ +package org.archive.accesscontrol.robotstxt; + +import java.io.IOException; +import java.util.Collection; + +import org.apache.commons.httpclient.HttpClient; +import org.apache.commons.httpclient.HttpMethod; +import org.apache.commons.httpclient.MultiThreadedHttpConnectionManager; +import org.apache.commons.httpclient.methods.GetMethod; + +/** + * HttpRobotClient allows fetching of robots.txt rules over HTTP. + * + * @author aosborne + * + */ +public class HttpRobotClient extends RobotClient { + protected HttpClient http = new HttpClient( + new MultiThreadedHttpConnectionManager()); + + public HttpClient getHttpClient() { + return http; + } + + public RobotRules getRulesForUrl(String url, String userAgent) throws IOException { + String robotsUrl = robotsUrlForUrl(url); + HttpMethod method = new GetMethod(robotsUrl); + method.addRequestHeader("User-Agent", userAgent); + http.executeMethod(method); + RobotRules rules = new RobotRules(); + rules.parse(method.getResponseBodyAsStream()); + return rules; + } + + @Override + public void prepare(Collection<String> urls, String userAgent) { + // no-op + } +} Added: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/RobotClient.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/RobotClient.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/RobotClient.java 2008-02-11 21:23:12 UTC (rev 2198) @@ -0,0 +1,58 @@ +package org.archive.accesscontrol.robotstxt; + +import java.io.IOException; +import java.util.Collection; + +import org.apache.commons.httpclient.URIException; +import org.archive.net.LaxURI; + +/** + * A client for checking whether a robot is allowed by a robots.txt file. + * + * @author aosborne + * + */ +public abstract class RobotClient { + /** + * Returns true if a robot with the given user-agent is allowed to access + * the given url. + * + * @param url + * @param userAgent + * @return + * @throws IOException + */ + public boolean isRobotPermitted(String url, String userAgent) + throws IOException { + RobotRules rules = getRulesForUrl(url, userAgent); + return !rules.blocksPathForUA(new LaxURI(url, false).getPath(), + userAgent); + } + + /** + * Fetch the applicable ruleset for the given url and robot. + * + * @param url + * @param userAgent + * @return + * @throws IOException + */ + public abstract RobotRules getRulesForUrl(String url, String userAgent) + throws IOException; + + public static String robotsUrlForUrl(String url) throws URIException { + LaxURI uri = new LaxURI(url, false); + uri.setPath("/robots.txt"); + uri.setQuery(null); + uri.setFragment(null); + return uri.toString(); + } + + /** + * Prepare the cache to lookup info for a given set of urls. The fetches + * happen in parallel so this also makes a good option for speeding up bulk lookups. + * + * This may be a no-op. + */ + public abstract void prepare(Collection<String> urls, String userAgent); +} Added: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/RobotRules.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/RobotRules.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/robotstxt/RobotRules.java 2008-02-11 21:23:12 UTC (rev 2198) @@ -0,0 +1,180 @@ +/* RobotRules + * + * $Id$ + * + * Created on 2:51:20 PM Mar 12, 2007. + * + * Copyright (C) 2007 Internet Archive. + * + * This file is part of wayback-svn. + * + * wayback-svn is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * any later version. + * + * wayback-svn is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser Public License for more details. + * + * You should have received a copy of the GNU Lesser Public License + * along with wayback-svn; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ +package org.archive.accesscontrol.robotstxt; + +import java.io.BufferedReader; +import java.io.IOException; +import java.io.InputStream; +import java.io.InputStreamReader; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.Iterator; +import java.util.LinkedList; +import java.util.List; +import java.util.logging.Logger; + +/** + * Class which parses a robots.txt file, storing the rules contained therein, + * and then allows for testing if path/userAgent tuples are blocked by those + * rules. + * + * @author brad + * @version $Date$, $Revision$ + */ +public class RobotRules { + + private static final long serialVersionUID = 2917420727021840982L; + private static final Logger LOGGER = Logger.getLogger(RobotRules.class + .getName()); + /** + * Special name for User-agent which matches all values + */ + public static final String GLOBAL_USER_AGENT = "*"; + + private boolean bSyntaxErrors = false; + private HashMap<String, ArrayList<String>> rules = + new HashMap<String, ArrayList<String>>(); + + private LinkedList<String> userAgents = new LinkedList<String>(); + + /** + * @return true if the robots.txt file looked suspicious, currently meaning + * we found a Disallow rule that was not preceded by a "User-agent:" line + */ + public boolean hasSyntaxErrors() { + return bSyntaxErrors; + } + + /** + * @return a List of all UserAgents Found in the Robots.txt document + */ + public List<String> getUserAgentsFound() { + return userAgents; + } + + /** + * Read rules from InputStream argument into this RobotRules, as a + * side-effect, sets the bSyntaxErrors property. + * + * @param is + * @throws IOException + */ + public void parse(InputStream is) throws IOException { + + BufferedReader br = new BufferedReader(new InputStreamReader( + (InputStream) is)); + String read; + ArrayList<String> current = null; + while (br != null) { + do { + read = br.readLine(); + // Skip comments & blanks + } while ((read != null) && ((read = read.trim()).startsWith("#") || + read.length() == 0)); + if (read == null) { + br.close(); + br = null; + } else { + int commentIndex = read.indexOf("#"); + if (commentIndex > -1) { + // Strip trailing comment + read = read.substring(0, commentIndex); + } + read = read.trim(); + if (read.matches("(?i)^User-agent:.*")) { + String ua = read.substring(11).trim().toLowerCase(); + if (current == null || current.size() != 0) { + // only create new rules-list if necessary + // otherwise share with previous user-agent + current = new ArrayList<String>(); + } + rules.put(ua, current); + LOGGER.fine("Found User-agent(" + ua + ") rules..."); + continue; + } + if (read.matches("(?i)Disallow:.*")) { + if (current == null) { + // buggy robots.txt + bSyntaxErrors = true; + continue; + } + String path = read.substring(9).trim(); + current.add(path); + continue; + } + // unknown line; do nothing for now + + // TODO: check for "Allow" lines, and flag a syntax error if + // we encounter any unknown lines? + } + } + } + + private boolean blocksPath(String path, String curUA, List<String> uaRules) { + + Iterator<String> disItr = uaRules.iterator(); + while (disItr.hasNext()) { + String disallowedPath = disItr.next(); + if (disallowedPath.length() == 0) { + + LOGGER.fine("UA(" + curUA + + ") has empty disallow: Go for it!"); + return false; + + } else { + LOGGER.fine("UA(" + curUA + ") has (" + + disallowedPath + ") blocked...(" + + disallowedPath.length() + ")"); + if (disallowedPath.equals("/") || path.startsWith(disallowedPath)) { + LOGGER.fine("THIS APPLIES!!!"); + return true; + } + } + } + return false; + } + + /** + * Checks first the specified ua UserAgent, if rules are present for it, + * and then falls back to using rules for the '*' UserAgent. + * + * @param path + * @param ua + * @return boolean value where true indicates the path is blocked for ua + */ + public boolean blocksPathForUA(String path, String ua) { + + if(rules.containsKey(ua.toLowerCase())) { + + return blocksPath(path,ua,rules.get(ua.toLowerCase())); + + } else if(rules.containsKey(GLOBAL_USER_AGENT)) { + + return blocksPath(path,GLOBAL_USER_AGENT, + rules.get(GLOBAL_USER_AGENT)); + } + return false; + } +} Modified: trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/AccessControlClientTest.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/AccessControlClientTest.java 2008-02-08 22:40:50 UTC (rev 2197) +++ trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/AccessControlClientTest.java 2008-02-11 21:23:12 UTC (rev 2198) @@ -1,7 +1,5 @@ package org.archive.accesscontrol; -import java.util.Date; - import org.archive.accesscontrol.AccessControlClient; import junit.framework.TestCase; @@ -13,7 +11,7 @@ protected void setUp() throws Exception { super.setUp(); System.out.println("hello world"); - client = new AccessControlClient(new HttpRuleDao(ORACLE_URL)); + client = new AccessControlClient(ORACLE_URL); } protected void tearDown() throws Exception { Added: trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/robotstxt/HttpRobotClientTest.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/robotstxt/HttpRobotClientTest.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/robotstxt/HttpRobotClientTest.java 2008-02-11 21:23:12 UTC (rev 2198) @@ -0,0 +1,25 @@ +package org.archive.accesscontrol.robotstxt; + +import org.apache.commons.httpclient.URIException; + +import junit.framework.TestCase; + +public class HttpRobotClientTest extends TestCase { + public void testRobotUrlForUrl() throws URIException { + assertEquals("http://example.com/robots.txt", HttpRobotClient.robotsUrlForUrl("http://example.com/")); + assertEquals("http://example.com/robots.txt", HttpRobotClient.robotsUrlForUrl("http://example.com/foo/bar.html?boozle#bazzle")); + assertEquals("https://example.com/robots.txt", HttpRobotClient.robotsUrlForUrl("https://example.com/foo/bar.html?boozle#bazzle")); + assertEquals("https://us...@ex.../robots.txt", HttpRobotClient.robotsUrlForUrl("https://us...@ex.../foo/bar.html?boozle#bazzle")); + assertEquals("http://user:pa...@ex.../robots.txt", HttpRobotClient.robotsUrlForUrl("http://user:pa...@ex.../foo/bar.html?boozle#bazzle")); + assertEquals("http://user:pa...@ex...:2311/robots.txt", HttpRobotClient.robotsUrlForUrl("http://user:pa...@ex...:2311/foo/bar.html?boozle#bazzle")); + } + + public void testBasic() throws Exception { + HttpRobotClient client = new HttpRobotClient(); + assertFalse(client.isRobotPermitted("http://web.archive.org/cgi-bin/fishbowl", "wayback-access-control-test")); + assertTrue(client.isRobotPermitted("http://www.archive.org/index.html", "wayback-access-control-test")); + assertTrue(client.isRobotPermitted("http://google.com/fish.html", "wayback-access-control-test")); + assertFalse(client.isRobotPermitted("http://google.com/news", "wayback-access-control-test")); + + } +} This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <al...@us...> - 2008-02-08 22:40:49
|
Revision: 2197 http://archive-access.svn.sourceforge.net/archive-access/?rev=2197&view=rev Author: alexoz Date: 2008-02-08 14:40:50 -0800 (Fri, 08 Feb 2008) Log Message: ----------- Added prepare() to client API, to allow cache preparation and parallel remote lookups. * AccessControlClient.java Added prepare() method. * RuleDao.java Added prepare() to interface. * CachingRuleDao.java Added prepare() method and threading logic to do parallel lookups to populate the cache. * HttpRuleDao.java, HibernateRuleDao.java Added no-op prepare(). Modified Paths: -------------- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/AccessControlClient.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/CachingRuleDao.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/HttpRuleDao.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/RuleDao.java trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/model/HibernateRuleDao.java Modified: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/AccessControlClient.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/AccessControlClient.java 2008-02-08 21:56:01 UTC (rev 2196) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/AccessControlClient.java 2008-02-08 22:40:50 UTC (rev 2197) @@ -1,10 +1,13 @@ package org.archive.accesscontrol; +import java.util.ArrayList; +import java.util.Collection; import java.util.Date; import org.archive.accesscontrol.model.Rule; import org.archive.accesscontrol.model.RuleSet; import org.archive.net.PublicSuffixes; +import org.archive.util.ArchiveUtils; import org.archive.util.SURT; /** @@ -70,6 +73,7 @@ */ public Rule getRule(String url, Date captureDate, Date retrievalDate, String who) { + url = ArchiveUtils.addImpliedHttpIfNecessary(url); String surt = SURT.fromURI(url); String publicSuffix = PublicSuffixes .reduceSurtToTopmostAssigned(getSurtAuthority(surt)); @@ -82,6 +86,24 @@ retrievalDate, who); return matchingRule; } + + + /** + * This method allows the client to prepare for lookups from a given set of + * urls. This can warm up a cache and/or enable a mass data transfer to be done in + * parallel. + * + * @param surts + */ + public void prepare(Collection<String> urls) { + ArrayList<String> publicSuffixes = new ArrayList<String>(urls.size()); + for (String url: urls) { + String surt = SURT.fromURI(ArchiveUtils.addImpliedHttpIfNecessary(url)); + publicSuffixes.add(PublicSuffixes + .reduceSurtToTopmostAssigned(getSurtAuthority(surt))); + } + ruleDao.prepare(publicSuffixes); + } protected String getSurtAuthority(String surt) { int indexOfOpen = surt.indexOf("://("); Modified: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/CachingRuleDao.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/CachingRuleDao.java 2008-02-08 21:56:01 UTC (rev 2196) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/CachingRuleDao.java 2008-02-08 22:40:50 UTC (rev 2197) @@ -1,5 +1,9 @@ package org.archive.accesscontrol; +import java.util.ArrayList; +import java.util.Collection; +import java.util.List; + import org.archive.accesscontrol.model.RuleSet; /** @@ -9,10 +13,11 @@ * @author aosborne * */ -public class CachingRuleDao extends LruCache<String, RuleSet> implements - RuleDao { +public class CachingRuleDao implements RuleDao { private static final long serialVersionUID = 1L; + private static final int PREPARE_THREAD_COUNT = 5; protected RuleDao ruleDao; + protected LruCache<String, RuleSet> cache = new LruCache<String, RuleSet>(); public CachingRuleDao(RuleDao ruleDao) { super(); @@ -32,11 +37,57 @@ } public RuleSet getRuleTree(String surt) { - RuleSet rules = super.get(surt); + RuleSet rules; + synchronized (cache) { + rules = cache.get(surt); + } if (rules == null) { rules = ruleDao.getRuleTree(surt); - super.put(surt, rules); + synchronized (cache) { + cache.put(surt, rules); + } } return rules; } + + class FetchThread extends Thread { + private List<String> surts; + + public FetchThread(List<String> surts) { + this.surts = surts; + } + + public void run() { + while (true) { + String surt; + synchronized (surts) { + if (surts.isEmpty()) + break; + surt = surts.remove(0); + } + getRuleTree(surt); + } + } + } + + /** + * Prepare the cache to lookup info for a given set of surts. The fetches + * happen in parallel so this also makes a good option for speeding up bulk lookups. + * + * @param surts + */ + public void prepare(Collection<String> surts) { + List<String> safeSurts = new ArrayList<String>(surts); + FetchThread threads[] = new FetchThread[PREPARE_THREAD_COUNT ]; + for (int i = 0; i < PREPARE_THREAD_COUNT ; i++) { + threads[i] = new FetchThread(safeSurts); + threads[i].start(); + } + for (int i = 0; i < PREPARE_THREAD_COUNT ; i++) { + try { + threads[i].join(); + } catch (InterruptedException e) { + } + } + } } Modified: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/HttpRuleDao.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/HttpRuleDao.java 2008-02-08 21:56:01 UTC (rev 2196) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/HttpRuleDao.java 2008-02-08 22:40:50 UTC (rev 2197) @@ -1,9 +1,11 @@ package org.archive.accesscontrol; import java.io.IOException; +import java.util.Collection; import org.apache.commons.httpclient.HttpClient; import org.apache.commons.httpclient.HttpMethod; +import org.apache.commons.httpclient.MultiThreadedHttpConnectionManager; import org.apache.commons.httpclient.methods.GetMethod; import org.archive.accesscontrol.model.Rule; import org.archive.accesscontrol.model.RuleSet; @@ -15,13 +17,14 @@ * the REST interface\xCAan oracle. * * For details of the protocol, see: - * http://webteam.archive.org/confluence/display/wayback/Exclusions+API + * http://webteam.archive.org/confluence/display/wayback/Exclusions+API * * @author aosborne * */ public class HttpRuleDao implements RuleDao { - protected HttpClient http = new HttpClient(); + protected HttpClient http = new HttpClient( + new MultiThreadedHttpConnectionManager()); protected XStream xstream = new XStream(); private String oracleUrl; @@ -30,14 +33,14 @@ xstream.alias("rule", Rule.class); xstream.alias("ruleSet", RuleSet.class); } - + /** * @see RuleDao#getRuleTree(String) */ public RuleSet getRuleTree(String surt) { HttpMethod method = new GetMethod(oracleUrl + "/rules/tree/" + surt); RuleSet rules; - + try { http.executeMethod(method); String response = method.getResponseBodyAsString(); @@ -46,7 +49,7 @@ } catch (IOException e) { e.printStackTrace(); return null; - } + } method.releaseConnection(); return rules; } @@ -59,10 +62,15 @@ } /** - * @param oracleUrl the oracleUrl to set + * @param oracleUrl + * the oracleUrl to set */ public void setOracleUrl(String oracleUrl) { this.oracleUrl = oracleUrl; } - + + public void prepare(Collection<String> surts) { + // no-op + } + } Modified: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/RuleDao.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/RuleDao.java 2008-02-08 21:56:01 UTC (rev 2196) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/RuleDao.java 2008-02-08 22:40:50 UTC (rev 2197) @@ -1,11 +1,13 @@ package org.archive.accesscontrol; +import java.util.Collection; + import org.apache.commons.httpclient.URIException; import org.archive.accesscontrol.model.RuleSet; /** - * A RuleDao provides methods for retrieving rule information from a local database or - * remote oracle. + * A RuleDao provides methods for retrieving rule information from a local + * database or remote oracle. * * @author aosborne * @@ -27,4 +29,12 @@ */ public RuleSet getRuleTree(String surt); + /** + * This method allows a RuleDao to prepare for lookups from a given set of + * surts. This can warm up a cache and/or enable a bulk lookup to be done in + * parallel. Many implementations may make it a no-op. + * + * @param surts + */ + public void prepare(Collection<String> surts); } Modified: trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/model/HibernateRuleDao.java =================================================================== --- trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/model/HibernateRuleDao.java 2008-02-08 21:56:01 UTC (rev 2196) +++ trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/model/HibernateRuleDao.java 2008-02-08 22:40:50 UTC (rev 2197) @@ -1,5 +1,6 @@ package org.archive.accesscontrol.model; +import java.util.Collection; import java.util.List; import org.apache.commons.httpclient.URIException; @@ -105,4 +106,8 @@ public void deleteAllRules() { getHibernateTemplate().bulkUpdate("delete from Rule"); } + + public void prepare(Collection<String> surts) { + // no-op + } } This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <al...@us...> - 2008-02-08 21:56:08
|
Revision: 2196 http://archive-access.svn.sourceforge.net/archive-access/?rev=2196&view=rev Author: alexoz Date: 2008-02-08 13:56:01 -0800 (Fri, 08 Feb 2008) Log Message: ----------- * HttpRuleDao.java, RuleDao.java Moved out of the model package for better organization. * CachingRuleDao.java, LruCache.java Implemented a simple LRU caching mechanism for rule trees. * AccessControlClient.java Changed to using CachingRuleDao by default. Modified Paths: -------------- trunk/archive-access/projects/access-control/.classpath trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/AccessControlClient.java trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/AccessControlClientTest.java trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/model/HibernateRuleDao.java Added Paths: ----------- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/CachingRuleDao.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/HttpRuleDao.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/LruCache.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/RuleDao.java trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/LruCacheTest.java Removed Paths: ------------- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/HttpRuleDao.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/RuleDao.java Modified: trunk/archive-access/projects/access-control/.classpath =================================================================== --- trunk/archive-access/projects/access-control/.classpath 2008-02-08 01:13:57 UTC (rev 2195) +++ trunk/archive-access/projects/access-control/.classpath 2008-02-08 21:56:01 UTC (rev 2196) @@ -3,7 +3,6 @@ <classpathentry kind="src" path="access-control/src/main/java"/> <classpathentry kind="src" path="access-control/src/test/java"/> <classpathentry excluding="**" kind="src" output="access-control/src/main/resources" path="access-control/src/main/resources"/> - <classpathentry excluding="**" kind="src" output="access-control/src/test/resources" path="access-control/src/test/resources"/> <classpathentry kind="src" path="oracle/src/main/java"/> <classpathentry kind="src" path="oracle/src/test/java"/> <classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER"/> Modified: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/AccessControlClient.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/AccessControlClient.java 2008-02-08 01:13:57 UTC (rev 2195) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/AccessControlClient.java 2008-02-08 21:56:01 UTC (rev 2196) @@ -2,9 +2,7 @@ import java.util.Date; -import org.archive.accesscontrol.model.HttpRuleDao; import org.archive.accesscontrol.model.Rule; -import org.archive.accesscontrol.model.RuleDao; import org.archive.accesscontrol.model.RuleSet; import org.archive.net.PublicSuffixes; import org.archive.util.SURT; @@ -27,14 +25,14 @@ } /** - * Create a new client to query a remote oracle. + * Create a new (caching) client to query a remote oracle. * * @param oracleUrl * Base url of the oracle webapp. eg. * "http://localhost:8080/exclusions-oracle/" */ public AccessControlClient(String oracleUrl) { - this(new HttpRuleDao(oracleUrl)); + this(new CachingRuleDao(oracleUrl)); } /** @@ -79,6 +77,7 @@ surt = stripScheme(surt); RuleSet rules = ruleDao.getRuleTree("(" + publicSuffix); + Rule matchingRule = rules.getMatchingRule(surt, captureDate, retrievalDate, who); return matchingRule; Added: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/CachingRuleDao.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/CachingRuleDao.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/CachingRuleDao.java 2008-02-08 21:56:01 UTC (rev 2196) @@ -0,0 +1,42 @@ +package org.archive.accesscontrol; + +import org.archive.accesscontrol.model.RuleSet; + +/** + * CachingRuleDao is a wrapper for another RuleDao that implements in-memory + * caching of the rule trees. + * + * @author aosborne + * + */ +public class CachingRuleDao extends LruCache<String, RuleSet> implements + RuleDao { + private static final long serialVersionUID = 1L; + protected RuleDao ruleDao; + + public CachingRuleDao(RuleDao ruleDao) { + super(); + this.ruleDao = ruleDao; + } + + public CachingRuleDao(String oracleUrl) { + this(new HttpRuleDao(oracleUrl)); + } + + public RuleDao getRuleDao() { + return ruleDao; + } + + public void setRuleDao(RuleDao ruleDao) { + this.ruleDao = ruleDao; + } + + public RuleSet getRuleTree(String surt) { + RuleSet rules = super.get(surt); + if (rules == null) { + rules = ruleDao.getRuleTree(surt); + super.put(surt, rules); + } + return rules; + } +} Copied: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/HttpRuleDao.java (from rev 2186, trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/HttpRuleDao.java) =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/HttpRuleDao.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/HttpRuleDao.java 2008-02-08 21:56:01 UTC (rev 2196) @@ -0,0 +1,68 @@ +package org.archive.accesscontrol; + +import java.io.IOException; + +import org.apache.commons.httpclient.HttpClient; +import org.apache.commons.httpclient.HttpMethod; +import org.apache.commons.httpclient.methods.GetMethod; +import org.archive.accesscontrol.model.Rule; +import org.archive.accesscontrol.model.RuleSet; + +import com.thoughtworks.xstream.XStream; + +/** + * The HTTP Rule Data Access Object enables a rule database to be queried via + * the REST interface\xCAan oracle. + * + * For details of the protocol, see: + * http://webteam.archive.org/confluence/display/wayback/Exclusions+API + * + * @author aosborne + * + */ +public class HttpRuleDao implements RuleDao { + protected HttpClient http = new HttpClient(); + protected XStream xstream = new XStream(); + private String oracleUrl; + + public HttpRuleDao(String oracleUrl) { + this.oracleUrl = oracleUrl; + xstream.alias("rule", Rule.class); + xstream.alias("ruleSet", RuleSet.class); + } + + /** + * @see RuleDao#getRuleTree(String) + */ + public RuleSet getRuleTree(String surt) { + HttpMethod method = new GetMethod(oracleUrl + "/rules/tree/" + surt); + RuleSet rules; + + try { + http.executeMethod(method); + String response = method.getResponseBodyAsString(); + System.out.println(response); + rules = (RuleSet) xstream.fromXML(method.getResponseBodyAsStream()); + } catch (IOException e) { + e.printStackTrace(); + return null; + } + method.releaseConnection(); + return rules; + } + + /** + * @return the oracleUrl + */ + public String getOracleUrl() { + return oracleUrl; + } + + /** + * @param oracleUrl the oracleUrl to set + */ + public void setOracleUrl(String oracleUrl) { + this.oracleUrl = oracleUrl; + } + +} Added: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/LruCache.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/LruCache.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/LruCache.java 2008-02-08 21:56:01 UTC (rev 2196) @@ -0,0 +1,66 @@ +package org.archive.accesscontrol; + +import java.util.Date; +import java.util.HashMap; +import java.util.LinkedHashMap; +import java.util.Map; + +/** + * A simple least recently used cache, with a discard policy based on maximum + * entry count and cache time. Access + * + * @author aosborne + * + * @param <K> + * @param <V> + */ +public class LruCache<K, V> extends LinkedHashMap<K, V> { + private static final long serialVersionUID = 1L; + private int maxEntries = 100; + private long maxCacheTime = 10 * 60 * 1000; // ten minutes + private Map<K, Date> refreshTimes = new HashMap<K, Date>(); + + protected boolean removeEldestEntry(Map.Entry<K, V> eldest) { + return size() > maxEntries + || new Date().getTime() + - refreshTimes.get(eldest.getKey()).getTime() > maxCacheTime; + } + + public int getMaxEntries() { + return maxEntries; + } + + /** + * Set the maximum number of entries to be stored in the cache. + * + * @param maxEntries + */ + public void setMaxEntries(int maxEntries) { + this.maxEntries = maxEntries; + } + + public long getMaxCacheTime() { + return maxCacheTime; + } + + /** + * Set the maximum time in milliseconds an entry should be cached for. + * + * @param maxCacheTime + */ + public void setMaxCacheTime(long maxCacheTime) { + this.maxCacheTime = maxCacheTime; + } + + @Override + public V put(K key, V value) { + refreshTimes.put(key, new Date()); + return super.put(key, value); + } + + @Override + public V remove(Object key) { + refreshTimes.remove(key); + return super.remove(key); + } +} Copied: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/RuleDao.java (from rev 2186, trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/RuleDao.java) =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/RuleDao.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/RuleDao.java 2008-02-08 21:56:01 UTC (rev 2196) @@ -0,0 +1,30 @@ +package org.archive.accesscontrol; + +import org.apache.commons.httpclient.URIException; +import org.archive.accesscontrol.model.RuleSet; + +/** + * A RuleDao provides methods for retrieving rule information from a local database or + * remote oracle. + * + * @author aosborne + * + */ +public interface RuleDao { + + /** + * Returns the "rule tree" for a given SURT. This is a sorted set of all + * rules equal or lower in specificity than the given SURT plus all rules on + * the path from this SURT to the root SURT "(". + * + * The intention is to call this function with a domain or public suffix, + * then queries within that domain can be made very fast by searching the + * resulting list. + * + * @param surt + * @return + * @throws URIException + */ + public RuleSet getRuleTree(String surt); + +} Deleted: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/HttpRuleDao.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/HttpRuleDao.java 2008-02-08 01:13:57 UTC (rev 2195) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/HttpRuleDao.java 2008-02-08 21:56:01 UTC (rev 2196) @@ -1,66 +0,0 @@ -package org.archive.accesscontrol.model; - -import java.io.IOException; - -import org.apache.commons.httpclient.HttpClient; -import org.apache.commons.httpclient.HttpMethod; -import org.apache.commons.httpclient.methods.GetMethod; - -import com.thoughtworks.xstream.XStream; - -/** - * The HTTP Rule Data Access Object enables a rule database to be queried via - * the REST interface\xCAan oracle. - * - * For details of the protocol, see: - * http://webteam.archive.org/confluence/display/wayback/Exclusions+API - * - * @author aosborne - * - */ -public class HttpRuleDao implements RuleDao { - protected HttpClient http = new HttpClient(); - protected XStream xstream = new XStream(); - private String oracleUrl; - - public HttpRuleDao(String oracleUrl) { - this.oracleUrl = oracleUrl; - xstream.alias("rule", Rule.class); - xstream.alias("ruleSet", RuleSet.class); - } - - /** - * @see RuleDao#getRuleTree(String) - */ - public RuleSet getRuleTree(String surt) { - HttpMethod method = new GetMethod(oracleUrl + "/rules/tree/" + surt); - RuleSet rules; - - try { - http.executeMethod(method); - String response = method.getResponseBodyAsString(); - System.out.println(response); - rules = (RuleSet) xstream.fromXML(method.getResponseBodyAsStream()); - } catch (IOException e) { - e.printStackTrace(); - return null; - } - method.releaseConnection(); - return rules; - } - - /** - * @return the oracleUrl - */ - public String getOracleUrl() { - return oracleUrl; - } - - /** - * @param oracleUrl the oracleUrl to set - */ - public void setOracleUrl(String oracleUrl) { - this.oracleUrl = oracleUrl; - } - -} Deleted: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/RuleDao.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/RuleDao.java 2008-02-08 01:13:57 UTC (rev 2195) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/RuleDao.java 2008-02-08 21:56:01 UTC (rev 2196) @@ -1,29 +0,0 @@ -package org.archive.accesscontrol.model; - -import org.apache.commons.httpclient.URIException; - -/** - * A RuleDao provides methods for retrieving rule information from a local database or - * remote oracle. - * - * @author aosborne - * - */ -public interface RuleDao { - - /** - * Returns the "rule tree" for a given SURT. This is a sorted set of all - * rules equal or lower in specificity than the given SURT plus all rules on - * the path from this SURT to the root SURT "(". - * - * The intention is to call this function with a domain or public suffix, - * then queries within that domain can be made very fast by searching the - * resulting list. - * - * @param surt - * @return - * @throws URIException - */ - public RuleSet getRuleTree(String surt); - -} Modified: trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/AccessControlClientTest.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/AccessControlClientTest.java 2008-02-08 01:13:57 UTC (rev 2195) +++ trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/AccessControlClientTest.java 2008-02-08 21:56:01 UTC (rev 2196) @@ -3,16 +3,16 @@ import java.util.Date; import org.archive.accesscontrol.AccessControlClient; -import org.archive.accesscontrol.model.HttpRuleDao; import junit.framework.TestCase; public class AccessControlClientTest extends TestCase { - public static final String ORACLE_URL = "http://localhost:8080/exclusions-oracle-0.0.1-SNAPSHOT/"; + public static final String ORACLE_URL = "http://localhost:8080/oracle-0.0.1-SNAPSHOT/"; private AccessControlClient client; protected void setUp() throws Exception { super.setUp(); + System.out.println("hello world"); client = new AccessControlClient(new HttpRuleDao(ORACLE_URL)); } @@ -22,7 +22,8 @@ } public void testBasicOkToShow() throws Exception { - //System.out.println(client.getPolicy("http://www.archive.org/secret/page.html", new Date(), new Date())); + //String policy = client.getPolicy("http://www.peagreenboat.com/", new Date(1987, 8, 30), new Date(), "blah"); + //System.out.println("Policy=" + policy); } } Added: trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/LruCacheTest.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/LruCacheTest.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/LruCacheTest.java 2008-02-08 21:56:01 UTC (rev 2196) @@ -0,0 +1,24 @@ +package org.archive.accesscontrol; + +import junit.framework.TestCase; + +public class LruCacheTest extends TestCase { + LruCache<String,Integer> cache; + + public void testMaxItems() { + cache = new LruCache<String,Integer>(); + + cache.setMaxEntries(3); + cache.put("one", 1); + cache.put("two", 2); + cache.put("three", 3); + + assertEquals(3, cache.size()); + + cache.put("four", 4); + + assertEquals("Maximum entry cap", 3, cache.size()); + assertNull("Ensure 'one' was the evicted object.", cache.get("one")); + + } +} Modified: trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/model/HibernateRuleDao.java =================================================================== --- trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/model/HibernateRuleDao.java 2008-02-08 01:13:57 UTC (rev 2195) +++ trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/model/HibernateRuleDao.java 2008-02-08 21:56:01 UTC (rev 2196) @@ -3,6 +3,7 @@ import java.util.List; import org.apache.commons.httpclient.URIException; +import org.archive.accesscontrol.RuleDao; import org.archive.surt.SURTTokenizer2; import org.hibernate.Session; import org.hibernate.Transaction; This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <al...@us...> - 2008-02-08 01:13:51
|
Revision: 2195 http://archive-access.svn.sourceforge.net/archive-access/?rev=2195&view=rev Author: alexoz Date: 2008-02-07 17:13:57 -0800 (Thu, 07 Feb 2008) Log Message: ----------- * HibernateRuleDao.java, DaoTestCase.java, HibernateRuleDaoTest.java Moved to oracle project, so that we can drop the spring and hibernate dependencies for the client. * pom.xml Drop spring and hibernate dependencies in client library. Modified Paths: -------------- trunk/archive-access/projects/access-control/.settings/org.eclipse.jdt.core.prefs trunk/archive-access/projects/access-control/access-control/pom.xml trunk/archive-access/projects/access-control/oracle/pom.xml Added Paths: ----------- trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/model/ trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/model/HibernateRuleDao.java trunk/archive-access/projects/access-control/oracle/src/test/java/org/archive/accesscontrol/model/ trunk/archive-access/projects/access-control/oracle/src/test/java/org/archive/accesscontrol/model/DaoTestCase.java trunk/archive-access/projects/access-control/oracle/src/test/java/org/archive/accesscontrol/model/HibernateRuleDaoTest.java trunk/archive-access/projects/access-control/oracle/src/test/resources/ Removed Paths: ------------- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/HibernateRuleDao.java trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/model/DaoTestCase.java trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/model/HibernateRuleDaoTest.java trunk/archive-access/projects/access-control/access-control/src/test/resources/ Modified: trunk/archive-access/projects/access-control/.settings/org.eclipse.jdt.core.prefs =================================================================== --- trunk/archive-access/projects/access-control/.settings/org.eclipse.jdt.core.prefs 2008-02-07 23:36:42 UTC (rev 2194) +++ trunk/archive-access/projects/access-control/.settings/org.eclipse.jdt.core.prefs 2008-02-08 01:13:57 UTC (rev 2195) @@ -1,4 +1,4 @@ -#Fri Feb 01 11:37:16 PST 2008 +#Thu Feb 07 15:35:49 PST 2008 eclipse.preferences.version=1 org.eclipse.jdt.core.compiler.codegen.targetPlatform=1.5 org.eclipse.jdt.core.compiler.compliance=1.5 Modified: trunk/archive-access/projects/access-control/access-control/pom.xml =================================================================== --- trunk/archive-access/projects/access-control/access-control/pom.xml 2008-02-07 23:36:42 UTC (rev 2194) +++ trunk/archive-access/projects/access-control/access-control/pom.xml 2008-02-08 01:13:57 UTC (rev 2195) @@ -14,13 +14,6 @@ </parent> <build> - <extensions> - <extension> - <groupId>hsqldb</groupId> - <artifactId>hsqldb</artifactId> - <version>1.8.0.7</version> - </extension> - </extensions> <plugins> <plugin> <artifactId>maven-compiler-plugin</artifactId> @@ -29,27 +22,7 @@ <target>1.5</target> </configuration> </plugin> - <plugin> - <groupId>org.codehaus.mojo</groupId> - <artifactId>hibernate3-maven-plugin</artifactId> - <version>2.0-alpha-2</version> - <configuration> - <components> - <component> - <name>hbm2ddl</name> - <implementation>jdbcconfiguration</implementation> - </component> - <component> - <name>hbm2hbmxml</name> - <outputDirectory>src/main/resources</outputDirectory> - </component> - </components> - <componentProperties> - <drop>true</drop> - <configurationfile>/src/main/resources/hibernate.cfg.xml</configurationfile> - </componentProperties> - </configuration> - </plugin> + </plugins> </build> <repositories> @@ -71,37 +44,14 @@ </repository> </repositories> <dependencies> - <dependency> - <groupId>hsqldb</groupId> - <artifactId>hsqldb</artifactId> - <version>1.8.0.7</version> - </dependency> + <dependency> - <groupId>org.springframework</groupId> - <artifactId>spring-webmvc</artifactId> - <version>2.5.1</version> - </dependency> - <dependency> - <groupId>org.springframework</groupId> - <artifactId>spring-orm</artifactId> - <version>2.5.1</version> - </dependency> - <dependency> - <groupId>org.springframework</groupId> - <artifactId>spring-jdbc</artifactId> - <version>2.5.1</version> - </dependency> - <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.4</version> </dependency> + <dependency> - <groupId>postgresql</groupId> - <artifactId>postgresql</artifactId> - <version>8.2-504.jdbc3</version> - </dependency> - <dependency> <groupId>com.thoughtworks.xstream</groupId> <artifactId>xstream</artifactId> <version>1.2.2</version> @@ -116,12 +66,8 @@ <artifactId>commons</artifactId> <version>2.0.0-RC1</version> </dependency> + <dependency> - <groupId>org.hibernate</groupId> - <artifactId>hibernate</artifactId> - <version>3.2.5.ga</version> - </dependency> - <dependency> <groupId>commons-pool</groupId> <artifactId>commons-pool</artifactId> <version>1.3</version> Deleted: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/HibernateRuleDao.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/HibernateRuleDao.java 2008-02-07 23:36:42 UTC (rev 2194) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/HibernateRuleDao.java 2008-02-08 01:13:57 UTC (rev 2195) @@ -1,107 +0,0 @@ - package org.archive.accesscontrol.model; - -import java.util.List; - -import org.apache.commons.httpclient.URIException; -import org.archive.surt.SURTTokenizer2; -import org.hibernate.Session; -import org.hibernate.Transaction; -import org.springframework.orm.hibernate3.support.HibernateDaoSupport; - -/** - * The rule data access object provides convenience methods for using Hibernate - * to access stored rules. The database connection is expected to be configured - * using the SpringFramework ORM layer. - * - * @author aosborne - */ -@SuppressWarnings("unchecked") -public class HibernateRuleDao extends HibernateDaoSupport implements RuleDao { - public Rule getRule(Long id) { - return (Rule) getHibernateTemplate().get(Rule.class, id); - } - - public List<Rule> getAllRules() { - return getHibernateTemplate().find("from Rule"); - } - - public List<Rule> getRulesWithSurtPrefix(String prefix) { - // escape wildcard characters % and _ using ! as the escape character. - prefix = prefix.replace("!", "!!").replace("%", "!%") - .replace("_", "!_"); - return getHibernateTemplate().find( - "from Rule rule where rule.surt like ? escape '!'", - prefix + "%"); - } - - public List<Rule> getRulesWithExactSurt(String surt) { - return getHibernateTemplate().find( - "from Rule rule where rule.surt = ?", surt); - } - - /** - * Returns the "rule tree" for a given SURT. This is a sorted set of all - * rules equal or lower in specificity than the given SURT plus all rules on - * the path from this SURT to the root SURT "(". - * - * The intention is to call this function with a domain or public suffix, - * then queries within that domain can be made very fast by searching the - * resulting list. - * - * @param surt - * @return - */ - public RuleSet getRuleTree(String surt) { - RuleSet rules = new RuleSet(); - - // add the root SURT - rules.addAll(getRulesWithExactSurt("(")); - - // now pull out all of the requested branch and a path to the root - SURTTokenizer2 tok = SURTTokenizer2.newFromSURT(surt); - while (true) { - String search = tok.nextSearch(); - if (search == null) break; - - if (!search.endsWith(SURTTokenizer2.EXACT_SUFFIX)) { - rules.addAll(getRulesWithExactSurt(search)); - } else { - rules.addAll(getRulesWithSurtPrefix(search.substring(0, search.length() - 1))); - tok.nextSearch(); // skip the duplicate exact-match - } - } - - return rules; - } - - public void saveRule(Rule rule) { - getHibernateTemplate().saveOrUpdate(rule); - } - - /** - * Save a rule and a change log entry in one go. (Uses a transaction). - * @param rule - * @param change - */ - public void saveRule(Rule rule, RuleChange change) { - Session session1 = getHibernateTemplate().getSessionFactory().openSession(); - Transaction tx = session1.beginTransaction(); - session1.saveOrUpdate(rule); - session1.save(change); - tx.commit(); - session1.close(); - } - - public void saveChange(RuleChange change) { - getHibernateTemplate().saveOrUpdate(change); - } - - public void deleteRule(Long id) { - Object record = getHibernateTemplate().load(Rule.class, id); - getHibernateTemplate().delete(record); - } - - public void deleteAllRules() { - getHibernateTemplate().bulkUpdate("delete from Rule"); - } -} Deleted: trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/model/DaoTestCase.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/model/DaoTestCase.java 2008-02-07 23:36:42 UTC (rev 2194) +++ trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/model/DaoTestCase.java 2008-02-08 01:13:57 UTC (rev 2195) @@ -1,16 +0,0 @@ -package org.archive.accesscontrol.model; - -import org.springframework.context.ApplicationContext; -import org.springframework.context.support.ClassPathXmlApplicationContext; - -import junit.framework.TestCase; - -public abstract class DaoTestCase extends TestCase { - protected ApplicationContext ctx = null; - - public DaoTestCase() { - // Should put in a parent class that extends TestCase - String[] paths = { "applicationContext.xml" }; - ctx = new ClassPathXmlApplicationContext(paths); - } -} Deleted: trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/model/HibernateRuleDaoTest.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/model/HibernateRuleDaoTest.java 2008-02-07 23:36:42 UTC (rev 2194) +++ trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/model/HibernateRuleDaoTest.java 2008-02-08 01:13:57 UTC (rev 2195) @@ -1,144 +0,0 @@ -package org.archive.accesscontrol.model; - -import java.util.Iterator; -import java.util.List; -import java.util.SortedSet; - -import org.archive.accesscontrol.model.HibernateRuleDao; -import org.archive.accesscontrol.model.Rule; -import org.archive.accesscontrol.model.RuleChange; - -import junit.framework.Assert; - -public class HibernateRuleDaoTest extends DaoTestCase { - private Rule rule = null; - private HibernateRuleDao dao = null; - - - protected void setUp() throws Exception { - super.setUp(); - dao = (HibernateRuleDao) ctx.getBean("ruleDao"); - - // clear database of rules - for (Rule rule: dao.getAllRules()) { - try { - dao.deleteRule(rule.getId()); - } catch (Exception e) {} - } - } - - protected void tearDown() throws Exception { - super.tearDown(); - dao = null; - } - - public void testSaveRecord() throws Exception { - rule = new Rule(); - rule.setSurt("org,archive"); - rule.setWho("admins"); - dao.saveRule(rule); - Assert.assertNotNull("primary key assigned", rule.getId()); - } - - public void testChange() throws Exception { - RuleChange change = new RuleChange(); - change.setSurt("org,archive"); - change.setWho("admins"); - dao.saveChange(change); - Assert.assertNotNull("primary key assigned", change.getId()); - } - - public void testSurtPrefixQuery() throws Exception { - rule = new Rule(); - rule.setSurt("http://(org,archive,unique)/%%__/fish"); - dao.saveRule(rule); - - Rule rule2 = new Rule(); - rule2.setSurt("http://(org,archive,unique)/blasted/fish"); - dao.saveRule(rule2); - - List<Rule> rules = dao.getRulesWithSurtPrefix("http://(org,archive,unique)/%%__"); - - Boolean foundRule1 = false; - Boolean foundRule2 = false; - for (Rule r: rules) { - if (rule.getId().equals(r.getId())) { - foundRule1 = true; - } - if (rule2.getId().equals(r.getId())) { - foundRule2 = true; - } - } - Assert.assertTrue("Should match rule 1", foundRule1); - Assert.assertFalse("Should not match rule 2", foundRule2); - } - - public void testExactSurtQuery() throws Exception { - rule = new Rule(); - rule.setSurt("http://(org,archive,unique)/%%__/fish"); - dao.saveRule(rule); - - Rule rule2 = new Rule(); - rule2.setSurt("http://(org,archive,unique)/blasted/fish"); - dao.saveRule(rule2); - - - List<Rule> rules = dao.getRulesWithExactSurt("http://(org,archive,unique)/%%__/fish"); - - Boolean foundRule1 = false; - Boolean foundRule2 = false; - for (Rule r: rules) { - if (rule.getId().equals(r.getId())) { - foundRule1 = true; - } - if (rule2.getId().equals(r.getId())) { - foundRule2 = true; - } - } - - Assert.assertTrue("Should match rule 1", foundRule1); - Assert.assertFalse("Should not match rule 2", foundRule2); - } - - public void testSurtTreeQuery() throws Exception { - rule = new Rule(); - rule.setSurt("(org,archive,unique,)/secret"); - dao.saveRule(rule); - - Rule rule1 = new Rule(); - rule1.setSurt("(org,archive,unique,)/"); - dao.saveRule(rule1); - - Rule rule2 = new Rule(); - rule2.setSurt("(org,archive,unique"); - dao.saveRule(rule2); - - Rule rule3 = new Rule(); - rule3.setSurt("(org,archive"); - dao.saveRule(rule3); - - Rule rule4 = new Rule(); - rule4.setSurt("("); - dao.saveRule(rule4); - - Rule rule5 = new Rule(); - rule5.setSurt("(org,archive,unique,)/other"); - dao.saveRule(rule5); - - Rule rule6 = new Rule(); - rule6.setSurt("(org,error,)/foobar"); - dao.saveRule(rule6); - - Iterable<Rule> rules = dao.getRuleTree("http://(org,archive,unique,)/"); - - for (Rule r: rules) { - assertTrue(r.getId().equals(rule.getId()) - || r.getId().equals(rule1.getId()) - || r.getId().equals(rule2.getId()) - || r.getId().equals(rule3.getId()) - || r.getId().equals(rule4.getId()) - || r.getId().equals(rule5.getId())); - } - } - -} Modified: trunk/archive-access/projects/access-control/oracle/pom.xml =================================================================== --- trunk/archive-access/projects/access-control/oracle/pom.xml 2008-02-07 23:36:42 UTC (rev 2194) +++ trunk/archive-access/projects/access-control/oracle/pom.xml 2008-02-08 01:13:57 UTC (rev 2195) @@ -112,6 +112,11 @@ <version>8.2-504.jdbc3</version> </dependency> <dependency> + <groupId>hsqldb</groupId> + <artifactId>hsqldb</artifactId> + <version>1.8.0.7</version> + </dependency> + <dependency> <groupId>com.thoughtworks.xstream</groupId> <artifactId>xstream</artifactId> <version>1.2.2</version> Copied: trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/model/HibernateRuleDao.java (from rev 2186, trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/HibernateRuleDao.java) =================================================================== --- trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/model/HibernateRuleDao.java (rev 0) +++ trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/model/HibernateRuleDao.java 2008-02-08 01:13:57 UTC (rev 2195) @@ -0,0 +1,107 @@ + package org.archive.accesscontrol.model; + +import java.util.List; + +import org.apache.commons.httpclient.URIException; +import org.archive.surt.SURTTokenizer2; +import org.hibernate.Session; +import org.hibernate.Transaction; +import org.springframework.orm.hibernate3.support.HibernateDaoSupport; + +/** + * The rule data access object provides convenience methods for using Hibernate + * to access stored rules. The database connection is expected to be configured + * using the SpringFramework ORM layer. + * + * @author aosborne + */ +@SuppressWarnings("unchecked") +public class HibernateRuleDao extends HibernateDaoSupport implements RuleDao { + public Rule getRule(Long id) { + return (Rule) getHibernateTemplate().get(Rule.class, id); + } + + public List<Rule> getAllRules() { + return getHibernateTemplate().find("from Rule"); + } + + public List<Rule> getRulesWithSurtPrefix(String prefix) { + // escape wildcard characters % and _ using ! as the escape character. + prefix = prefix.replace("!", "!!").replace("%", "!%") + .replace("_", "!_"); + return getHibernateTemplate().find( + "from Rule rule where rule.surt like ? escape '!'", + prefix + "%"); + } + + public List<Rule> getRulesWithExactSurt(String surt) { + return getHibernateTemplate().find( + "from Rule rule where rule.surt = ?", surt); + } + + /** + * Returns the "rule tree" for a given SURT. This is a sorted set of all + * rules equal or lower in specificity than the given SURT plus all rules on + * the path from this SURT to the root SURT "(". + * + * The intention is to call this function with a domain or public suffix, + * then queries within that domain can be made very fast by searching the + * resulting list. + * + * @param surt + * @return + */ + public RuleSet getRuleTree(String surt) { + RuleSet rules = new RuleSet(); + + // add the root SURT + rules.addAll(getRulesWithExactSurt("(")); + + // now pull out all of the requested branch and a path to the root + SURTTokenizer2 tok = SURTTokenizer2.newFromSURT(surt); + while (true) { + String search = tok.nextSearch(); + if (search == null) break; + + if (!search.endsWith(SURTTokenizer2.EXACT_SUFFIX)) { + rules.addAll(getRulesWithExactSurt(search)); + } else { + rules.addAll(getRulesWithSurtPrefix(search.substring(0, search.length() - 1))); + tok.nextSearch(); // skip the duplicate exact-match + } + } + + return rules; + } + + public void saveRule(Rule rule) { + getHibernateTemplate().saveOrUpdate(rule); + } + + /** + * Save a rule and a change log entry in one go. (Uses a transaction). + * @param rule + * @param change + */ + public void saveRule(Rule rule, RuleChange change) { + Session session1 = getHibernateTemplate().getSessionFactory().openSession(); + Transaction tx = session1.beginTransaction(); + session1.saveOrUpdate(rule); + session1.save(change); + tx.commit(); + session1.close(); + } + + public void saveChange(RuleChange change) { + getHibernateTemplate().saveOrUpdate(change); + } + + public void deleteRule(Long id) { + Object record = getHibernateTemplate().load(Rule.class, id); + getHibernateTemplate().delete(record); + } + + public void deleteAllRules() { + getHibernateTemplate().bulkUpdate("delete from Rule"); + } +} Copied: trunk/archive-access/projects/access-control/oracle/src/test/java/org/archive/accesscontrol/model/DaoTestCase.java (from rev 2186, trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/model/DaoTestCase.java) =================================================================== --- trunk/archive-access/projects/access-control/oracle/src/test/java/org/archive/accesscontrol/model/DaoTestCase.java (rev 0) +++ trunk/archive-access/projects/access-control/oracle/src/test/java/org/archive/accesscontrol/model/DaoTestCase.java 2008-02-08 01:13:57 UTC (rev 2195) @@ -0,0 +1,16 @@ +package org.archive.accesscontrol.model; + +import org.springframework.context.ApplicationContext; +import org.springframework.context.support.ClassPathXmlApplicationContext; + +import junit.framework.TestCase; + +public abstract class DaoTestCase extends TestCase { + protected ApplicationContext ctx = null; + + public DaoTestCase() { + // Should put in a parent class that extends TestCase + String[] paths = { "applicationContext.xml" }; + ctx = new ClassPathXmlApplicationContext(paths); + } +} Copied: trunk/archive-access/projects/access-control/oracle/src/test/java/org/archive/accesscontrol/model/HibernateRuleDaoTest.java (from rev 2186, trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/model/HibernateRuleDaoTest.java) =================================================================== --- trunk/archive-access/projects/access-control/oracle/src/test/java/org/archive/accesscontrol/model/HibernateRuleDaoTest.java (rev 0) +++ trunk/archive-access/projects/access-control/oracle/src/test/java/org/archive/accesscontrol/model/HibernateRuleDaoTest.java 2008-02-08 01:13:57 UTC (rev 2195) @@ -0,0 +1,144 @@ +package org.archive.accesscontrol.model; + +import java.util.Iterator; +import java.util.List; +import java.util.SortedSet; + +import org.archive.accesscontrol.model.HibernateRuleDao; +import org.archive.accesscontrol.model.Rule; +import org.archive.accesscontrol.model.RuleChange; + +import junit.framework.Assert; + +public class HibernateRuleDaoTest extends DaoTestCase { + private Rule rule = null; + private HibernateRuleDao dao = null; + + + protected void setUp() throws Exception { + super.setUp(); + dao = (HibernateRuleDao) ctx.getBean("ruleDao"); + + // clear database of rules + for (Rule rule: dao.getAllRules()) { + try { + dao.deleteRule(rule.getId()); + } catch (Exception e) {} + } + } + + protected void tearDown() throws Exception { + super.tearDown(); + dao = null; + } + + public void testSaveRecord() throws Exception { + rule = new Rule(); + rule.setSurt("org,archive"); + rule.setWho("admins"); + dao.saveRule(rule); + Assert.assertNotNull("primary key assigned", rule.getId()); + } + + public void testChange() throws Exception { + RuleChange change = new RuleChange(); + change.setSurt("org,archive"); + change.setWho("admins"); + dao.saveChange(change); + Assert.assertNotNull("primary key assigned", change.getId()); + } + + public void testSurtPrefixQuery() throws Exception { + rule = new Rule(); + rule.setSurt("http://(org,archive,unique)/%%__/fish"); + dao.saveRule(rule); + + Rule rule2 = new Rule(); + rule2.setSurt("http://(org,archive,unique)/blasted/fish"); + dao.saveRule(rule2); + + List<Rule> rules = dao.getRulesWithSurtPrefix("http://(org,archive,unique)/%%__"); + + Boolean foundRule1 = false; + Boolean foundRule2 = false; + for (Rule r: rules) { + if (rule.getId().equals(r.getId())) { + foundRule1 = true; + } + if (rule2.getId().equals(r.getId())) { + foundRule2 = true; + } + } + Assert.assertTrue("Should match rule 1", foundRule1); + Assert.assertFalse("Should not match rule 2", foundRule2); + } + + public void testExactSurtQuery() throws Exception { + rule = new Rule(); + rule.setSurt("http://(org,archive,unique)/%%__/fish"); + dao.saveRule(rule); + + Rule rule2 = new Rule(); + rule2.setSurt("http://(org,archive,unique)/blasted/fish"); + dao.saveRule(rule2); + + + List<Rule> rules = dao.getRulesWithExactSurt("http://(org,archive,unique)/%%__/fish"); + + Boolean foundRule1 = false; + Boolean foundRule2 = false; + for (Rule r: rules) { + if (rule.getId().equals(r.getId())) { + foundRule1 = true; + } + if (rule2.getId().equals(r.getId())) { + foundRule2 = true; + } + } + + Assert.assertTrue("Should match rule 1", foundRule1); + Assert.assertFalse("Should not match rule 2", foundRule2); + } + + public void testSurtTreeQuery() throws Exception { + rule = new Rule(); + rule.setSurt("(org,archive,unique,)/secret"); + dao.saveRule(rule); + + Rule rule1 = new Rule(); + rule1.setSurt("(org,archive,unique,)/"); + dao.saveRule(rule1); + + Rule rule2 = new Rule(); + rule2.setSurt("(org,archive,unique"); + dao.saveRule(rule2); + + Rule rule3 = new Rule(); + rule3.setSurt("(org,archive"); + dao.saveRule(rule3); + + Rule rule4 = new Rule(); + rule4.setSurt("("); + dao.saveRule(rule4); + + Rule rule5 = new Rule(); + rule5.setSurt("(org,archive,unique,)/other"); + dao.saveRule(rule5); + + Rule rule6 = new Rule(); + rule6.setSurt("(org,error,)/foobar"); + dao.saveRule(rule6); + + Iterable<Rule> rules = dao.getRuleTree("http://(org,archive,unique,)/"); + + for (Rule r: rules) { + assertTrue(r.getId().equals(rule.getId()) + || r.getId().equals(rule1.getId()) + || r.getId().equals(rule2.getId()) + || r.getId().equals(rule3.getId()) + || r.getId().equals(rule4.getId()) + || r.getId().equals(rule5.getId())); + } + } + +} Copied: trunk/archive-access/projects/access-control/oracle/src/test/resources (from rev 2186, trunk/archive-access/projects/access-control/access-control/src/test/resources) This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <al...@us...> - 2008-02-07 23:36:41
|
Revision: 2194 http://archive-access.svn.sourceforge.net/archive-access/?rev=2194&view=rev Author: alexoz Date: 2008-02-07 15:36:42 -0800 (Thu, 07 Feb 2008) Log Message: ----------- * pom.xml Removed URL filter dependency from core library and forced servlet-api 2.4 dependency to match wayback. Modified Paths: -------------- trunk/archive-access/projects/access-control/access-control/pom.xml trunk/archive-access/projects/access-control/oracle/pom.xml Modified: trunk/archive-access/projects/access-control/access-control/pom.xml =================================================================== --- trunk/archive-access/projects/access-control/access-control/pom.xml 2008-02-07 23:25:50 UTC (rev 2193) +++ trunk/archive-access/projects/access-control/access-control/pom.xml 2008-02-07 23:36:42 UTC (rev 2194) @@ -97,11 +97,6 @@ <version>4.4</version> </dependency> <dependency> - <groupId>org.tuckey</groupId> - <artifactId>urlrewritefilter</artifactId> - <version>3.0.4</version> - </dependency> - <dependency> <groupId>postgresql</groupId> <artifactId>postgresql</artifactId> <version>8.2-504.jdbc3</version> Modified: trunk/archive-access/projects/access-control/oracle/pom.xml =================================================================== --- trunk/archive-access/projects/access-control/oracle/pom.xml 2008-02-07 23:25:50 UTC (rev 2193) +++ trunk/archive-access/projects/access-control/oracle/pom.xml 2008-02-07 23:36:42 UTC (rev 2194) @@ -71,6 +71,11 @@ </repository> </repositories> <dependencies> + <dependency> + <groupId>javax.servlet</groupId> + <artifactId>servlet-api</artifactId> + <version>2.4</version> + </dependency> <dependency> <groupId>org.archive.access-control</groupId> <artifactId>access-control</artifactId> This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <al...@us...> - 2008-02-07 23:25:51
|
Revision: 2193 http://archive-access.svn.sourceforge.net/archive-access/?rev=2193&view=rev Author: alexoz Date: 2008-02-07 15:25:50 -0800 (Thu, 07 Feb 2008) Log Message: ----------- Bumped Heritrix dependency version to 2.0.1-SNAPSHOT to match wayback. Modified Paths: -------------- trunk/archive-access/projects/access-control/oracle/pom.xml Modified: trunk/archive-access/projects/access-control/oracle/pom.xml =================================================================== --- trunk/archive-access/projects/access-control/oracle/pom.xml 2008-02-07 22:56:30 UTC (rev 2192) +++ trunk/archive-access/projects/access-control/oracle/pom.xml 2008-02-07 23:25:50 UTC (rev 2193) @@ -119,7 +119,7 @@ <dependency> <groupId>org.archive.heritrix</groupId> <artifactId>commons</artifactId> - <version>2.0.0-RC1</version> + <version>2.0.1-SNAPSHOT</version> </dependency> <dependency> <groupId>org.hibernate</groupId> This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
Revision: 2192 http://archive-access.svn.sourceforge.net/archive-access/?rev=2192&view=rev Author: alexoz Date: 2008-02-07 14:56:30 -0800 (Thu, 07 Feb 2008) Log Message: ----------- * CreatedView.java Added class comment. Modified Paths: -------------- trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/oracle/CreatedView.java Modified: trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/oracle/CreatedView.java =================================================================== --- trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/oracle/CreatedView.java 2008-02-07 22:51:38 UTC (rev 2191) +++ trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/oracle/CreatedView.java 2008-02-07 22:56:30 UTC (rev 2192) @@ -7,11 +7,14 @@ import org.springframework.web.servlet.View; +/** + * View returning the HTTP 201 Created response, to indicate a resouce was + * created successfully. + */ public class CreatedView implements View { private String path; - - + public CreatedView(String path) { super(); this.path = path; @@ -25,7 +28,8 @@ } /** - * @param path the path to set + * @param path + * the path to set */ public void setPath(String path) { this.path = path; @@ -37,21 +41,23 @@ public void render(Map model, HttpServletRequest request, HttpServletResponse response) throws Exception { - response.addHeader("Location", getContextUrl(request) + path); + response.addHeader("Location", getContextUrl(request) + path); response.setStatus(201); } - + /** * Return the full URL of the context. * * eg. http://localhost:8080/exclusions-oracle-0.0.1-SNAPSHOT + * * @param request * @return */ public String getContextUrl(HttpServletRequest request) { StringBuffer url = request.getRequestURL(); String context = request.getContextPath(); - String contextUrl = url.substring(0, url.indexOf(context) + context.length()); + String contextUrl = url.substring(0, url.indexOf(context) + + context.length()); return contextUrl; } } This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <al...@us...> - 2008-02-07 22:51:37
|
Revision: 2191 http://archive-access.svn.sourceforge.net/archive-access/?rev=2191&view=rev Author: alexoz Date: 2008-02-07 14:51:38 -0800 (Thu, 07 Feb 2008) Log Message: ----------- * pom.xml Changed ${artifactId} to ${pom.artifactId} to satisfy Maven's deprecation warnings. Modified Paths: -------------- trunk/archive-access/projects/access-control/pom.xml Modified: trunk/archive-access/projects/access-control/pom.xml =================================================================== --- trunk/archive-access/projects/access-control/pom.xml 2008-02-07 22:33:24 UTC (rev 2190) +++ trunk/archive-access/projects/access-control/pom.xml 2008-02-07 22:51:38 UTC (rev 2191) @@ -37,7 +37,7 @@ <id>website</id> <name>Website</name> <!--Pass as command-line system property to maven--> - <url>${website.url}/projects/${artifactId}</url> + <url>${website.url}/projects/${pom.artifactId}</url> </site> </distributionManagement> </project> This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <al...@us...> - 2008-02-07 22:33:30
|
Revision: 2190 http://archive-access.svn.sourceforge.net/archive-access/?rev=2190&view=rev Author: alexoz Date: 2008-02-07 14:33:24 -0800 (Thu, 07 Feb 2008) Log Message: ----------- * pom.xml Added name elements to prevent from showing up as "unnamed" in Continuum. Modified Paths: -------------- trunk/archive-access/projects/access-control/access-control/pom.xml trunk/archive-access/projects/access-control/oracle/pom.xml trunk/archive-access/projects/access-control/pom.xml Modified: trunk/archive-access/projects/access-control/access-control/pom.xml =================================================================== --- trunk/archive-access/projects/access-control/access-control/pom.xml 2008-02-07 22:28:38 UTC (rev 2189) +++ trunk/archive-access/projects/access-control/access-control/pom.xml 2008-02-07 22:33:24 UTC (rev 2190) @@ -5,6 +5,7 @@ <packaging>jar</packaging> <version>0.0.1-SNAPSHOT</version> <description></description> + <name>Access-Control: core library and client</name> <parent> <groupId>org.archive</groupId> Modified: trunk/archive-access/projects/access-control/oracle/pom.xml =================================================================== --- trunk/archive-access/projects/access-control/oracle/pom.xml 2008-02-07 22:28:38 UTC (rev 2189) +++ trunk/archive-access/projects/access-control/oracle/pom.xml 2008-02-07 22:33:24 UTC (rev 2190) @@ -5,6 +5,7 @@ <packaging>war</packaging> <version>0.0.1-SNAPSHOT</version> <description></description> + <name>Access-Control: Oracle Webapp</name> <parent> <groupId>org.archive</groupId> Modified: trunk/archive-access/projects/access-control/pom.xml =================================================================== --- trunk/archive-access/projects/access-control/pom.xml 2008-02-07 22:28:38 UTC (rev 2189) +++ trunk/archive-access/projects/access-control/pom.xml 2008-02-07 22:33:24 UTC (rev 2190) @@ -4,6 +4,7 @@ <artifactId>access-control</artifactId> <version>0.0.1-SNAPSHOT</version> <packaging>pom</packaging> + <name>Access-Control</name> <modules> <module>access-control</module> This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <al...@us...> - 2008-02-07 22:28:40
|
Revision: 2189 http://archive-access.svn.sourceforge.net/archive-access/?rev=2189&view=rev Author: alexoz Date: 2008-02-07 14:28:38 -0800 (Thu, 07 Feb 2008) Log Message: ----------- * pom.xml Added sun java maven repository to satisfy JTA dependency. Modified Paths: -------------- trunk/archive-access/projects/access-control/access-control/pom.xml trunk/archive-access/projects/access-control/oracle/pom.xml Modified: trunk/archive-access/projects/access-control/access-control/pom.xml =================================================================== --- trunk/archive-access/projects/access-control/access-control/pom.xml 2008-02-07 03:57:02 UTC (rev 2188) +++ trunk/archive-access/projects/access-control/access-control/pom.xml 2008-02-07 22:28:38 UTC (rev 2189) @@ -60,6 +60,14 @@ <id>archive</id> <url>http://builds.archive.org:8080/maven2</url> </repository> + <repository> + <releases /> + <snapshots> + <enabled>false</enabled> + </snapshots> + <id>sun-java</id> + <url>http://download.java.net/maven/2</url> + </repository> </repositories> <dependencies> <dependency> Modified: trunk/archive-access/projects/access-control/oracle/pom.xml =================================================================== --- trunk/archive-access/projects/access-control/oracle/pom.xml 2008-02-07 03:57:02 UTC (rev 2188) +++ trunk/archive-access/projects/access-control/oracle/pom.xml 2008-02-07 22:28:38 UTC (rev 2189) @@ -60,6 +60,14 @@ <id>archive</id> <url>http://builds.archive.org:8080/maven2</url> </repository> + <repository> + <releases /> + <snapshots> + <enabled>false</enabled> + </snapshots> + <id>sun-java</id> + <url>http://download.java.net/maven/2</url> + </repository> </repositories> <dependencies> <dependency> This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <al...@us...> - 2008-02-07 03:56:58
|
Revision: 2188 http://archive-access.svn.sourceforge.net/archive-access/?rev=2188&view=rev Author: alexoz Date: 2008-02-06 19:57:02 -0800 (Wed, 06 Feb 2008) Log Message: ----------- * pom.xml Added distributionManagement section for Continuum's deploy. Modified Paths: -------------- trunk/archive-access/projects/access-control/pom.xml Modified: trunk/archive-access/projects/access-control/pom.xml =================================================================== --- trunk/archive-access/projects/access-control/pom.xml 2008-02-07 03:42:46 UTC (rev 2187) +++ trunk/archive-access/projects/access-control/pom.xml 2008-02-07 03:57:02 UTC (rev 2188) @@ -24,4 +24,19 @@ <system>continuum</system> <url>http://builds.archive.org:8081/continuum/</url> </ciManagement> + + <distributionManagement> + <repository> + <id>repository</id> + <name>Repository</name> + <!--Pass as command-line system property to maven--> + <url>${repository.url}</url> + </repository> + <site> + <id>website</id> + <name>Website</name> + <!--Pass as command-line system property to maven--> + <url>${website.url}/projects/${artifactId}</url> + </site> + </distributionManagement> </project> This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <al...@us...> - 2008-02-07 03:42:43
|
Revision: 2187 http://archive-access.svn.sourceforge.net/archive-access/?rev=2187&view=rev Author: alexoz Date: 2008-02-06 19:42:46 -0800 (Wed, 06 Feb 2008) Log Message: ----------- * pom.xml Added scm and ciManagement sections so Continuum can find the code. Modified Paths: -------------- trunk/archive-access/projects/access-control/pom.xml Modified: trunk/archive-access/projects/access-control/pom.xml =================================================================== --- trunk/archive-access/projects/access-control/pom.xml 2008-02-07 03:17:24 UTC (rev 2186) +++ trunk/archive-access/projects/access-control/pom.xml 2008-02-07 03:42:46 UTC (rev 2187) @@ -9,4 +9,19 @@ <module>access-control</module> <module>oracle</module> </modules> + + <scm> + <connection> + scm:svn:https://archive-access.svn.sourceforge.net/svnroot/archive-access/trunk/archive-access/projects/access-control + </connection> + <tag>HEAD</tag> + <url> + https://archive-access.svn.sourceforge.net/svnroot/archive-access/trunk/archive-access/projects/access-control + </url> + </scm> + + <ciManagement> + <system>continuum</system> + <url>http://builds.archive.org:8081/continuum/</url> + </ciManagement> </project> This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <al...@us...> - 2008-02-07 03:17:19
|
Revision: 2186 http://archive-access.svn.sourceforge.net/archive-access/?rev=2186&view=rev Author: alexoz Date: 2008-02-06 19:17:24 -0800 (Wed, 06 Feb 2008) Log Message: ----------- Initial commit of wayback access-control library and oracle. Basic rule management and querying functionality is in place but it will require much more extensive testing. Added Paths: ----------- trunk/archive-access/projects/access-control/pom.xml Added: trunk/archive-access/projects/access-control/pom.xml =================================================================== --- trunk/archive-access/projects/access-control/pom.xml (rev 0) +++ trunk/archive-access/projects/access-control/pom.xml 2008-02-07 03:17:24 UTC (rev 2186) @@ -0,0 +1,12 @@ +<?xml version="1.0" encoding="UTF-8"?><project> + <modelVersion>4.0.0</modelVersion> + <groupId>org.archive</groupId> + <artifactId>access-control</artifactId> + <version>0.0.1-SNAPSHOT</version> + <packaging>pom</packaging> + + <modules> + <module>access-control</module> + <module>oracle</module> + </modules> +</project> This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <al...@us...> - 2008-02-07 03:15:06
|
Revision: 2185 http://archive-access.svn.sourceforge.net/archive-access/?rev=2185&view=rev Author: alexoz Date: 2008-02-06 19:15:11 -0800 (Wed, 06 Feb 2008) Log Message: ----------- Initial commit of wayback access-control library and oracle. Basic rule management and querying functionality is in place but it will require much more extensive testing. Added Paths: ----------- trunk/archive-access/projects/access-control/.classpath trunk/archive-access/projects/access-control/.project trunk/archive-access/projects/access-control/.settings/ trunk/archive-access/projects/access-control/.settings/org.eclipse.jdt.core.prefs trunk/archive-access/projects/access-control/access-control/ trunk/archive-access/projects/access-control/access-control/pom.xml trunk/archive-access/projects/access-control/access-control/src/ trunk/archive-access/projects/access-control/access-control/src/main/ trunk/archive-access/projects/access-control/access-control/src/main/java/ trunk/archive-access/projects/access-control/access-control/src/main/java/org/ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/AccessControlClient.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/HibernateRuleDao.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/HttpRuleDao.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/Rule.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/RuleChange.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/RuleDao.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/RuleSet.java trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/surt/ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/surt/SURTTokenizer2.java trunk/archive-access/projects/access-control/access-control/src/main/resources/ trunk/archive-access/projects/access-control/access-control/src/main/resources/Rule.hbm.xml trunk/archive-access/projects/access-control/access-control/src/main/resources/RuleChange.hbm.xml trunk/archive-access/projects/access-control/access-control/src/main/resources/hibernate.cfg.xml trunk/archive-access/projects/access-control/access-control/src/test/ trunk/archive-access/projects/access-control/access-control/src/test/java/ trunk/archive-access/projects/access-control/access-control/src/test/java/org/ trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/ trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/ trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/AccessControlClientTest.java trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/model/ trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/model/DaoTestCase.java trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/model/HibernateRuleDaoTest.java trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/model/RuleSetTest.java trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/model/RuleTest.java trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/surt/ trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/surt/SURTTokenizer2Test.java trunk/archive-access/projects/access-control/access-control/src/test/resources/ trunk/archive-access/projects/access-control/access-control/src/test/resources/applicationContext.xml trunk/archive-access/projects/access-control/oracle/ trunk/archive-access/projects/access-control/oracle/pom.xml trunk/archive-access/projects/access-control/oracle/src/ trunk/archive-access/projects/access-control/oracle/src/main/ trunk/archive-access/projects/access-control/oracle/src/main/java/ trunk/archive-access/projects/access-control/oracle/src/main/java/org/ trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/ trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/ trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/oracle/ trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/oracle/AutoFormatView.java trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/oracle/CreatedView.java trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/oracle/RulesController.java trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/oracle/SimpleError.java trunk/archive-access/projects/access-control/oracle/src/main/java/org/archive/accesscontrol/oracle/XStreamView.java trunk/archive-access/projects/access-control/oracle/src/main/webapp/ trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/ trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/applicationContext.xml trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/exclusions-oracle-servlet.xml trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/urlrewrite.xml trunk/archive-access/projects/access-control/oracle/src/main/webapp/WEB-INF/web.xml trunk/archive-access/projects/access-control/oracle/src/main/webapp/index.html trunk/archive-access/projects/access-control/oracle/src/test/ trunk/archive-access/projects/access-control/oracle/src/test/java/ trunk/archive-access/projects/access-control/oracle/src/test/java/org/ trunk/archive-access/projects/access-control/oracle/src/test/java/org/archive/ trunk/archive-access/projects/access-control/oracle/src/test/java/org/archive/accesscontrol/ Added: trunk/archive-access/projects/access-control/.classpath =================================================================== --- trunk/archive-access/projects/access-control/.classpath (rev 0) +++ trunk/archive-access/projects/access-control/.classpath 2008-02-07 03:15:11 UTC (rev 2185) @@ -0,0 +1,12 @@ +<?xml version="1.0" encoding="UTF-8"?> +<classpath> + <classpathentry kind="src" path="access-control/src/main/java"/> + <classpathentry kind="src" path="access-control/src/test/java"/> + <classpathentry excluding="**" kind="src" output="access-control/src/main/resources" path="access-control/src/main/resources"/> + <classpathentry excluding="**" kind="src" output="access-control/src/test/resources" path="access-control/src/test/resources"/> + <classpathentry kind="src" path="oracle/src/main/java"/> + <classpathentry kind="src" path="oracle/src/test/java"/> + <classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER"/> + <classpathentry kind="con" path="org.maven.ide.eclipse.MAVEN2_CLASSPATH_CONTAINER/modules"/> + <classpathentry kind="output" path="target/classes"/> +</classpath> Added: trunk/archive-access/projects/access-control/.project =================================================================== --- trunk/archive-access/projects/access-control/.project (rev 0) +++ trunk/archive-access/projects/access-control/.project 2008-02-07 03:15:11 UTC (rev 2185) @@ -0,0 +1,23 @@ +<?xml version="1.0" encoding="UTF-8"?> +<projectDescription> + <name>access-control</name> + <comment></comment> + <projects> + </projects> + <buildSpec> + <buildCommand> + <name>org.maven.ide.eclipse.maven2Builder</name> + <arguments> + </arguments> + </buildCommand> + <buildCommand> + <name>org.eclipse.jdt.core.javabuilder</name> + <arguments> + </arguments> + </buildCommand> + </buildSpec> + <natures> + <nature>org.eclipse.jdt.core.javanature</nature> + <nature>org.maven.ide.eclipse.maven2Nature</nature> + </natures> +</projectDescription> Added: trunk/archive-access/projects/access-control/.settings/org.eclipse.jdt.core.prefs =================================================================== --- trunk/archive-access/projects/access-control/.settings/org.eclipse.jdt.core.prefs (rev 0) +++ trunk/archive-access/projects/access-control/.settings/org.eclipse.jdt.core.prefs 2008-02-07 03:15:11 UTC (rev 2185) @@ -0,0 +1,7 @@ +#Fri Feb 01 11:37:16 PST 2008 +eclipse.preferences.version=1 +org.eclipse.jdt.core.compiler.codegen.targetPlatform=1.5 +org.eclipse.jdt.core.compiler.compliance=1.5 +org.eclipse.jdt.core.compiler.problem.assertIdentifier=error +org.eclipse.jdt.core.compiler.problem.enumIdentifier=error +org.eclipse.jdt.core.compiler.source=1.5 Added: trunk/archive-access/projects/access-control/access-control/pom.xml =================================================================== --- trunk/archive-access/projects/access-control/access-control/pom.xml (rev 0) +++ trunk/archive-access/projects/access-control/access-control/pom.xml 2008-02-07 03:15:11 UTC (rev 2185) @@ -0,0 +1,131 @@ +<?xml version="1.0" encoding="UTF-8"?><project> + <modelVersion>4.0.0</modelVersion> + <groupId>org.archive.access-control</groupId> + <artifactId>access-control</artifactId> + <packaging>jar</packaging> + <version>0.0.1-SNAPSHOT</version> + <description></description> + + <parent> + <groupId>org.archive</groupId> + <artifactId>access-control</artifactId> + <version>0.0.1-SNAPSHOT</version> + </parent> + + <build> + <extensions> + <extension> + <groupId>hsqldb</groupId> + <artifactId>hsqldb</artifactId> + <version>1.8.0.7</version> + </extension> + </extensions> + <plugins> + <plugin> + <artifactId>maven-compiler-plugin</artifactId> + <configuration> + <source>1.5</source> + <target>1.5</target> + </configuration> + </plugin> + <plugin> + <groupId>org.codehaus.mojo</groupId> + <artifactId>hibernate3-maven-plugin</artifactId> + <version>2.0-alpha-2</version> + <configuration> + <components> + <component> + <name>hbm2ddl</name> + <implementation>jdbcconfiguration</implementation> + </component> + <component> + <name>hbm2hbmxml</name> + <outputDirectory>src/main/resources</outputDirectory> + </component> + </components> + <componentProperties> + <drop>true</drop> + <configurationfile>/src/main/resources/hibernate.cfg.xml</configurationfile> + </componentProperties> + </configuration> + </plugin> + </plugins> + </build> + <repositories> + <repository> + <releases /> + <snapshots> + <enabled>false</enabled> + </snapshots> + <id>archive</id> + <url>http://builds.archive.org:8080/maven2</url> + </repository> + </repositories> + <dependencies> + <dependency> + <groupId>hsqldb</groupId> + <artifactId>hsqldb</artifactId> + <version>1.8.0.7</version> + </dependency> + <dependency> + <groupId>org.springframework</groupId> + <artifactId>spring-webmvc</artifactId> + <version>2.5.1</version> + </dependency> + <dependency> + <groupId>org.springframework</groupId> + <artifactId>spring-orm</artifactId> + <version>2.5.1</version> + </dependency> + <dependency> + <groupId>org.springframework</groupId> + <artifactId>spring-jdbc</artifactId> + <version>2.5.1</version> + </dependency> + <dependency> + <groupId>junit</groupId> + <artifactId>junit</artifactId> + <version>4.4</version> + </dependency> + <dependency> + <groupId>org.tuckey</groupId> + <artifactId>urlrewritefilter</artifactId> + <version>3.0.4</version> + </dependency> + <dependency> + <groupId>postgresql</groupId> + <artifactId>postgresql</artifactId> + <version>8.2-504.jdbc3</version> + </dependency> + <dependency> + <groupId>com.thoughtworks.xstream</groupId> + <artifactId>xstream</artifactId> + <version>1.2.2</version> + </dependency> + <dependency> + <groupId>org.codehaus.jettison</groupId> + <artifactId>jettison</artifactId> + <version>1.0-beta-1</version> + </dependency> + <dependency> + <groupId>org.archive.heritrix</groupId> + <artifactId>commons</artifactId> + <version>2.0.0-RC1</version> + </dependency> + <dependency> + <groupId>org.hibernate</groupId> + <artifactId>hibernate</artifactId> + <version>3.2.5.ga</version> + </dependency> + <dependency> + <groupId>commons-pool</groupId> + <artifactId>commons-pool</artifactId> + <version>1.3</version> + </dependency> + <dependency> + <groupId>commons-dbcp</groupId> + <artifactId>commons-dbcp</artifactId> + <version>1.2.2</version> + </dependency> + </dependencies> +</project> Added: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/AccessControlClient.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/AccessControlClient.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/AccessControlClient.java 2008-02-07 03:15:11 UTC (rev 2185) @@ -0,0 +1,106 @@ +package org.archive.accesscontrol; + +import java.util.Date; + +import org.archive.accesscontrol.model.HttpRuleDao; +import org.archive.accesscontrol.model.Rule; +import org.archive.accesscontrol.model.RuleDao; +import org.archive.accesscontrol.model.RuleSet; +import org.archive.net.PublicSuffixes; +import org.archive.util.SURT; + +/** + * The Exclusions Client provides a facade for accessing a remote or local + * exclusions oracle. + * + * In future it will perform heavy caching to prevent queries about related and + * recently-accessed pages from needing to hit the oracle. + * + * @author aosborne + */ +public class AccessControlClient { + protected RuleDao ruleDao; + + public AccessControlClient(RuleDao ruleDao) { + super(); + this.ruleDao = ruleDao; + } + + /** + * Create a new client to query a remote oracle. + * + * @param oracleUrl + * Base url of the oracle webapp. eg. + * "http://localhost:8080/exclusions-oracle/" + */ + public AccessControlClient(String oracleUrl) { + this(new HttpRuleDao(oracleUrl)); + } + + /** + * Return the best-matching policy for the requested document. + * + * @param url + * URL of the requested document. + * @param captureDate + * Date the document was archived. + * @param retrievalDate + * Date of retrieval (usually now). + * @param who + * Group name of the user accessing the document. + * @return Access-control policy that should be enforced. eg "robots", + * "block" or "allow". + */ + public String getPolicy(String url, Date captureDate, Date retrievalDate, + String who) { + Rule matchingRule = getRule(url, captureDate, retrievalDate, who); + return matchingRule.getPolicy(); + } + + /** + * Return the most specific matching rule for the requested document. + * + * @param url + * URL of the requested document. + * @param captureDate + * Date the document was archived. + * @param retrievalDate + * Date of retrieval (usually now). + * @param who + * Group name of the user accessing the document. + * @return + */ + public Rule getRule(String url, Date captureDate, Date retrievalDate, + String who) { + String surt = SURT.fromURI(url); + String publicSuffix = PublicSuffixes + .reduceSurtToTopmostAssigned(getSurtAuthority(surt)); + + surt = stripScheme(surt); + + RuleSet rules = ruleDao.getRuleTree("(" + publicSuffix); + Rule matchingRule = rules.getMatchingRule(surt, captureDate, + retrievalDate, who); + return matchingRule; + } + + protected String getSurtAuthority(String surt) { + int indexOfOpen = surt.indexOf("://("); + int indexOfClose = surt.indexOf(")"); + if (indexOfOpen == -1 || indexOfClose == -1 + || ((indexOfOpen + 4) >= indexOfClose)) { + return surt; + } + return surt.substring(indexOfOpen + 4, indexOfClose); + } + + protected static String stripScheme(String surt) { + int i = surt.indexOf("://"); + int j = surt.indexOf(":"); + if (i >= 0 && i == j) { + return surt.substring(i + 3); + } else { + return surt; + } + } +} Added: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/HibernateRuleDao.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/HibernateRuleDao.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/HibernateRuleDao.java 2008-02-07 03:15:11 UTC (rev 2185) @@ -0,0 +1,107 @@ + package org.archive.accesscontrol.model; + +import java.util.List; + +import org.apache.commons.httpclient.URIException; +import org.archive.surt.SURTTokenizer2; +import org.hibernate.Session; +import org.hibernate.Transaction; +import org.springframework.orm.hibernate3.support.HibernateDaoSupport; + +/** + * The rule data access object provides convenience methods for using Hibernate + * to access stored rules. The database connection is expected to be configured + * using the SpringFramework ORM layer. + * + * @author aosborne + */ +@SuppressWarnings("unchecked") +public class HibernateRuleDao extends HibernateDaoSupport implements RuleDao { + public Rule getRule(Long id) { + return (Rule) getHibernateTemplate().get(Rule.class, id); + } + + public List<Rule> getAllRules() { + return getHibernateTemplate().find("from Rule"); + } + + public List<Rule> getRulesWithSurtPrefix(String prefix) { + // escape wildcard characters % and _ using ! as the escape character. + prefix = prefix.replace("!", "!!").replace("%", "!%") + .replace("_", "!_"); + return getHibernateTemplate().find( + "from Rule rule where rule.surt like ? escape '!'", + prefix + "%"); + } + + public List<Rule> getRulesWithExactSurt(String surt) { + return getHibernateTemplate().find( + "from Rule rule where rule.surt = ?", surt); + } + + /** + * Returns the "rule tree" for a given SURT. This is a sorted set of all + * rules equal or lower in specificity than the given SURT plus all rules on + * the path from this SURT to the root SURT "(". + * + * The intention is to call this function with a domain or public suffix, + * then queries within that domain can be made very fast by searching the + * resulting list. + * + * @param surt + * @return + */ + public RuleSet getRuleTree(String surt) { + RuleSet rules = new RuleSet(); + + // add the root SURT + rules.addAll(getRulesWithExactSurt("(")); + + // now pull out all of the requested branch and a path to the root + SURTTokenizer2 tok = SURTTokenizer2.newFromSURT(surt); + while (true) { + String search = tok.nextSearch(); + if (search == null) break; + + if (!search.endsWith(SURTTokenizer2.EXACT_SUFFIX)) { + rules.addAll(getRulesWithExactSurt(search)); + } else { + rules.addAll(getRulesWithSurtPrefix(search.substring(0, search.length() - 1))); + tok.nextSearch(); // skip the duplicate exact-match + } + } + + return rules; + } + + public void saveRule(Rule rule) { + getHibernateTemplate().saveOrUpdate(rule); + } + + /** + * Save a rule and a change log entry in one go. (Uses a transaction). + * @param rule + * @param change + */ + public void saveRule(Rule rule, RuleChange change) { + Session session1 = getHibernateTemplate().getSessionFactory().openSession(); + Transaction tx = session1.beginTransaction(); + session1.saveOrUpdate(rule); + session1.save(change); + tx.commit(); + session1.close(); + } + + public void saveChange(RuleChange change) { + getHibernateTemplate().saveOrUpdate(change); + } + + public void deleteRule(Long id) { + Object record = getHibernateTemplate().load(Rule.class, id); + getHibernateTemplate().delete(record); + } + + public void deleteAllRules() { + getHibernateTemplate().bulkUpdate("delete from Rule"); + } +} Added: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/HttpRuleDao.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/HttpRuleDao.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/HttpRuleDao.java 2008-02-07 03:15:11 UTC (rev 2185) @@ -0,0 +1,66 @@ +package org.archive.accesscontrol.model; + +import java.io.IOException; + +import org.apache.commons.httpclient.HttpClient; +import org.apache.commons.httpclient.HttpMethod; +import org.apache.commons.httpclient.methods.GetMethod; + +import com.thoughtworks.xstream.XStream; + +/** + * The HTTP Rule Data Access Object enables a rule database to be queried via + * the REST interface\xCAan oracle. + * + * For details of the protocol, see: + * http://webteam.archive.org/confluence/display/wayback/Exclusions+API + * + * @author aosborne + * + */ +public class HttpRuleDao implements RuleDao { + protected HttpClient http = new HttpClient(); + protected XStream xstream = new XStream(); + private String oracleUrl; + + public HttpRuleDao(String oracleUrl) { + this.oracleUrl = oracleUrl; + xstream.alias("rule", Rule.class); + xstream.alias("ruleSet", RuleSet.class); + } + + /** + * @see RuleDao#getRuleTree(String) + */ + public RuleSet getRuleTree(String surt) { + HttpMethod method = new GetMethod(oracleUrl + "/rules/tree/" + surt); + RuleSet rules; + + try { + http.executeMethod(method); + String response = method.getResponseBodyAsString(); + System.out.println(response); + rules = (RuleSet) xstream.fromXML(method.getResponseBodyAsStream()); + } catch (IOException e) { + e.printStackTrace(); + return null; + } + method.releaseConnection(); + return rules; + } + + /** + * @return the oracleUrl + */ + public String getOracleUrl() { + return oracleUrl; + } + + /** + * @param oracleUrl the oracleUrl to set + */ + public void setOracleUrl(String oracleUrl) { + this.oracleUrl = oracleUrl; + } + +} Added: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/Rule.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/Rule.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/Rule.java 2008-02-07 03:15:11 UTC (rev 2185) @@ -0,0 +1,332 @@ +package org.archive.accesscontrol.model; + +import java.util.Calendar; +import java.util.Date; +import java.util.GregorianCalendar; +import java.util.SortedSet; + +import org.apache.commons.lang.ArrayUtils; + +/** + * An access control rules. Rules are organized into a tree based on SURT + * components. The leafmost (most specific) matching rule takes precedence. In + * the case that there are multiple matching rules for a particular node, allow + * takes precedence over deny which takes precedence over robots. + * + * @author aosborne + * + */ +public class Rule implements Comparable<Rule> { + + // in decreasing order of precedence + public static final String[] POLICIES = { "allow", "block", "robots" }; + + private Long id; + private String policy; + private String surt; + private Date captureStart; + private Date captureEnd; + private Date retrievalStart; + private Date retrievalEnd; + private Integer secondsSinceCapture; + private String who; + private String privateComment; + private String publicComment; + private Boolean enabled; + + public Rule() { + + } + + public Rule(String policy, String surt) { + super(); + this.policy = policy; + this.surt = surt; + } + + public Rule(String policy, String surt, Integer secondsSinceCapture) { + super(); + this.policy = policy; + this.surt = surt; + this.secondsSinceCapture = secondsSinceCapture; + } + + public Rule(String policy, String surt, String who) { + this(policy, surt); + this.who = who; + } + + + /** + * @return the enabled + */ + public Boolean getEnabled() { + return enabled; + } + + /** + * @param enabled the enabled to set + */ + public void setEnabled(Boolean enabled) { + this.enabled = enabled; + } + + /** + * Populate the rule data fields by copying them from a given rule. + */ + public void copyFrom(Rule rule) { + setPolicy(rule.getPolicy()); + setCaptureStart(rule.getCaptureStart()); + setCaptureEnd(rule.getCaptureEnd()); + setPrivateComment(rule.getPrivateComment()); + setPublicComment(rule.getPublicComment()); + setRetrievalStart(rule.getRetrievalStart()); + setRetrievalEnd(rule.getRetrievalEnd()); + setSecondsSinceCapture(rule.getSecondsSinceCapture()); + setSurt(rule.getSurt()); + setWho(rule.getWho()); + setEnabled(rule.getEnabled()); + } + + /** + * @return the id + */ + public Long getId() { + return id; + } + + /** + * @param id + * the id to set + */ + public void setId(Long id) { + this.id = id; + } + + /** + * @return the policy + */ + public String getPolicy() { + return policy; + } + + /** + * @param policy + * the policy to set + */ + public void setPolicy(String policy) { + this.policy = policy; + } + + /** + * @return the surt + */ + public String getSurt() { + return surt; + } + + /** + * @param surt + * the surt to set + */ + public void setSurt(String surt) { + this.surt = surt; + } + + /** + * @return the captureStart + */ + public Date getCaptureStart() { + return captureStart; + } + + /** + * @param captureStart + * the captureStart to set + */ + public void setCaptureStart(Date captureStart) { + this.captureStart = captureStart; + } + + /** + * @return the captureEnd + */ + public Date getCaptureEnd() { + return captureEnd; + } + + /** + * @param captureEnd + * the captureEnd to set + */ + public void setCaptureEnd(Date captureEnd) { + this.captureEnd = captureEnd; + } + + /** + * @return the retrievalStart + */ + public Date getRetrievalStart() { + return retrievalStart; + } + + /** + * @param retrievalStart + * the retrievalStart to set + */ + public void setRetrievalStart(Date retrievalStart) { + this.retrievalStart = retrievalStart; + } + + /** + * @return the retrievalEnd + */ + public Date getRetrievalEnd() { + return retrievalEnd; + } + + /** + * @param retrievalEnd + * the retrievalEnd to set + */ + public void setRetrievalEnd(Date retrievalEnd) { + this.retrievalEnd = retrievalEnd; + } + + /** + * @return the secondsSinceCapture + */ + public Integer getSecondsSinceCapture() { + return secondsSinceCapture; + } + + /** + * @param secondsSinceCapture + * the secondsSinceCapture to set + */ + public void setSecondsSinceCapture(Integer secondsSinceCapture) { + this.secondsSinceCapture = secondsSinceCapture; + } + + /** + * @return the who + */ + public String getWho() { + return who; + } + + /** + * @param who + * the who to set + */ + public void setWho(String who) { + this.who = who; + } + + /** + * @return the privateComment + */ + public String getPrivateComment() { + return privateComment; + } + + /** + * @param privateComment + * the privateComment to set + */ + public void setPrivateComment(String privateComment) { + this.privateComment = privateComment; + } + + /** + * @return the publicComment + */ + public String getPublicComment() { + return publicComment; + } + + /** + * @param publicComment + * the publicComment to set + */ + public void setPublicComment(String publicComment) { + this.publicComment = publicComment; + } + + public Integer getPolicyId() { + return ArrayUtils.indexOf(POLICIES, getPolicy()); + } + + /* + * Rules are sorted in descending order of "specificity". + * So we order first by SURT, then group, then policy. + */ + public int compareTo(Rule o) { + int i = getSurt().compareTo(o.getSurt()); + if (i == 0) { + if (getWho() != null && o.getWho() == null) { + i = -1; + } else if (getWho() == null && o.getWho() != null) { + i = 1; + } else { + i = getPolicyId().compareTo(o.getPolicyId()); + } + } + return i; + } + + /** + * @see #matches(String, Date, Date, String) + */ + public boolean matches(String surt) { + return surt.startsWith(getSurt()); + } + + /** + * @see #matches(String, Date, Date, String) + */ + public boolean matches(String surt, Date captureDate) { + if (captureDate == null) { + return matches(surt); + } + return matches(surt) + && (getCaptureStart() == null || captureDate.after(getCaptureStart())) + && (getCaptureEnd() == null || captureDate.before(getCaptureEnd())); + } + + /** + * @see #matches(String, Date, Date, String) + */ + public boolean matches(String surt, Date captureDate, Date retrievalDate) { + if (retrievalDate == null) { + return matches(surt, captureDate); + } + + // embargo period + if (getSecondsSinceCapture() != null) { + GregorianCalendar periodEnd = new GregorianCalendar(); + periodEnd.setTime(captureDate); + periodEnd.add(Calendar.SECOND, getSecondsSinceCapture()); + + if (retrievalDate.before(periodEnd.getTime())) { + return false; + } + } + return matches(surt, captureDate) + && (getRetrievalStart() == null || retrievalDate.after(getRetrievalStart())) + && (getRetrievalEnd() == null || retrievalDate.before(getRetrievalEnd())); + } + + /** + * Return true if the given request matches against this rule. + * + * @param surt SURT of requested document + * @param captureDate Capture date of document + * @param retrievalDate + * @param who2 Group name of requesting user + * @return + */ + public boolean matches(String surt, Date captureDate, Date retrievalDate, String who2) { + return (who == null || who == who2) && matches(surt, captureDate, retrievalDate); + } +} Added: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/RuleChange.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/RuleChange.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/RuleChange.java 2008-02-07 03:15:11 UTC (rev 2185) @@ -0,0 +1,125 @@ +package org.archive.accesscontrol.model; + +import java.util.Date; + +/** + * The oracle keeps track of the history of rules by recording each change. This + * class describes a single change (create, update or delete). + * + * @author aosborne + * + */ +public class RuleChange extends Rule { + public static final String CREATED = "created"; + public static final String UPDATED = "updated"; + public static final String DELETED = "deleted"; + + private Date changeDate; + private String changeUser; + private String changeComment; + private String changeType; + private Long ruleId; + + public RuleChange() { + super(); + } + + public RuleChange(Rule rule, String changeType, Date changeDate, + String changeUser, String changeComment) { + super(); + setRule(rule); + setChangeType(changeType); + setChangeDate(changeDate); + setChangeUser(changeUser); + setChangeComment(changeComment); + copyFrom(rule); + } + + /** + * @return the date the rule was replaced. + */ + public Date getChangeDate() { + return changeDate; + } + + /** + * @param changeDate + * the changeDate to set + */ + public void setChangeDate(Date changeDate) { + this.changeDate = changeDate; + } + + /** + * @return the user who changed the rule + */ + public String getChangeUser() { + return changeUser; + } + + /** + * @param changeUser + * the user who changed the rule + */ + public void setChangeUser(String changeUser) { + this.changeUser = changeUser; + } + + /** + * @return a comment describing the change + */ + public String getChangeComment() { + return changeComment; + } + + /** + * @param changeComment + * a comment describing the change + */ + public void setChangeComment(String changeComment) { + this.changeComment = changeComment; + } + + /** + * @param rule + * the rule to set + */ + public void setRule(Rule rule) { + if (rule == null) { + setRuleId(null); + } else { + setRuleId(rule.getId()); + } + } + + /** + * @return the changeType + */ + public String getChangeType() { + return changeType; + } + + /** + * @param changeType + * the changeType to set + */ + public void setChangeType(String changeType) { + this.changeType = changeType; + } + + /** + * @return the ruleId + */ + public Long getRuleId() { + return ruleId; + } + + /** + * @param ruleId + * the ruleId to set + */ + public void setRuleId(Long ruleId) { + this.ruleId = ruleId; + } + +} Added: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/RuleDao.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/RuleDao.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/RuleDao.java 2008-02-07 03:15:11 UTC (rev 2185) @@ -0,0 +1,29 @@ +package org.archive.accesscontrol.model; + +import org.apache.commons.httpclient.URIException; + +/** + * A RuleDao provides methods for retrieving rule information from a local database or + * remote oracle. + * + * @author aosborne + * + */ +public interface RuleDao { + + /** + * Returns the "rule tree" for a given SURT. This is a sorted set of all + * rules equal or lower in specificity than the given SURT plus all rules on + * the path from this SURT to the root SURT "(". + * + * The intention is to call this function with a domain or public suffix, + * then queries within that domain can be made very fast by searching the + * resulting list. + * + * @param surt + * @return + * @throws URIException + */ + public RuleSet getRuleTree(String surt); + +} Added: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/RuleSet.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/RuleSet.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/accesscontrol/model/RuleSet.java 2008-02-07 03:15:11 UTC (rev 2185) @@ -0,0 +1,112 @@ +package org.archive.accesscontrol.model; + +import java.util.Date; +import java.util.HashMap; +import java.util.Iterator; +import java.util.TreeSet; + +import org.archive.surt.SURTTokenizer2; + +import sun.reflect.generics.reflectiveObjects.NotImplementedException; + +/** + * A set of acess control rules which can be queried to find the governing rule + * for a particular request. + * + * @author aosborne + * + */ +public class RuleSet implements Iterable<Rule> { + protected HashMap<String, TreeSet<Rule>> rulemap = new HashMap<String, TreeSet<Rule>>(); + + class RuleSetIterator implements Iterator<Rule> { + private Iterator<TreeSet<Rule>> mapIterator; + private Iterator<Rule> setIterator; + + public RuleSetIterator() { + mapIterator = rulemap.values().iterator(); + setIterator = mapIterator.next().iterator(); + } + + public boolean hasNext() { + while (true) { + if (setIterator.hasNext()) + return true; + if (!mapIterator.hasNext()) + return false; + setIterator = mapIterator.next().iterator(); + } + } + + public Rule next() { + if (hasNext()) { + return setIterator.next(); + } + return null; + } + + public void remove() { + throw new NotImplementedException(); + } + } + + public RuleSet() { + super(); + } + + /** + * Return the most specific matching rule for the given request. + * + * @param surt + * @param captureDate + * @param retrievalDate + * @param who + * group + * @return + */ + public Rule getMatchingRule(String surt, Date captureDate, + Date retrievalDate, String who) { + + SURTTokenizer2 tok = SURTTokenizer2.newFromSURT(surt); + boolean done = false; + + while (!done) { + String key = tok.nextSearch(); + if (key == null) { + key = "("; + done = true; + } + + Iterable<Rule> rules = rulemap.get(key); + if (rules != null) { + for (Rule rule : rules) { + if (rule.matches(surt, captureDate, retrievalDate, who)) { + return rule; + } + } + } + } + return null; + } + + public void addAll(Iterable<Rule> rules) { + for (Rule rule : rules) { + add(rule); + } + } + + public void add(Rule rule) { + String surt = rule.getSurt(); + TreeSet<Rule> set = rulemap.get(surt); + if (set == null) { + set = new TreeSet<Rule>(); + rulemap.put(surt, set); + } + set.add(rule); + } + + public Iterator<Rule> iterator() { + return new RuleSetIterator(); + } + +} Added: trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/surt/SURTTokenizer2.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/surt/SURTTokenizer2.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/main/java/org/archive/surt/SURTTokenizer2.java 2008-02-07 03:15:11 UTC (rev 2185) @@ -0,0 +1,201 @@ +/* SURTTokenizer + * + * $Id: SURTTokenizer.java 4795 2006-12-12 23:42:09Z paul_jack $ + * + * Created on 3:21:49 PM May 11, 2006. + * + * Copyright (C) 2006 Internet Archive. + * + * This file is part of wayback. + * + * wayback is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * any later version. + * + * wayback is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser Public License for more details. + * + * You should have received a copy of the GNU Lesser Public License + * along with wayback; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ +package org.archive.surt; + +import java.util.Iterator; + +import org.apache.commons.httpclient.URIException; +import org.archive.net.UURI; +import org.archive.net.UURIFactory; +import org.archive.util.ArchiveUtils; +import org.archive.util.SURT; + +import sun.reflect.generics.reflectiveObjects.NotImplementedException; + +/** + * provides iterative Url reduction for prefix matching to find ever coarser + * grained URL-specific configuration. Assumes that a prefix binary search is + * being attempted for each returned value. First value is the entire SURT + * url String, with TAB appended. Second removes CGI ARGs. Then each subsequent + * path segment ('/' separated) is removed. Then the login:password, if present + * is removed. Then the port, if not :80 or omitted on the initial URL. Then + * each subsequent authority segment(. separated) is removed. + * + * the nextSearch() method will return null, finally, when no broader searches + * can be attempted on the URL. + * + * @author brad + * @version $Date: 2006-12-12 15:42:09 -0800 (Tue, 12 Dec 2006) $, $Revision: 4795 $ + */ +public class SURTTokenizer2 { + + public final static String EXACT_SUFFIX = "\t"; + private String remainder; + private boolean triedExact; + private boolean triedFull; + private boolean choppedArgs; + private boolean choppedPath; + private boolean choppedLogin; + private boolean choppedPort; + + public SURTTokenizer2() { + } + + public void setSURT(String surt) { + remainder = getKeyFromSURT(surt, false); + } + + /** + * constructor + * + * @param url String URL + * @throws URIException + */ + public SURTTokenizer2(final String url) throws URIException { + remainder = getKey(url,false); + } + /** + * update internal state and return the next smaller search string + * for the url + * + * @return string to lookup for prefix match for relevant information. + */ + public String nextSearch() { + if(!triedExact) { + triedExact = true; + //remainder = remainder.substring(0,remainder.length()-1); + return remainder + EXACT_SUFFIX; + } + if(!triedFull) { + triedFull = true; + return remainder; + } + if(!choppedArgs) { + choppedArgs = true; + int argStart = remainder.indexOf('?'); + if(argStart != -1) { + remainder = remainder.substring(0,argStart); + return remainder; + } + } + if(!choppedPath) { + int lastSlash = remainder.lastIndexOf('/'); + if(lastSlash != -1) { + remainder = remainder.substring(0,lastSlash); + if(remainder.endsWith(")")) { + remainder = remainder.substring(0,remainder.length()-1); + } + return remainder; + } + choppedPath = true; + } + if(!choppedLogin) { + choppedLogin = true; + int lastAt = remainder.lastIndexOf('@'); + if(lastAt != -1) { + remainder = remainder.substring(0,lastAt); + if(remainder.endsWith(",")) { + remainder = remainder.substring(0,remainder.length()-1); + } + return remainder; + } + } + if(!choppedPort) { + choppedPort = true; + int lastColon = remainder.lastIndexOf(':'); + if(lastColon != -1) { + remainder = remainder.substring(0,lastColon); + if(remainder.endsWith(",")) { + remainder = remainder.substring(0,remainder.length()-1); + } + return remainder; + } + } + // now just remove ','s + int lastComma = remainder.lastIndexOf(','); + if(lastComma == -1) { + return null; + } + remainder = remainder.substring(0,lastComma); + return remainder; + } + + /** + * @param url + * @return String SURT which will match exactly argument url + * @throws URIException + */ + public static String exactKey(String url) throws URIException { + return getKey(url,false); + } + + /** + * @param url + * @return String SURT which will match urls prefixed with the argument url + * @throws URIException + */ + public static String prefixKey(String url) throws URIException { + return getKey(url,true); + } + + private static String getKey(String url, boolean prefix) + throws URIException { + + String key = ArchiveUtils.addImpliedHttpIfNecessary(url); + UURI uuri = UURIFactory.getInstance(key); + key = uuri.getScheme() + "://" + uuri.getAuthority() + + uuri.getEscapedPathQuery(); + + key = SURT.fromURI(key); + key = getKeyFromSURT(key, prefix); + return key; + } + + private static String getKeyFromSURT(String surtKey, boolean prefix) { + int hashPos = surtKey.indexOf('#'); + if(hashPos != -1) { + surtKey = surtKey.substring(0,hashPos); + } + + if(surtKey.startsWith("http://")) { + surtKey = surtKey.substring(7); + } + if(prefix) { + if(surtKey.endsWith(")/")) { + surtKey = surtKey.substring(0,surtKey.length()-2); + } + } + return surtKey; + } + + /** + * Create new SURTTokenizer from a SURT rather than a URL. + */ + public static SURTTokenizer2 newFromSURT(String surt) { + SURTTokenizer2 tok = new SURTTokenizer2(); + tok.setSURT(surt); + return tok; + } +} Added: trunk/archive-access/projects/access-control/access-control/src/main/resources/Rule.hbm.xml =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/resources/Rule.hbm.xml (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/main/resources/Rule.hbm.xml 2008-02-07 03:15:11 UTC (rev 2185) @@ -0,0 +1,30 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE hibernate-mapping PUBLIC "-//Hibernate/Hibernate Mapping DTD 3.0//EN" "http://hibernate.sourceforge.net/hibernate-mapping-3.0.dtd" > +<hibernate-mapping> + <class name="org.archive.accesscontrol.model.Rule" table="rules" polymorphism="explicit"> + <id name="id" column="id" type="java.lang.Long"> + <generator class="sequence"> + <param name="sequence">rule_id_seq</param> + </generator> + + </id> + + <property name="policy" column="policy" type="java.lang.String" /> + <property name="surt" column="surt" type="text" /> + <property name="captureStart" column="capture_start" + type="java.util.Date" /> + <property name="captureEnd" column="capture_end" + type="java.util.Date" /> + <property name="retrievalStart" column="retrieval_start" + type="java.util.Date" /> + <property name="retrievalEnd" column="retrieval_end" + type="java.util.Date" /> + <property name="secondsSinceCapture" + column="seconds_since_capture" type="java.lang.Integer" /> + <property name="who" column="who" type="java.lang.String" /> + <property name="privateComment" column="private_comment" type="text" /> + <property name="publicComment" column="public_comment" type="text" /> + <property name="enabled" column="enabled" type="java.lang.Boolean" /> + + </class> +</hibernate-mapping> \ No newline at end of file Added: trunk/archive-access/projects/access-control/access-control/src/main/resources/RuleChange.hbm.xml =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/resources/RuleChange.hbm.xml (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/main/resources/RuleChange.hbm.xml 2008-02-07 03:15:11 UTC (rev 2185) @@ -0,0 +1,39 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE hibernate-mapping PUBLIC "-//Hibernate/Hibernate Mapping DTD 3.0//EN" "http://hibernate.sourceforge.net/hibernate-mapping-3.0.dtd" > +<hibernate-mapping> + <class name="org.archive.accesscontrol.model.RuleChange" table="rule_changes" polymorphism="explicit"> + <id name="id" column="id" type="java.lang.Long"> + <generator class="sequence"> + <param name="sequence">rule_change_id_seq</param> + </generator> + </id> + + <property name="ruleId" column="rule_id" type="java.lang.Long" /> + + <property name="changeDate" column="change_date" + type="java.util.Date" /> + <property name="changeUser" column="change_user" + type="java.lang.String" /> + <property name="changeComment" column="change_comment" + type="text" /> + <property name="changeType" column="change_type" + type="java.lang.String" /> + + <property name="policy" column="policy" type="java.lang.String" /> + <property name="surt" column="surt" type="text" /> + <property name="captureStart" column="capture_start" + type="java.util.Date" /> + <property name="captureEnd" column="capture_end" + type="java.util.Date" /> + <property name="retrievalStart" column="retrieval_start" + type="java.util.Date" /> + <property name="retrievalEnd" column="retrieval_end" + type="java.util.Date" /> + <property name="secondsSinceCapture" + column="seconds_since_capture" type="java.lang.Long" /> + <property name="who" column="who" type="java.lang.String" /> + <property name="privateComment" column="private_comment" type="text" /> + <property name="publicComment" column="public_comment" type="text" /> + <property name="enabled" column="enabled" type="java.lang.Boolean" /> + </class> +</hibernate-mapping> \ No newline at end of file Added: trunk/archive-access/projects/access-control/access-control/src/main/resources/hibernate.cfg.xml =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/main/resources/hibernate.cfg.xml (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/main/resources/hibernate.cfg.xml 2008-02-07 03:15:11 UTC (rev 2185) @@ -0,0 +1,26 @@ +<?xml version='1.0' encoding='utf-8'?> +<!DOCTYPE hibernate-configuration PUBLIC +"-//Hibernate/Hibernate Configuration DTD 2.0//EN" +"http://hibernate.sourceforge.net/hibernate-configuration-2.0.dtd"> +<hibernate-configuration> + <session-factory> + <property name="connection.driver_class"> + org.hsqldb.jdbcDriver + </property> + <property name="connection.url"> + jdbc:hsqldb:data/userejb + </property> + <property name="connection.username">sa</property> + <property name="connection.password"></property> + <property name="show_sql">true</property> + <property name="dialect"> + org.hibernate.dialect.HSQLDialect + </property> + <property name="transaction.factory_class"> + org.hibernate.transaction.JDBCTransactionFactory + </property> + <mapping resource="Rule.hbm.xml" /> + <mapping resource="RuleChange.hbm.xml" /> + + </session-factory> +</hibernate-configuration> Added: trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/AccessControlClientTest.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/AccessControlClientTest.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/AccessControlClientTest.java 2008-02-07 03:15:11 UTC (rev 2185) @@ -0,0 +1,28 @@ +package org.archive.accesscontrol; + +import java.util.Date; + +import org.archive.accesscontrol.AccessControlClient; +import org.archive.accesscontrol.model.HttpRuleDao; + +import junit.framework.TestCase; + +public class AccessControlClientTest extends TestCase { + public static final String ORACLE_URL = "http://localhost:8080/exclusions-oracle-0.0.1-SNAPSHOT/"; + private AccessControlClient client; + + protected void setUp() throws Exception { + super.setUp(); + client = new AccessControlClient(new HttpRuleDao(ORACLE_URL)); + } + + protected void tearDown() throws Exception { + super.tearDown(); + client = null; + } + + public void testBasicOkToShow() throws Exception { + //System.out.println(client.getPolicy("http://www.archive.org/secret/page.html", new Date(), new Date())); + } + +} Added: trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/model/DaoTestCase.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/model/DaoTestCase.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/model/DaoTestCase.java 2008-02-07 03:15:11 UTC (rev 2185) @@ -0,0 +1,16 @@ +package org.archive.accesscontrol.model; + +import org.springframework.context.ApplicationContext; +import org.springframework.context.support.ClassPathXmlApplicationContext; + +import junit.framework.TestCase; + +public abstract class DaoTestCase extends TestCase { + protected ApplicationContext ctx = null; + + public DaoTestCase() { + // Should put in a parent class that extends TestCase + String[] paths = { "applicationContext.xml" }; + ctx = new ClassPathXmlApplicationContext(paths); + } +} Added: trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/model/HibernateRuleDaoTest.java =================================================================== --- trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/model/HibernateRuleDaoTest.java (rev 0) +++ trunk/archive-access/projects/access-control/access-control/src/test/java/org/archive/accesscontrol/model/HibernateRuleDaoTest.java 2008-02-07 03:15:11 UTC (rev 2185) @@ -0,0 +1,144 @@ +package org.archive.accesscontrol.model; + +import java.util.Iterator; +import java.util.List; +import java.util.SortedSet; + +import org.archive.accesscontrol.model.HibernateRuleDao; +import org.archive.accesscontrol.model.Rule; +import org.archive.accesscontrol.model.RuleChange; + +import junit.framework.Assert; + +public class HibernateRuleDaoTest extends DaoTestCase { + private Rule rule = null; + private HibernateRuleDao dao = null; + + + protected void setUp() throws Exception { + super.setUp(); + dao = (HibernateRuleDao) ctx.getBean("ruleDao"); + + // clear database of rules + for (Rule rule: dao.getAllRules()) { + try { + dao.deleteRule(rule.getId()); + } catch (Exception e) {} + } + } + + protected void tearDown() throws Exception { + super.tearDown(); + dao = null; + } + + public void testSaveRecord() throws Exception { + rule = new Rule(); + rule.setSurt("org,archive"); + rule.setWho("admins"); + dao.saveRule(rule); + Assert.assertNotNull("primary key assigned", rule.getId()); + } + + public void testChange() throws Exception { + RuleChange change = new RuleChange(); + change.setSurt("org,archive"); + change.setWho("admins"); + dao.saveChange(change); + Assert.assertNotNull("primary key assigned", change.getId()); + } + + public void testSurtPrefixQuery() throws Exception { + rule = new Rule(); + rule.setSurt("http://(org,archive,unique)/%%__/fish"); + dao.saveRule(rule); + + Rule rule2 = new Rule(); + rule2.setSurt("http://(org,archive,unique)/blasted/fish"); + dao.saveRule(rule2); + + List<Rule> rules = dao.getRulesWithSurtPrefix("http://(org,archive,unique)/%%__"); + + Boolean foundRule1 = false; + Boolean foundRule2 = false; + for (Rule r: rules) { + if (rule.getId().e... [truncated message content] |
From: <al...@us...> - 2008-02-07 02:40:46
|
Revision: 2184 http://archive-access.svn.sourceforge.net/archive-access/?rev=2184&view=rev Author: alexoz Date: 2008-02-06 18:40:44 -0800 (Wed, 06 Feb 2008) Log Message: ----------- Created access-control project for Wayback exclusions oracle and client library. Added Paths: ----------- trunk/archive-access/projects/access-control/ This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <bra...@us...> - 2008-02-07 01:45:04
|
Revision: 2183 http://archive-access.svn.sourceforge.net/archive-access/?rev=2183&view=rev Author: bradtofel Date: 2008-02-06 17:45:08 -0800 (Wed, 06 Feb 2008) Log Message: ----------- POST-RELEASE 1.3.0 version Modified Paths: -------------- trunk/archive-access/projects/wayback/dist/pom.xml trunk/archive-access/projects/wayback/pom.xml trunk/archive-access/projects/wayback/wayback-core/pom.xml trunk/archive-access/projects/wayback/wayback-mapreduce/pom.xml trunk/archive-access/projects/wayback/wayback-mapreduce-prereq/pom.xml trunk/archive-access/projects/wayback/wayback-webapp/pom.xml Modified: trunk/archive-access/projects/wayback/dist/pom.xml =================================================================== --- trunk/archive-access/projects/wayback/dist/pom.xml 2008-02-07 01:42:05 UTC (rev 2182) +++ trunk/archive-access/projects/wayback/dist/pom.xml 2008-02-07 01:45:08 UTC (rev 2183) @@ -3,7 +3,7 @@ <parent> <groupId>org.archive</groupId> <artifactId>wayback</artifactId> - <version>1.2.0</version> + <version>1.3.0-SNAPSHOT</version> </parent> <modelVersion>4.0.0</modelVersion> @@ -54,13 +54,13 @@ <dependency> <groupId>org.archive.wayback</groupId> <artifactId>wayback-webapp</artifactId> - <version>1.2.0</version> + <version>1.3.0-SNAPSHOT</version> <type>war</type> </dependency> <dependency> <groupId>org.archive.wayback</groupId> <artifactId>wayback-mapreduce</artifactId> - <version>1.2.0</version> + <version>1.3.0-SNAPSHOT</version> </dependency> </dependencies> Modified: trunk/archive-access/projects/wayback/pom.xml =================================================================== --- trunk/archive-access/projects/wayback/pom.xml 2008-02-07 01:42:05 UTC (rev 2182) +++ trunk/archive-access/projects/wayback/pom.xml 2008-02-07 01:45:08 UTC (rev 2183) @@ -16,7 +16,7 @@ <modelVersion>4.0.0</modelVersion> <groupId>org.archive</groupId> <artifactId>wayback</artifactId> - <version>1.2.0</version> + <version>1.3.0-SNAPSHOT</version> <packaging>pom</packaging> <name>Wayback</name> Modified: trunk/archive-access/projects/wayback/wayback-core/pom.xml =================================================================== --- trunk/archive-access/projects/wayback/wayback-core/pom.xml 2008-02-07 01:42:05 UTC (rev 2182) +++ trunk/archive-access/projects/wayback/wayback-core/pom.xml 2008-02-07 01:45:08 UTC (rev 2183) @@ -17,7 +17,7 @@ <parent> <groupId>org.archive</groupId> <artifactId>wayback</artifactId> - <version>1.2.0</version> + <version>1.3.0-SNAPSHOT</version> </parent> <groupId>org.archive.wayback</groupId> <artifactId>wayback-core</artifactId> Modified: trunk/archive-access/projects/wayback/wayback-mapreduce/pom.xml =================================================================== --- trunk/archive-access/projects/wayback/wayback-mapreduce/pom.xml 2008-02-07 01:42:05 UTC (rev 2182) +++ trunk/archive-access/projects/wayback/wayback-mapreduce/pom.xml 2008-02-07 01:45:08 UTC (rev 2183) @@ -12,7 +12,7 @@ <parent> <groupId>org.archive</groupId> <artifactId>wayback</artifactId> - <version>1.2.0</version> + <version>1.3.0-SNAPSHOT</version> </parent> <groupId>org.archive.wayback</groupId> <artifactId>wayback-mapreduce</artifactId> Modified: trunk/archive-access/projects/wayback/wayback-mapreduce-prereq/pom.xml =================================================================== --- trunk/archive-access/projects/wayback/wayback-mapreduce-prereq/pom.xml 2008-02-07 01:42:05 UTC (rev 2182) +++ trunk/archive-access/projects/wayback/wayback-mapreduce-prereq/pom.xml 2008-02-07 01:45:08 UTC (rev 2183) @@ -10,7 +10,7 @@ <parent> <groupId>org.archive</groupId> <artifactId>wayback</artifactId> - <version>1.2.0</version> + <version>1.3.0-SNAPSHOT</version> </parent> <groupId>org.archive.wayback</groupId> <artifactId>wayback-mapreduce-prereq</artifactId> Modified: trunk/archive-access/projects/wayback/wayback-webapp/pom.xml =================================================================== --- trunk/archive-access/projects/wayback/wayback-webapp/pom.xml 2008-02-07 01:42:05 UTC (rev 2182) +++ trunk/archive-access/projects/wayback/wayback-webapp/pom.xml 2008-02-07 01:45:08 UTC (rev 2183) @@ -3,7 +3,7 @@ <parent> <artifactId>wayback</artifactId> <groupId>org.archive</groupId> - <version>1.2.0</version> + <version>1.3.0-SNAPSHOT</version> </parent> <modelVersion>4.0.0</modelVersion> <groupId>org.archive.wayback</groupId> This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <bra...@us...> - 2008-02-07 01:42:00
|
Revision: 2182 http://archive-access.svn.sourceforge.net/archive-access/?rev=2182&view=rev Author: bradtofel Date: 2008-02-06 17:42:05 -0800 (Wed, 06 Feb 2008) Log Message: ----------- Release 1.2.0 Added Paths: ----------- branches/wayback-1_2_0/wayback/ Copied: branches/wayback-1_2_0/wayback (from rev 2181, trunk/archive-access/projects/wayback) This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <bra...@us...> - 2008-02-07 01:39:45
|
Revision: 2181 http://archive-access.svn.sourceforge.net/archive-access/?rev=2181&view=rev Author: bradtofel Date: 2008-02-06 17:39:49 -0800 (Wed, 06 Feb 2008) Log Message: ----------- Added Paths: ----------- branches/wayback-1_2_0/ This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |