Re: Two possible refinements
"James Courtney" <[email protected]> Thu, 18 Sep 2008 09:02:48 -0700
| Newsgroups | gmane.comp.web.httpunit.devel |
|---|---|
| Message-ID | <026a01c919a8$0034f300$009ed900$@com> |
Just wondering if anyone has had a chance to look at my proposed changes. Thanks! Jamey -----Original Message----- From: James Courtney [mailto:[email protected]] Sent: Monday, August 18, 2008 10:59 PM To: 'Discussion of use and development of HttpUnit' Subject: RE: [Httpunit-develop] Two possible refinements Here's the patch file. I'd love some feedback on the changes I made for redirection loop detection/handling. 1) I added two ClientProperties properties to control this. The first (resolveRedirectLoops) tells the code to either utilize the standard URL comparison (with name resolution) for detecting loops or to ignore this and simply string compare the domain names. The second property specifies a maximum number of redirects in the chain defaulting to 10 and settable to -1 (or any negative value) to allow infinite. 2) I also changed the exception handling slightly by introducing the RedirectionException (which RecursiveRedirectionException now extends) for throwing if the number of redirects is exceeded or if the URL created for the redirection is invalid. The update for taking into account strict (or lax) domain matching for cookies was much simpler and I was able to easily include some unit tests for this. The code for redirection loop handling I'm not sure how to create unit tests for BUT the current unit tests all run save the two that were failing even with fresh code checked out and the unit tests for my application which depends on HTTPUnit and which originally triggered the issues I saw now runs fine. There I was able to test enabling and disabling the ClientProperties parameters mentioned and these behaved as expected. The risky changes I see for current users are: 1) The exception handling change which still throws just a subclass of RuntimeException but now RedirectionException in addition to RecursiveRedirectionException. You might make the argument that the RecursiveRedirectionException subclass is excessive and could simply be reduced to a RedirectionException with thrown with the correct message. This might break some existing code BUT it's explicit and forces the developer, if they ever cared about handling those exceptions, to note the change and make a small adaptation. 2) Defaulting to simple string comparison for redirection loop detection vs. the previous URL.equals which attempted to resolve the domain names and compare IPs. While the default of not doing the resolution comparison seems more sensible to me it is a change in behavior and maybe the existing behavior should continue as the default. 3) The cap on redirections by quantity currently defaulted to 10 in my code. This seems reasonable to me but who knows what silly stuff is out there and may break with this new restriction. This could be defaulted to -1 to agree with current behavior. Thanks for your consideration and assistance. Jamey -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Wolfgang Fahl - BITPlan Sent: Monday, August 18, 2008 1:02 AM To: Discussion of use and development of HttpUnit Subject: Re: [Httpunit-develop] Two possible refinements James wrote: > Thank you Russell. I'll take a look at making the specific changes > and including unit test code for submission in the next day or two. I trust you want patches generated by something like the following: > svn diff File.java >> patchfile.txt The area you have commented on has been part a few of the patch-proposals of the 1.7 release. Some of the proposals looked conflicting to me so not everything was implemented as the suggestions in the patch-proposals asked for. It's good that you are going to clarify the situation. You might want to not hesitate to contact me for the patch submission. I have volunteered to do commit work like this since last year. Wolfgang BITPlan - smart solutions Pater-Delp-Str. 1, D-47877 Willich-Schiefbahn Tel. +49 1805 - BITPLAN / +49 1805 248 752, Fax +49 2154 811-481 Web: http://www.bitplan.de bitplan GmbH, Willich - HRB 6820 Krefeld, VAT-ID: 10258040584, Geschäftsführer: Wolfgang Fahl Mit BITPlan effizient zum Erfolg: http://www.bitplan.de ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ Httpunit-develop mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/httpunit-develop No virus found in this incoming message. Checked by AVG - http://www.avg.com Version: 8.0.138 / Virus Database: 270.6.5/1618 - Release Date: 8/18/2008 6:51 AM No virus found in this incoming message. Checked by AVG - http://www.avg.com Version: 8.0.169 / Virus Database: 270.6.19/1664 - Release Date: 9/17/2008 9:33 AM ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ Httpunit-develop mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/httpunit-develop
patchfile.txt
(text/plain, 16.8 KB)
Index: test/com/meterware/httpunit/cookies/CookieTest.java
===================================================================
--- test/com/meterware/httpunit/cookies/CookieTest.java (revision 989)
+++ test/com/meterware/httpunit/cookies/CookieTest.java (working copy)
@@ -88,6 +88,9 @@
checkMatching( 5, true, new URL( "http://www.meterware.com/servlets/sample" ), "www.meterware.com", "/servlets" );
checkMatching( 6, false, new URL( "http://www.meterware.com/servlets/sample" ), "www.meterware.com", "/servlets/sample/data" );
+
+ checkMatching( 7, true, new URL( "http://www.meterware.com/servlets/sample" ), "www.MeterWare.com", "/servlets/sample" );
+ checkMatching( 8, true, new URL( "http://WWW.meterware.com/servlets/sample" ), ".MeterWare.com", "/servlets/sample" );
}
@@ -284,6 +287,7 @@
checkAcceptance( 1, true, "www.some.meterware.com/servlets/special", ".meterware.com", null );
checkAcceptance( 2, false, "www.meterware.com/servlets/special", ".meterware.com", "/servlets/ordinary" );
checkAcceptance( 3, true, "www.meterware.com/servlets/special", "www.meterware.com", null );
+ checkAcceptance( 1, true, "www.some.meterware.com/servlets/special", "meterware.com", null );
CookieProperties.setPathMatchingStrict( false );
checkAcceptance( 11, true, "www.meterware.com/servlets/special", ".meterware.com", "/servlets/ordinary" );
Index: src/com/meterware/httpunit/cookies/Cookie.java
===================================================================
--- src/com/meterware/httpunit/cookies/Cookie.java (revision 989)
+++ src/com/meterware/httpunit/cookies/Cookie.java (working copy)
@@ -247,8 +247,9 @@
* @return true if there is a fit
*/
private static boolean acceptHost( String hostPattern, String hostName ) {
- return hostPattern.equalsIgnoreCase( hostName ) ||
- (hostPattern.startsWith( "." ) && hostName.endsWith( hostPattern ));
+ return !CookieProperties.isDomainMatchingStrict() ||
+ hostPattern.equalsIgnoreCase( hostName ) ||
+ (hostPattern.startsWith( "." ) && hostName.toLowerCase().endsWith(hostPattern.toLowerCase()));
}
Index: src/com/meterware/httpunit/ClientProperties.java
===================================================================
--- src/com/meterware/httpunit/ClientProperties.java (revision 989)
+++ src/com/meterware/httpunit/ClientProperties.java (working copy)
@@ -180,6 +180,48 @@
/**
+ * Returns the maximum redirects to be followed in a chain before erring out.
+ * By default, this is 10.
+ **/
+ public int getMaxRedirects() {
+ return _maxRedirects;
+ }
+
+
+ /**
+ * Sets the maximum number of redirects the client should follow in a chain
+ * before erring out. -1 tells the client to follow an infinite number of
+ * redirects in the chain, 10 is the default. 0 has the effect of failing on
+ * any and all redirects.
+ **/
+ public void setMaxRedirects( int maxRedirects ) {
+ _maxRedirects = maxRedirects;
+ }
+
+
+ /**
+ * Returns true if the client should resolve redirect domain names in a
+ * redirect chain and compare by IP address rather than simply domain name
+ * during redirect loop detection. If the domain names cannot be resolved
+ * the fall-back is a string comparison of domain names.
+ **/
+ public boolean resolveRedirectLoops() {
+ return _resolveRedirectLoops;
+ }
+
+
+ /**
+ * Determines whether the client should resolve redirect domain names
+ * when comparing for the detection of redirect loops. If true domain
+ * names are resolved and the IPs compared. If the resolution is not
+ * successful the domain names are string compared as a fall-back.
+ **/
+ public void setResolveRedirectLoops( boolean resolveRedirectLoops ) {
+ _resolveRedirectLoops = resolveRedirectLoops;
+ }
+
+
+ /**
* Returns true if the client should automatically follow page refresh requests.
* By default, this is false, so that programs can verify the redirect page presented
* to users before the browser switches to the new page.
@@ -269,20 +311,22 @@
}
- private String _applicationCodeName = "httpunit";
- private String _applicationName = "HttpUnit";
- private String _applicationVersion = "1.5";
+ private String _applicationCodeName = "httpunit";
+ private String _applicationName = "HttpUnit";
+ private String _applicationVersion = "1.5";
private String _userAgent;
- private String _platform = "Java";
- private String _overrideContextType = null;
- private int _availWidth = 800;
- private int _availHeight = 600;
+ private String _platform = "Java";
+ private String _overrideContextType = null;
+ private int _availWidth = 800;
+ private int _availHeight = 600;
- private boolean _iframeSupported = true;
- private boolean _acceptCookies = true;
- private boolean _acceptGzip = true;
- private boolean _autoRedirect = true;
- private boolean _autoRefresh = false;
+ private boolean _iframeSupported = true;
+ private boolean _acceptCookies = true;
+ private boolean _acceptGzip = true;
+ private boolean _autoRedirect = true;
+ private int _maxRedirects = 10;
+ private boolean _resolveRedirectLoops = false;
+ private boolean _autoRefresh = false;
private DNSListener _dnsListener;
private boolean _sendReferer;
@@ -304,18 +348,20 @@
* @param source - the ClientProperties to copy from
*/
private ClientProperties( ClientProperties source ) {
- _applicationCodeName = source._applicationCodeName;
- _applicationName = source._applicationName;
- _applicationVersion = source._applicationVersion;
- _userAgent = source._userAgent;
- _platform = source._platform;
- _overrideContextType = source._overrideContextType;
- _iframeSupported = source._iframeSupported;
- _acceptCookies = source._acceptCookies;
- _acceptGzip = source._acceptGzip;
- _autoRedirect = source._autoRedirect;
- _autoRefresh = source._autoRefresh;
- _sendReferer = source._sendReferer;
+ _applicationCodeName = source._applicationCodeName;
+ _applicationName = source._applicationName;
+ _applicationVersion = source._applicationVersion;
+ _userAgent = source._userAgent;
+ _platform = source._platform;
+ _overrideContextType = source._overrideContextType;
+ _iframeSupported = source._iframeSupported;
+ _acceptCookies = source._acceptCookies;
+ _acceptGzip = source._acceptGzip;
+ _autoRedirect = source._autoRedirect;
+ _maxRedirects = source._maxRedirects;
+ _resolveRedirectLoops = source._resolveRedirectLoops;
+ _autoRefresh = source._autoRefresh;
+ _sendReferer = source._sendReferer;
}
Index: src/com/meterware/httpunit/WebWindow.java
===================================================================
--- src/com/meterware/httpunit/WebWindow.java (revision 989)
+++ src/com/meterware/httpunit/WebWindow.java (working copy)
@@ -23,9 +23,9 @@
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
-import java.util.HashSet;
+import java.util.HashMap;
import java.util.List;
-import java.util.Set;
+import java.util.Map;
import org.xml.sax.SAXException;
import com.meterware.httpunit.scripting.ScriptingHandler;
@@ -60,7 +60,7 @@
* of a single client-initiated request
* @since patch [ 1155415 ] Handle redirect instructions which can lead to a loop
*/
- private final Set _redirects;
+ private final Map _redirects;
/** True if seen initial request
* @since patch [ 1155415 ] Handle redirect instructions which can lead to a loop
@@ -300,7 +300,7 @@
_client = client;
_frameContents = new FrameHolder( this );
_name = NO_NAME + _client.getOpenWindows().length;
- _redirects = new HashSet();
+ _redirects = new HashMap();
}
@@ -350,7 +350,7 @@
}
/**
- * check wether we should follow the redirect given in the response
+ * check whether we should follow the redirect given in the response
* make sure we don't run into a recursion
* @param response
* @return
@@ -371,22 +371,42 @@
url = new URL(response.getURL(), redirectLocation);
}
} catch (MalformedURLException e) {
- // Fall through and allow existing exception handling code deal
- // with any exception - we don't know at this stage whether it is
- // a redirect instruction, although it is highly likely, given
- // there is a location header present in the response!
+ throw new RedirectionException(response.getURL(),
+ "Error constructing redirect URL using location '"
+ + redirectLocation + "'.", e);
+
}
switch (response.getResponseCode()) {
case HttpURLConnection.HTTP_MOVED_PERM:
case HttpURLConnection.HTTP_MOVED_TEMP: // Fall through
- if (null != url && _redirects.contains(url)) {
- // We have already been instructed to redirect to that location in
- // the course of this attempt to resolve the resource
- throw new RecursiveRedirectionException(url,
+ ClientProperties props = _client.getClientProperties();
+
+ // First check for an excessive number of redirects
+ if ((props.getMaxRedirects() >= 0) &&
+ (_redirects.size() >= props.getMaxRedirects()))
+ {
+ throw new RedirectionException(url,
+ "Maximum redirection chain length of " + props.getMaxRedirects() + " exceeded.");
+ }
+
+ // Now check for repeat URLs
+ if (null != url)
+ {
+ URL prev = (URL) _redirects.put(url, url);
+ // if we have a URL comparison match
+ // if resolving redirects we're done
+ // else be a little more lax and just compare host names
+ if ((prev != null) &&
+ (props.resolveRedirectLoops() ||
+ url.getHost().equalsIgnoreCase(prev.getHost())))
+ {
+ // We have already been instructed to redirect to that location in
+ // the course of this attempt to resolve the resource
+ throw new RecursiveRedirectionException(url,
"Unable to process request due to redirection loop");
+ }
}
- _redirects.add(url);
break;
}
return redirectLocation != null;
Index: src/com/meterware/httpunit/RedirectionException.java
===================================================================
--- src/com/meterware/httpunit/RedirectionException.java (revision 0)
+++ src/com/meterware/httpunit/RedirectionException.java (revision 0)
@@ -0,0 +1,100 @@
+package com.meterware.httpunit;
+
+/********************************************************************************************************************
+ * $Id: RedirectionException.java 908 2008-04-05 08:24:51Z wolfgang_fahl $
+ *
+ * Copyright (c) 2002-2008, Russell Gold
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
+ * documentation files (the "Software"), to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and
+ * to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in all copies or substantial portions
+ * of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO
+ * THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF
+ * CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ *
+ *******************************************************************************************************************/
+import java.net.URL;
+
+/**
+ * Class used to indicate when a request to a resource resulted in an HTTP
+ * redirect response which fails.
+ *
+ * @author <a href="mailto:[email protected]">James Abley </a>
+ */
+public class RedirectionException extends RuntimeException {
+
+ private URL url;
+
+ /**
+ * Create a new <code>RedirectionException</code> with the
+ * specified URL and cause.
+ *
+ * @param url
+ * the <code>URL</code> that caused the redirection issue.
+ * The URL is saved for later retrieval by
+ * {@link #getURL()}
+ * @param cause
+ * the cause (which is saved for later retrieval by the
+ * {@link #getCause()}method). (A null value is permitted, and
+ * indicates that the cause is nonexistent or unknown.)
+ */
+ public RedirectionException(URL url, Throwable cause) {
+ super(cause);
+ this.url = url;
+ }
+
+ /**
+ * Create a new <code>RedirectionException</code> with the
+ * specified URL and detail message.
+ *
+ * @param url
+ * the <code>URL</code> that caused the redirection issue.
+ * The URL is saved for later retrieval by
+ * {@link #getURL()}
+ * @param message
+ * the detail message. The detail message is saved for later
+ * retrieval by {@link #getMessage()}
+ */
+ public RedirectionException(URL url, String message) {
+ super(message);
+ this.url = url;
+ }
+
+ /**
+ * Create a new <code>RedirectionException</code> with the
+ * specified URL, detail message and cause.
+ *
+ * @param url
+ * the <code>URL</code> that caused the redirection issue.
+ * The URL is saved for later retrieval by
+ * {@link #getURL()}
+ * @param message
+ * the detail message. The detail message is saved for later
+ * retrieval by {@link #getMessage()}
+ * @param cause
+ * the cause (which is saved for later retrieval by the
+ * {@link #getCause()}method). (A null value is permitted, and
+ * indicates that the cause is nonexistent or unknown.)
+ */
+ public RedirectionException(URL url, String message,
+ Throwable cause) {
+ super(message, cause);
+ this.url = url;
+ }
+
+ /**
+ * Returns the URL that caused this exception to be thrown.
+ *
+ * @return the <code>URL</code> that gave rise to this Exception
+ */
+ public URL getURL() {
+ return url;
+ }
+}
Index: src/com/meterware/httpunit/RecursiveRedirectionException.java
===================================================================
--- src/com/meterware/httpunit/RecursiveRedirectionException.java (revision 989)
+++ src/com/meterware/httpunit/RecursiveRedirectionException.java (working copy)
@@ -28,10 +28,8 @@
*
* @author <a href="mailto:[email protected]">James Abley </a>
*/
-public class RecursiveRedirectionException extends RuntimeException {
+public class RecursiveRedirectionException extends RedirectionException {
- private URL url;
-
/**
* Create a new <code>RecursiveRedirectionException</code> with the
* specified URL and cause.
@@ -44,8 +42,7 @@
* indicates that the cause is nonexistent or unknown.)
*/
public RecursiveRedirectionException(URL url, Throwable cause) {
- super(cause);
- this.url = url;
+ super(url, cause);
}
/**
@@ -61,8 +58,7 @@
* retrieval by {@link #getMessage()}
*/
public RecursiveRedirectionException(URL url, String message) {
- super(message);
- this.url = url;
+ super(url, message);
}
/**
@@ -83,16 +79,6 @@
*/
public RecursiveRedirectionException(URL url, String message,
Throwable cause) {
- super(message, cause);
- this.url = url;
+ super(url, message, cause);
}
-
- /**
- * Returns the URL that caused this exception to be thrown.
- *
- * @return the <code>URL</code> that gave rise to this Exception
- */
- public URL getURL() {
- return url;
- }
}