Re: ConcurrentModificationException in StepExecutionListener.saveCurrentResponseIfNeeded

Hi John,

I was running into the same issue.  It's actually the Cookie Manager that's throwing the ConcurrentModificationException.  I believe this can occur when cookies are being set by JavaScript after WebTest thinks the step is complete and is trying to write the response to file.  I made the following change to avoid this and haven't seen it since:

>svn diff StepExecutionListener.java
Index: StepExecutionListener.java
===================================================================
--- StepExecutionListener.java (revision 124264)
+++ StepExecutionListener.java (working copy)
@@ -1,18 +1,5 @@
 package com.canoo.webtest.reporting;

-import java.io.File;
-import java.io.IOException;
-import java.util.ArrayList;
-import java.util.List;
-import java.util.Map;
-import java.util.Set;
-
-import org.apache.commons.io.FileUtils;
-import org.apache.commons.lang.StringUtils;
-import org.apache.log4j.Logger;
-import org.apache.tools.ant.BuildEvent;
-import org.apache.tools.ant.Task;
-
 import com.canoo.webtest.ant.IPropertyExpansionListener;
 import com.canoo.webtest.ant.TestStepSequence;
 import com.canoo.webtest.engine.Context;
@@ -22,6 +9,7 @@
 import com.canoo.webtest.engine.WebClientContext;
 import com.canoo.webtest.steps.HtmlParserMessage;
 import com.canoo.webtest.util.ConversionUtil;
+import com.gargoylesoftware.htmlunit.CookieManager;
 import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;
 import com.gargoylesoftware.htmlunit.Page;
 import com.gargoylesoftware.htmlunit.ScriptException;
@@ -30,7 +18,19 @@
 import com.gargoylesoftware.htmlunit.html.DomChangeListener;
 import com.gargoylesoftware.htmlunit.html.HtmlPage;
 import com.gargoylesoftware.htmlunit.util.Cookie;
+import org.apache.commons.io.FileUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.log4j.Logger;
+import org.apache.tools.ant.BuildEvent;
+import org.apache.tools.ant.Task;

+import java.io.File;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
 /**
  * Listens for task execution to extract {@link StepResult}s to generate the report.
  *
@@ -245,16 +245,19 @@
  final StringBuilder sb = new StringBuilder();
  sb.append("url=").append(resp.getWebRequest().getUrl()).append('\n');

- final Set<Cookie> cookies = fContext.getWebClient().getCookieManager().getCookies();
- sb.append("cookies=").append(cookies.size()).append('\n');
- int i = 0;
- for (final Cookie cookie : cookies) {
- String prefix = "cookie." + (i++);
- sb.append(prefix).append(".name=").append(cookie.getName()).append('\n');
- sb.append(prefix).append(".domain=").append(cookie.getDomain()).append('\n');
- sb.append(prefix).append(".value=").append(cookie.getValue()).append('\n');
- sb.append(prefix).append(".path=").append(cookie.getPath()).append('\n');
- }
+        final CookieManager cookieManager = fContext.getWebClient().getCookieManager();
+        synchronized (cookieManager) {
+            final Set<Cookie> cookies = cookieManager.getCookies();
+            sb.append("cookies=").append(cookies.size()).append('\n');
+            int i = 0;
+     for (final Cookie cookie : cookies) {
+     String prefix = "cookie." + (i++);
+     sb.append(prefix).append(".name=").append(cookie.getName()).append('\n');
+     sb.append(prefix).append(".domain=").append(cookie.getDomain()).append('\n');
+     sb.append(prefix).append(".value=").append(cookie.getValue()).append('\n');
+     sb.append(prefix).append(".path=").append(cookie.getPath()).append('\n');
+     }
+        }

  writeStringToFile(infoFile, sb.toString());

Cheers,

John Spann

Manager, Engineering
T: +1 805 690 3489 | M: +1 805 729 0008
[email protected]<x-msg://29/[email protected]>

[cid:38585AC2-FF0B-4CD2-A94A-F07D640E6665-lprmP+jngUqJDkItSesLSU5Kex9jXQ5X@public.gmane.org]

Powering mobile workstyles and cloud services




On Jun 17, 2013, at 1:54 AM, John Farrelly <john-TOd7PIbbG7tWk0Htik3J/[email protected]<mailto:[email protected]>>
 wrote:

Hi there,

Apologies if this has been asked before - I couldn't find a search feature for the mailing list.

When I run my suite of webtest tests, regularly one or more will fail with a ConcurrentModificationException, nearly always with the same stack trace:

Caused by: java.util.ConcurrentModificationException at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372) at java.util.AbstractList$Itr.next(AbstractList.java:343) at org.apache.commons.collections.set.ListOrderedSet$OrderedSetIterator.next(ListOrderedSet.java:302) at java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1008) at com.canoo.webtest.reporting.StepExecutionListener.saveCurrentResponseIfNeeded(StepExecutionListener.java:251) at com.canoo.webtest.reporting.StepExecutionListener.taskFinished(StepExecutionListener.java:180) at org.apache.tools.ant.Project.fireTaskFinished(Project.java:2206) at org.apache.tools.ant.Task.perform(Task.java:364) at com.canoo.webtest.steps.AbstractStepContainer.executeContainedStep(AbstractStepContainer.java:72) at com.canoo.webtest.steps.AbstractStepContainer.executeContainedSteps(AbstractStepContainer.java:80) at com.canoo.webtest.steps.control.GroupStep.doExecute(GroupStep.java:14) at com.canoo.webtest.steps.Step.execute(Step.java:102)

Different tests will fail with this randomly, which makes me think is a threading / race condition issue in webtest.  Is there a solution to this, or any way to stop it from happening?  I'm using R_1823

Thanks,
John.