Re: ConcurrentModificationException in StepExecutionListener.saveCurrentResponseIfNeeded

Thanks John.  I'll chuck that patch into a local build and run it a few 
times to see how it reacts.

John.

On 06/17/2013 04:58 PM, John Spann wrote:
> Hi John,
>
> I was running into the same issue.  It's actually the Cookie Manager that's throwing the ConcurrentModificationException.  I believe this can occur when cookies are being set by JavaScript after WebTest thinks the step is complete and is trying to write the response to file.  I made the following change to avoid this and haven't seen it since:
>
>> svn diff StepExecutionListener.java
> Index: StepExecutionListener.java
> ===================================================================
> --- StepExecutionListener.java (revision 124264)
> +++ StepExecutionListener.java (working copy)
> @@ -1,18 +1,5 @@
>   package com.canoo.webtest.reporting;
>
> -import java.io.File;
> -import java.io.IOException;
> -import java.util.ArrayList;
> -import java.util.List;
> -import java.util.Map;
> -import java.util.Set;
> -
> -import org.apache.commons.io.FileUtils;
> -import org.apache.commons.lang.StringUtils;
> -import org.apache.log4j.Logger;
> -import org.apache.tools.ant.BuildEvent;
> -import org.apache.tools.ant.Task;
> -
>   import com.canoo.webtest.ant.IPropertyExpansionListener;
>   import com.canoo.webtest.ant.TestStepSequence;
>   import com.canoo.webtest.engine.Context;
> @@ -22,6 +9,7 @@
>   import com.canoo.webtest.engine.WebClientContext;
>   import com.canoo.webtest.steps.HtmlParserMessage;
>   import com.canoo.webtest.util.ConversionUtil;
> +import com.gargoylesoftware.htmlunit.CookieManager;
>   import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;
>   import com.gargoylesoftware.htmlunit.Page;
>   import com.gargoylesoftware.htmlunit.ScriptException;
> @@ -30,7 +18,19 @@
>   import com.gargoylesoftware.htmlunit.html.DomChangeListener;
>   import com.gargoylesoftware.htmlunit.html.HtmlPage;
>   import com.gargoylesoftware.htmlunit.util.Cookie;
> +import org.apache.commons.io.FileUtils;
> +import org.apache.commons.lang3.StringUtils;
> +import org.apache.log4j.Logger;
> +import org.apache.tools.ant.BuildEvent;
> +import org.apache.tools.ant.Task;
>
> +import java.io.File;
> +import java.io.IOException;
> +import java.util.ArrayList;
> +import java.util.List;
> +import java.util.Map;
> +import java.util.Set;
> +
>   /**
>    * Listens for task execution to extract {@link StepResult}s to generate the report.
>    *
> @@ -245,16 +245,19 @@
>    final StringBuilder sb = new StringBuilder();
>    sb.append("url=").append(resp.getWebRequest().getUrl()).append('\n');
>
> - final Set<Cookie> cookies = fContext.getWebClient().getCookieManager().getCookies();
> - sb.append("cookies=").append(cookies.size()).append('\n');
> - int i = 0;
> - for (final Cookie cookie : cookies) {
> - String prefix = "cookie." + (i++);
> - sb.append(prefix).append(".name=").append(cookie.getName()).append('\n');
> - sb.append(prefix).append(".domain=").append(cookie.getDomain()).append('\n');
> - sb.append(prefix).append(".value=").append(cookie.getValue()).append('\n');
> - sb.append(prefix).append(".path=").append(cookie.getPath()).append('\n');
> - }
> +        final CookieManager cookieManager = fContext.getWebClient().getCookieManager();
> +        synchronized (cookieManager) {
> +            final Set<Cookie> cookies = cookieManager.getCookies();
> +            sb.append("cookies=").append(cookies.size()).append('\n');
> +            int i = 0;
> +     for (final Cookie cookie : cookies) {
> +     String prefix = "cookie." + (i++);
> +     sb.append(prefix).append(".name=").append(cookie.getName()).append('\n');
> +     sb.append(prefix).append(".domain=").append(cookie.getDomain()).append('\n');
> +     sb.append(prefix).append(".value=").append(cookie.getValue()).append('\n');
> +     sb.append(prefix).append(".path=").append(cookie.getPath()).append('\n');
> +     }
> +        }
>
>    writeStringToFile(infoFile, sb.toString());
>
> Cheers,
>
> John Spann
>
> Manager, Engineering
> T: +1 805 690 3489 | M: +1 805 729 0008
> [email protected]<x-msg://29/[email protected]>
>
> [cid:38585AC2-FF0B-4CD2-A94A-F07D640E6665-lprmP+jngUqJDkItSesLSU5Kex9jXQ5X@public.gmane.org]
>
> Powering mobile workstyles and cloud services
>
>
>
>
> On Jun 17, 2013, at 1:54 AM, John Farrelly <john-TOd7PIbbG7tWk0Htik3J/[email protected]<mailto:john-TOd7PIbbG7tWk0Htik3J/[email protected]>>
>   wrote:
>
> Hi there,
>
> Apologies if this has been asked before - I couldn't find a search feature for the mailing list.
>
> When I run my suite of webtest tests, regularly one or more will fail with a ConcurrentModificationException, nearly always with the same stack trace:
>
> Caused by: java.util.ConcurrentModificationException at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372) at java.util.AbstractList$Itr.next(AbstractList.java:343) at org.apache.commons.collections.set.ListOrderedSet$OrderedSetIterator.next(ListOrderedSet.java:302) at java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1008) at com.canoo.webtest.reporting.StepExecutionListener.saveCurrentResponseIfNeeded(StepExecutionListener.java:251) at com.canoo.webtest.reporting.StepExecutionListener.taskFinished(StepExecutionListener.java:180) at org.apache.tools.ant.Project.fireTaskFinished(Project.java:2206) at org.apache.tools.ant.Task.perform(Task.java:364) at com.canoo.webtest.steps.AbstractStepContainer.executeContainedStep(AbstractStepContainer.java:72) at com.canoo.webtest.steps.AbstractStepContainer.executeContainedSteps(AbstractStepContainer.java:80) at com.canoo.webtest.steps.control.GroupStep.doExecute(GroupStep.java:14) at com.canoo.webtest.steps.Step.execute(Step.java:102)
>
> Different tests will fail with this randomly, which makes me think is a threading / race condition issue in webtest.  Is there a solution to this, or any way to stop it from happening?  I'm using R_1823
>
> Thanks,
> John.
>
>