[HtmlUnit] [htmlunit:bugs] #1799 HtmlAnchor click() and getPage() not working

This is a multi-part message in MIME format.
--===============7155604490240765625==
Content-Type: multipart/related;
 boundary="===============4210269071683773649=="

This is a multi-part message in MIME format.
--===============4210269071683773649==
Content-Type: multipart/alternative;
 boundary="===============1283120535136054195=="
MIME-Version: 1.0

--===============1283120535136054195==
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit

test was moved to XMLHttpRequest2Test.readPropertyFromPrototypeShouldThrow()


---

** [bugs:#1799] HtmlAnchor click() and getPage() not working**

**Status:** accepted
**Group:** 2.22
**Created:** Sun Jun 12, 2016 07:50 PM UTC by Timothy Nessus
**Last Updated:** Mon Jun 13, 2016 04:28 PM UTC
**Owner:** nobody


I'm trying to analyze a website (shop.lego.com), but when using either click or getPage on the anchor (corresponding to the text "View All") the HtmlPage produced is the same as the original page. I believe the problem has to do with a HtmlUnit running a javascript on the page (javascript is enabled on the webclient).

The specific webpage is http://shop.lego.com/en-CA/Sales-And-Deals. I create a webClient with the following settings:

~~~
WebClient webClient = new WebClient(BrowserVersion.FIREFOX_38);
webClient.getOptions().setThrowExceptionOnScriptError(false);
webClient.getOptions().setJavaScriptEnabled(true);
webClient.getOptions().setActiveXNative(true);
webClient.getOptions().setAppletEnabled(true);
webClient.getOptions().setActiveXNative(true);
webClient.getOptions().setCssEnabled(true);
webClient.getOptions().setDoNotTrackEnabled(false);
webClient.getOptions().setGeolocationEnabled(true);
webClient.getOptions().setPopupBlockerEnabled(false);
webClient.getOptions().setPrintContentOnFailingStatusCode(true);
webClient.getOptions().setRedirectEnabled(true);
webClient.getCache().clear();
webClient.getCache().setMaxSize(0);
webClient.setAjaxController(new NicelyResynchronizingAjaxController());
webClient.getCookieManager().setCookiesEnabled(true);
~~~
    
Then I open the page (I put loop in place to ensure that the page returned is an Html page), and retrieve the anchor.
    
~~~
final URL url = new URL("http://shop.lego.com/en-CA/Sales-And-Deals");

int count = 0;
Page pageOri = webClient.getPage(url);
while (count < 5 && !(pageOri instanceof HtmlPage)) {
    pageOri = webClient.getPage(url);
    count++;
    System.out.println("Read Attempt #" + count + " failed.");
}
final HtmlPage page = (HtmlPage) pageOri;
HtmlAnchor choosen = page.getAnchorByText("View All");
~~~
  
  Then I click on the anchor and print the text representation, but the text representation is the same as the original.
  
~~~
HtmlPage allResults = (HtmlPage) choosen.getPage();
System.out.println(allResults.asText());
~~~



---

Sent from sourceforge.net because [email protected] is subscribed to https://sourceforge.net/p/htmlunit/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/htmlunit/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.
--===============1283120535136054195==
MIME-Version: 1.0
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: 7bit

<div class="markdown_content"><p>test was moved to XMLHttpRequest2Test.readPropertyFromPrototypeShouldThrow()</p>
<hr/>
<p><strong> <a class="alink" href="https://sourceforge.net/p/htmlunit/bugs/1799/">[bugs:#1799]</a> HtmlAnchor click() and getPage() not working</strong></p>
<p><strong>Status:</strong> accepted<br/>
<strong>Group:</strong> 2.22<br/>
<strong>Created:</strong> Sun Jun 12, 2016 07:50 PM UTC by Timothy Nessus<br/>
<strong>Last Updated:</strong> Mon Jun 13, 2016 04:28 PM UTC<br/>
<strong>Owner:</strong> nobody</p>
<p>I'm trying to analyze a website (shop.lego.com), but when using either click or getPage on the anchor (corresponding to the text "View All") the HtmlPage produced is the same as the original page. I believe the problem has to do with a HtmlUnit running a javascript on the page (javascript is enabled on the webclient).</p>
<p>The specific webpage is <a href="http://shop.lego.com/en-CA/Sales-And-Deals." rel="nofollow">http://shop.lego.com/en-CA/Sales-And-Deals.</a> I create a webClient with the following settings:</p>
<div class="codehilite"><pre><span></span>WebClient webClient = new WebClient(BrowserVersion.FIREFOX_38);
webClient.getOptions().setThrowExceptionOnScriptError(false);
webClient.getOptions().setJavaScriptEnabled(true);
webClient.getOptions().setActiveXNative(true);
webClient.getOptions().setAppletEnabled(true);
webClient.getOptions().setActiveXNative(true);
webClient.getOptions().setCssEnabled(true);
webClient.getOptions().setDoNotTrackEnabled(false);
webClient.getOptions().setGeolocationEnabled(true);
webClient.getOptions().setPopupBlockerEnabled(false);
webClient.getOptions().setPrintContentOnFailingStatusCode(true);
webClient.getOptions().setRedirectEnabled(true);
webClient.getCache().clear();
webClient.getCache().setMaxSize(0);
webClient.setAjaxController(new NicelyResynchronizingAjaxController());
webClient.getCookieManager().setCookiesEnabled(true);
</pre></div>


<p>Then I open the page (I put loop in place to ensure that the page returned is an Html page), and retrieve the anchor.</p>
<div class="codehilite"><pre><span></span><span class="nt">final</span> <span class="nt">URL</span> <span class="nt">url</span> <span class="o">=</span> <span class="nt">new</span> <span class="nt">URL</span><span class="o">(</span><span class="s2">"http://shop.lego.com/en-CA/Sales-And-Deals"</span><span class="o">);</span>

<span class="nt">int</span> <span class="nt">count</span> <span class="o">=</span> <span class="nt">0</span><span class="o">;</span>
<span class="nt">Page</span> <span class="nt">pageOri</span> <span class="o">=</span> <span class="nt">webClient</span><span class="p">.</span><span class="nc">getPage</span><span class="o">(</span><span class="nt">url</span><span class="o">);</span>
<span class="nt">while</span> <span class="o">(</span><span class="nt">count</span> <span class="o">&lt;</span> <span class="nt">5</span> <span class="o">&amp;&amp;</span> <span class="o">!(</span><span class="nt">pageOri</span> <span class="nt">instanceof</span> <span class="nt">HtmlPage</span><span class="o">))</span> <span class="p">{</span>
    <span class="err">pageOri</span> <span class="err">=</span> <span class="err">webClient.getPage(url)</span><span class="p">;</span>
    <span class="err">count++</span><span class="p">;</span>
    <span class="err">System.out.println("Read</span> <span class="err">Attempt</span> <span class="err">#"</span> <span class="err">+</span> <span class="err">count</span> <span class="err">+</span> <span class="err">"</span> <span class="err">failed.")</span><span class="p">;</span>
<span class="p">}</span>
<span class="nt">final</span> <span class="nt">HtmlPage</span> <span class="nt">page</span> <span class="o">=</span> <span class="o">(</span><span class="nt">HtmlPage</span><span class="o">)</span> <span class="nt">pageOri</span><span class="o">;</span>
<span class="nt">HtmlAnchor</span> <span class="nt">choosen</span> <span class="o">=</span> <span class="nt">page</span><span class="p">.</span><span class="nc">getAnchorByText</span><span class="o">(</span><span class="s2">"View All"</span><span class="o">);</span>
</pre></div>


<p>Then I click on the anchor and print the text representation, but the text representation is the same as the original.</p>
<div class="codehilite"><pre><span></span>HtmlPage allResults = (HtmlPage) choosen.getPage();
System.out.println(allResults.asText());
</pre></div>


<hr/>
<p>Sent from sourceforge.net because [email protected] is subscribed to <a href="https://sourceforge.net/p/htmlunit/bugs/">https://sourceforge.net/p/htmlunit/bugs/</a></p>
<p>To unsubscribe from further messages, a project admin can change settings at <a href="https://sourceforge.net/p/htmlunit/admin/bugs/options.">https://sourceforge.net/p/htmlunit/admin/bugs/options.</a>  Or, if this is a mailing list, you can unsubscribe from the mailing list.</p></div>
--===============1283120535136054195==--
--===============4210269071683773649==--


--===============7155604490240765625==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline


--===============7155604490240765625==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
HtmlUnit-develop mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/htmlunit-develop

--===============7155604490240765625==--