Re: Httpunit Support for document.write
"Wolfgang Fahl - BITPlan" <[email protected]> Fri, 02 May 2008 10:14:52 +0200
| Newsgroups | gmane.comp.web.httpunit.devel |
|---|---|
| Organization | bitplan GmbH |
| Message-ID | <[email protected]> |
Catalin, thank you for your excellent description of this problem. Would you please create a JUnit testcase from your code and make it available via this mailinglist or via the patch or bug tracker at: https://sourceforge.net/projects/httpunit If you use the tracker you can be pretty sure that your suggestion will be worked on in due time. It is very good input for the scripting engine issues that we are going to work on after the 1.7 release is out. Yours Wolfgang Date sent: Sat, 26 Apr 2008 19:01:21 -0400 From: "Catalin Patulea" <[email protected]> To: [email protected] Subject: [Httpunit-develop] Httpunit Support for document.write Send reply to: Discussion of use and development of HttpUnit <httpunit- [email protected]> <httpunit-develop.lists.sourceforge.net> <mailto:[email protected]?subject=unsubscribe> <mailto:[email protected]?subject=subscribe> > Hi, > > I am trying to use Httpunit to scrape information off a local bus > operator's web site. I realize that httpunit wasn't really meant for > that, but it's also the best tool I've found for the job so far. I > have tried asking nicely for the information, but they were > uncooperative. With that in mind, here is the problem I am having: > > The web site uses <script> blocks and document.write's to generate > <option> tags within a <select>. Httpunit seems to only apply content > generated from document.write to replace the entire page, and only > once document.close has been called. For example: > <html><head><title>httpunit-test</title></head><body> > <a href="http://www.google.ca/">Link 1</a> > <script language="javascript"> > <!-- > document.writeln("<a href='http://www.google.ca'>Link 2<" + "/a>"); > document.close(); > --> > </script> > <a href="http://www.google.ca/">Link 3</a> > </body></html> > > In Httpunit, the page above generates only one link, Link 2 (as > inspected via WebResponse.getLinks). If I remove document.close, I see > only Link 1 and Link 3. > > The expected behaviour is to see all three links, in the correct order > (this is what Firefox does, for instance). > > Since my only objective is scraping the site, I can always just put in > a small hack in my code. But I would like to at least try to do it the > right way first. > > Thanks, > > Catalin > > ------------------------------------------------------------------------- > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > Don't miss this year's exciting event. There's still time to save $100. > Use priority code J8TL2D2. > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone > _______________________________________________ > Httpunit-develop mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/httpunit-develop BITPlan - smart solutions Pater-Delp-Str. 1, D-47877 Willich Schiefbahn Tel. +49 1805 - BITPLAN / +49 1805 248 752, Fax +49 2154 811-481 Web: http://www.bitplan.de bitplan GmbH, Willich - HRB 6820 Krefeld, VAT-ID: 10258040548, Geschäftsführer: Wolfgang Fahl ------------------------------------------------------------------------- This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone _______________________________________________ Httpunit-develop mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/httpunit-develop