Re: scripting an interactive web page

On Wed, Mar 24, 2010 at 12:17 AM, Voytek Eymont <[email protected]> wrote:

>
> On Mon, March 22, 2010 10:49 pm, Joel Heenan wrote:
> > Do you need to parse the data? If so you can't go past beautifulsoup :
> > http://www.crummy.com/software/BeautifulSoup/
>
> Joel, thanks
>
> ahem, dumb Q:
>
> where do I put it ?
>
> , and, once I've installed it, how do I use it...?
>

Oh so beautifulsoup is a python module specifically designed for parsing of
HTML. Do you know how to program in python? If not then start with the
python tutorial! I've seen a few attempts to parse HTML, and beautifulsoup
just feels right to me.

To install it you run "python setup.py install"

>
>
> > The devil is in the details here: how do you login ? If its just
> > simply HTTP auth, or if it is using GET parameters (and you don't care
> > about the security of your login) you can pass that into wget. If its
> more
> > complex I think you are looking at writing your own script.
>
> it's not HTTP auth, it's a GET
>
> I had a few goes without much luck, so far.
>
> all I get, is a cookie created
>
> I've been trying stuff like below, and similar, based on 'input' fields in
> the /form:
>
> wget
>
> --post-data='username=voytek&password=none&_action_login=_login&document.login_form._action.value=_login&_action_login=_login&_action.value=_login'
> --keep-session-cookies --save-cookies=session.tmp dom.tld/main.htm
>
>
If it's GET, you don't need to pass in post-data. Browsing the website using
a Firefox tool like Firebug or Tamper Data will show you exactly what is
going on if you are confused about what exactly is being posted where.

Joel

-- 
SLUG - Sydney Linux User Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html