Re: scripting an interactive web page

On Mon, March 22, 2010 10:49 pm, Joel Heenan wrote:
> Do you need to parse the data? If so you can't go past beautifulsoup :
> http://www.crummy.com/software/BeautifulSoup/

Joel, thanks

ahem, dumb Q:

where do I put it ?

, and, once I've installed it, how do I use it...?

# ls
BeautifulSoup.py   BeautifulSoupTests.py   PKG-INFO
BeautifulSoup.pyc  BeautifulSoupTests.pyc  setup.py

# python setup.py
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

# python -V
Python 2.3.4
# whereis python
python: /usr/bin/python2.3 /usr/bin/python /usr/lib/python2.3
/usr/include/python2.3 /opt/splunk/bin/python2.4 /opt/splunk/bin/python
/usr/share/man/man1/python.1.gz



> The devil is in the details here: how do you login ? If its just
> simply HTTP auth, or if it is using GET parameters (and you don't care
> about the security of your login) you can pass that into wget. If its more
> complex I think you are looking at writing your own script.

it's not HTTP auth, it's a GET

I had a few goes without much luck, so far.

all I get, is a cookie created

I've been trying stuff like below, and similar, based on 'input' fields in
the /form:

wget
--post-data='username=voytek&password=none&_action_login=_login&document.login_form._action.value=_login&_action_login=_login&_action.value=_login'
--keep-session-cookies --save-cookies=session.tmp dom.tld/main.htm



-- 
Voytek

-- 
SLUG - Sydney Linux User Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html