Re: keep wget --recursive away from "delete" links

max630-JGs/[email protected] writes:

> On Tue, Jun 12, 2007 at 01:08:43AM +0800, [email protected] wrote:
> > (The following is a warning to fellow WWWOFFLE-users to
> > keep anything recursive away from reading pages via
> > WWWOFFLE, if you are an AddCacheInfo user.)
> 
> > That must have been it, my hogwild recursive request.
> > Therefore please add a CAPTCHA math test to that delete
> > page...
> 
> This is wrong solution.
> 
> To avoid such nasty things, all irreversible operations must
> be performed by POST requests. This is what recommended by
> w3c for "all state-changing" requests.

This is what I said in wwwoffle-users last time that jidanni raised
this point (just over a year ago).

: In WWWOFFLE either a POST or a GET request can be used for the page
: that deletes a URL (the code accepts either).  I chose to have a GET
: request for deleting pages from the cache.  There are lots of things
: in WWWOFFLE that have side effects; any time that a page is requested
: there is a change to the cache.  These cannot all have POST requests.

> For now you should disable modify-html when you use wget
> (not only because of this issue). There is a tricky way to
> achieve it without changing configs.  WWWOFFLE does not
> modify pages when ht/dig or some other indexing bot requests
> it. It looks info "User-Agent" header.  So, if you run wget
> with --user-agent="ht/dig" (read the sources for exact value
> for user-agent, I don't remember), you will not receive the
> links to delete and other links from footer.

Another header would be "Cache-Control: no-transform" or you could use
"Pragma: wwwoffle-client".  Either of these two would be prefered to
faking a web indexer.

> btw, there is already http authentication for deletion. how
> did wget pass it?

The authentication is only optional.

-- 
Andrew.
----------------------------------------------------------------------
Andrew M. Bishop                             amb-Uxr6IM1mbv2TY6FTCsQk+9Bc4/[email protected]
                                      http://www.gedanken.demon.co.uk/

WWWOFFLE users page:
        http://www.gedanken.demon.co.uk/wwwoffle/version-2.9/user.html