Automatic crawling for feed templates: 0watch

Hello all,

I maintain a set of feeds for 3rd party apps at http://0install.de/feeds/
using 0repo and 0template.
While updating a feed when a new version of an app is released only takes a
couple of keystrokes I'd still much prefer if this were a fully automatic
process.

I was thinking of introducing a new tool to help with this. Going with the
name "0watch" for now.
0watch would take a .xml.template file as input and look for a
yet-to-be-defined <template> tag in a new XML namespace.
This tag would point to an HTTP URI along with regular expressions for
extracting a set of available versions and metadata (e.g. release date) from
an upstream website.
Next, 0watch would fetch the current state of the feed the template points
to in its <feed-for> tag.
Finally, 0watch would call 0template once for each version found using the
regular expressions but not currently present in the feed.

Then I could create a simple cronjob like this:
cd templates
0watch templates/*.xml.template
mv templates/*.xml incoming/
0repo

I think this might be best built as a Python tool, pulling in Python
0install and 0template as dependencies.

What do you guys think? Would this be a useful tool for you as well?
Any recommendations for Python libs for parsing/crawling websites?
Possible alternatives for using RegEx to extract content?
Name suggestions?

Regards
Bastian


------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape