Bug#781351: www.debian.org: Please provide "date" field for each item of DPN RSS feed (feedparser complains about that)
Laura Arjona Reina <[email protected]>
| Newsgroups | gmane.linux.debian.devel.www |
|---|---|
| Message-ID | <20150327200859.24710.77687.reportbug__48814.4818846651$1427487170$gmane$org@larjona-laptop> |
Package: www.debian.org Severity: wishlist Tags: patch Dear all I'm having trouble when using Spigot[1] to publish the DPN feed[2] in Debian's identi.ca account: [1] https://pypi.python.org/pypi/spigot/ [2] https://www.debian.org/News/weekly/dwn.en.rdf Spigot is a Python script using Feedparser. Spigot "needs" a "date" field for each RSS item, in order to handle publication of only the "new" RSS items each time. The Perl script[3] that generates the DPN feed stores the date in the "channel" section but not in each item. [3] http://anonscm.debian.org/viewvc/webwml/webwml/english/News/weekly/dwn-to- rdf.pl Feedparser throws an exception when it does not find the requested field in the RSS feed, and then, Spigot crashes (I'm attaching the trace). I've contacted the developer of Spigot in order to get this exception controlled, but maybe it's convenient to add the date field to each item anyway? I'm attaching a patch to the Perl script that generates the RDF feed. My Perl skills are very very limited so I kindly ask for review, if you consider it's a good thing to add the date to each item. Thanks! -- System Information: Debian Release: 8.0 APT prefers testing APT policy: (500, 'testing') Architecture: i386 (i686) Kernel: Linux 3.16.0-4-686-pae (SMP w/2 CPU cores) Locale: LANG=es_ES.UTF-8, LC_CTYPE=es_ES.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system)
spigot_trace.txt
(text/plain, 919 B)
2015-03-27 18:30:53,472 DEBUG: Polling feed https://www.debian.org/News/weekly/dwn.en.rdf for new items
2015-03-27 18:30:54,143 DEBUG: Found 9 items in feed https://www.debian.org/News/weekly/dwn.en.rdf
2015-03-27 18:30:54,143 DEBUG: Processing item 0
2015-03-27 18:30:54,143 DEBUG: Title: Reports
2015-03-27 18:30:54,143 DEBUG: Link: https://www.debian.org/News/project/2015/02/index.en.html#reports
Traceback (most recent call last):
File "/usr/local/bin/spigot.py", line 541, in <module>
spigot_feed.poll_feeds()
File "/usr/local/bin/spigot.py", line 383, in poll_feeds
self.scan_feed(url)
File "/usr/local/bin/spigot.py", line 409, in scan_feed
date = p.entries[i].updated_parsed
File "/usr/local/lib/python2.7/dist-packages/feedparser.py", line 416, in __getattr__
raise AttributeError, "object has no attribute '%s'" % key
AttributeError: object has no attribute 'updated_parsed'
dwn-to-rdf.pl.diff
(text/x-diff, 1001 B)
--- /home/larjona/Documentos/l10n/Debian/webwml/english/News/weekly/dwn-to-rdf.pl
+++ /home/larjona/Documentos/l10n/Debian/webwml/english/News/weekly/dwn-to-rdf.new.pl
@@ -94,6 +94,7 @@
my $count = shift;
my $headline = shift;
my $body = shift;
+ my $pubdate = shift;
$body =~ s~\$\(HOME\)~https://www.debian.org~g;
$body =~ s/\\\n//g;
@@ -130,6 +131,9 @@
$rss->add_item (title => $headline,
description => $body,
link => $url.'#'.$count,
+ dc => {
+ date => $pubdate,
+ },
);
}
@@ -195,10 +199,10 @@
if ($name !~ /(newcontributors|rcstats|dsa|nnwp|wnpp|continuedpn)/) {
if (!$headline) {
- rdf_add ($rss, 'content', $locdesc, $body) if ($count == 0);
+ rdf_add ($rss, 'content', $locdesc, $body, $pubdate) if ($count == 0);
$count++
} else {
- rdf_add ($rss, $name, $headline, $body);
+ rdf_add ($rss, $name, $headline, $body, $pubdate);
}
}