Re: Importing an Old INN News Spool.

Hi Patrik,

> Ralph Corderoy <[email protected]> wrote:
> > My current plan is create the groups with snnewgroup (what do I do
> > for a moderated newsgroup, e.g. comp.compilers, I'd guess stick
> > something in comp.compilers/.outgoing?) and then turn each of the
> > articles in my old
> 
> Quoting from an old mail from Harold:
> > For normal newsgroups, those are automatically moderated anyway,
> > because the upstream site would forward the article to the
> > moderator.
>
> So apparently you just set up the .outgoing link, and see if your
> upstream server does the right thing. ("Apparently" because I've never
> posted to moderated groups with sn, so I'm not totally sure.)

Ah, OK, I'd never have considered that but I guess it makes sense since
the ISP saves a lot of hassle taking care of it on behalf of its
customers compared to getting all the customers to correctly configure
their news software.  I've posted to one of my ISP's newsgroups to seek
an explicit confirmation that they'll do this.

It still leaves sn answering `list active', etc., with either an `n' for
no-post, or a `y', but never an `m' for moderated.  I guess that will
affect what some news clients report for a newsgroup.

> > spool into wire format before catting them all into snstore.  Does
> > that sound like I'm on the right path?
> 
> Sounds like a good plan, yes. If you create any scripts for this,
> there probably might be others interested in them too so please share.
> :)

OK, it seemed to work fine, and I managed to process my old .newsrc to
generate a new one by translating the post numbers.  All the scripting I
done is a bit hit-and-miss since I could manually inspect the results so
these shouldn't just be used straight off, but are really just a starter
for those that come reading through the list archives later.

inntowire turns inn spool news articles into the wire format expected by
snstore.

==== inntowire begin ====
#! /usr/bin/perl

@ARGV or @ARGV = ('-');

undef $/;

for $f (@ARGV) {
    open F, $f or die "couldn't open $f: $!\n";
    $_ = <F>;
    s/^\./../gm;
    s/\z/.\n/;
    s/\n/\r\n/g;
    print;
}
==== inntowire end ====

This next one takes a .newsrc with article numbers referring to the old
inn spool area.  For each article it gets the Message-Id and uses
`snscan -i' to find the new article id.  (Hence my earlier email about
snscan always returning just the main article and never any aliases.)  
Unfortunately, snstore can discard some `dodgy' Message-Ids and make up
fresh ones so some articles can't be found.

The output is a long list of old and new article id numbers.  All I did
then was sort the output by new article id number and use awk to cat all
the article id numbers into one line per newsgroup, e.g.

    comp.compilers: 10,11,12,13,14,15..,1004,1005

Note, I didn't bother working out the ranges, e.g. 1-1005.

After starting trn on that new .newsrc file and perusing a group, trn
would re-write the newsgroup's entry back with ranges, vastly shortening
the lines.

==== recalcnewsrc begin ====
#! /bin/sh

PATH=/home/sn/bin:$PATH

old=/o/home/ralph/.newsrc
spool=/o/home/var/spool/news

for f in `cd ~sn/news && ls`; do
    esc=`echo $f | sed 's/\./\\\\&/g'`
    dir=`echo $f | sed 's/\./\\//g'`

    grep "^$esc:" $old |
    sed 's/^/== /'

    grep "^$esc:" $old |
    awk '{print $2}' |
    tr , '\012' |
    while read range; do
        echo "==   $range"
        set -- `echo $range | sed -e 's/^[0-9][0-9]*$/&-&/' -e 's/-/ /'`
        seq $1 $2 |
        while read art; do
            artf=$spool/$dir/$art
            test -f $artf || continue
            msgid=`egrep -i '^message-id:' $artf | awk '{print $2;exit}'`
            t=`snscan -i $msgid`
            if test "$t" = ''; then
                echo "not found: $f $art $msgid"
            else
                echo $art $t
            fi
        done
    done
done
==== recalcnewsrc end ====

Cheers,


Ralph.