Populating a local message store AND multithreading

Hello all,

I just installed sn to try to keep up with a couple of binaries groups
during the days that I'm away at work and can't manually pull the articles
I'm interested in. My news server SUCKS at retention.

I have a couple of questions.

1. No matter what I tried as the -m value I couldn't seem to get
snget/snfetch to pull more than a few thousand articles. I am working from
remote via ssh and was trying to populate the local message store with all
of the articles currently available in the NG. 

I telneted into the news server and switched to the group. Start/finish
article numbers indicated about 30,000 articles total. I pulled the header
for the lowest number article and it was about 3 days old.

I dropped the telnet connection and ran 'snget -c 4 -p 4 -m 31000 NG.NAME'

It pulled about 1000 - 1500 articles then exited cleanly. I ran it again
and got the same behavior. After running it about 5 times it stopped
pulling more articles just reporting "No New Articles". When I reviewed the
local message store there are no articles older than yesterday. It's as if
it completely ignored the fact that I was trying to grab ALL articles
available on the remote server.

2. Speed and parallel connections. 

I was watching eth0 with iptraf while pulling the articles and saw speeds
staying under 150 kiloBytes/sec. The machine is connected by cable and I
know that it can maintain nntp speeds over 300 kiloBytes/sec BUT the news
server limits the speed of each connection and one must maintain 4
connections to get that speed (and 4 is the max allowed). Doing the math
after snget reports total bytes retrieved/time shows a speed of about 90
kiloBytes/sec which is consistent with the speed limit of a single
connection.

This makes it appear as if the -c and -p flags are having no effect. Can
someone explain them (especially -c) more clearly?

3. File locking

The 2 groups I want to pull are VERY active. In order to stay current I'd
like to set a cron job to run snget every 15 or 30 minutes to try to make
sure I get all of the articles. If one of the groups has received a LOT of
articles and the previous run has not finished when cron calls for a new
run will it be handled in a clean manner?

4. Symlinked groups.

Because I expect to pull upwards of 20 GIG of files over the 3 - 5 days I
am away from home each week I split the two groups I pull on two different
hard drives by using snnewgroup to create them then moving one to another
drive and symlinking it back to SNROOT. 

What's interesting is that snget doesn't seem to see this group as
'non-local'. If I run snget with no switches it gets articles from the
group located directly in SNROOT but completely skips the symlinked group.
If I call snget with the linked group name it works fine though. It has it
listed in .newsgroups and articles are placed properly. The group shows up
in my nntp client when I get a list of groups and I can subscribe and
download from it. The only thing that doesn't work is pulling it
automatically with snget. It has to be specified.

Not a big problem since I'm only interested in 2 groups but though it
should be noted.

5. Article expiration.

Is there any way to force article expiration based on something other than
date? I've set up sn to pull messages while I'm not home (where the fast
connection is located). When I get home I want to pull those articles I'm
interested in then expire the whole lot to recover the space. I'll want to
expire everything up to the last download.

It would be nice also to be able to expire based on drive space in the
event I'm away for longer than expected. Set a limit on the amount of space
a single group can occupy and have it expire the oldest articles if it
reaches that limit.

Thanks for your time,

Gerald

--
Gerald V. Livingston II

Configure your Email to send TEXT ONLY -- See the following page:
http://expita.com/nomime.html