Re: multiple nntp connections to same server in ubhdb

I'm not sure if this solves what your trying to do, but I will run=20
several concurrent ubh processes sometimes, and the effect is multiple=20
concurrent connections. - Although, I only connect to a single nntp=20
server.

On Mar 18, 2005, at 1:11 AM, Ben wrote:

>  Hello again,
>
>  I was curious if any more thought or development had been made on
>  enabling multiple, concurrent connections to the same nntp servers in
>  ubhdb.=A0
>
>  I was considering how to implement this and wanted to get some
>  feedback on the approach.
>
>  Currently, in Ubh::ServerConnections::connect_to_servers, we take an
>  arrayref of hashrefs containing server connection properties and stash
>  the Ubh::NNTP object in the nntp element.
>
>  Then in freshen_headers(), we iterate through each newsgroup, then
>  each server, fetch the range of articles available, delete from the
>  database anything expired from that server, figure out the range of
>  articles we do have and what new ranges we should retrieve.=A0 Finally,
>  we split that into full sips, retrieve, then fetch the final partial
>  sip (if any).
>
>  I am considering modifying Ubh::ServerConnections::connect_to_servers
>  to create an arrayref of Ubh::NNTP objects.=A0 The servers table would
>  be modified to hold a maxconns field indicating the maximum number of
>  concurrent connections permitted and perhaps another field to indicate
>  how many of those connections to actually utilize.=A0
>
>  In the sipping logic of freshen_headers(), we would then fork and call
>  socketpair to establish pipes for bidirectional communication with
>  each child.=A0 The child would block on reading the parent pipe for a
>  range to sip or command to exit.=A0 The parent, after handing sips to
>  the children, would select on the pipes to see when they have
>  completed that sip, and either send another sip or instruct the child
>  to exit.=A0 When all sips have been exhausted, the parent does a waitpid
>  on each child to avoid zombies.
>
>  That sounds kind of complex but I've written some test programs to
>  verify how it would all work and it actually doesn't seem too bad,
>  Granted I buried my face in the the manpages for select, vec,
>  socketpair, perlipc, IO::Handle, waitpid, and friends for a while.
>
>  However, it still leaves the case where you have 5 servers defined in
>  your servers table and 1 of them only permits one connection at 10kbps
>  or some similarly slow rate.=A0 Your other servers would be blocked
>  until that server finished freshening it's headers, though I suppose
>  you could use the priority field to make it update last?
>
>  Otherwise, I had thought we could also fork each main server
>  definition and iterate through the newsgroups within each child,
>  applying the above described sipping logic in each of those children.
>
>  I don't know if any of this would work under Windows.=A0 It also adds a
>  lot of memory and filehandle consumption.=A0 But I think it would
>  increase article fetching speed quite a bit.
>
>  I've probably left out some details that are in my head.=A0 Does this
>  sound crazy?=A0 Is there an easier way I'm overlooking?=A0 Maybe manuall=
y
>  running additional ubh processes and making ubhdb store it's state in
>  the database so other processes can not conflict?
>
>  Ben



=20
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/ubh/

<*> To unsubscribe from this group, send an email to:
    [email protected]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
=20