Re: Re: Trackerless BitTorrent

>From: "Olaf van der Spek" <[email protected]>
> > of the central tracker. That of keeping the network widely
> > interconnected. With this scenario a client will tend to connect with
> > nearby neighbors, forming cliches. With the tracker you'll get some
> > nearby connections, but also a bunch that span the network, a client
> > cannot do this.
> 
> Can't that be solved by keeping track of all IP addresses you received, 
> connecting at least once to all, fetching their peer list and then connect 
> at random to let's say 50 peers of your own list?
> This idea would at least improve the situation in which the tracker goes 
> down but you already have a list of peers and it'd increase the discovery 
> rate after you first start without abusing the tracker. 

You're not solving the problem. You're also likely expending a *lot* of
bandwidth and needing to do a *lot* of bookkeeping on the client.

First, assuming 50 peers and retrieving 50 peers from each out those.
Resulting in a list of 2500 address/ports, minimum bandwidth of 15,000
bytes (45,000 with IPv6) for the peer list. In a small swarm there will
also be a *lot* of redundancy here merely because there aren't that many
actual clients.

The real problem is the one I was pointing to above. From your peers
you'll get other peers that are already close to you in network terms
(degree 2). This is extraordinarily unhealthy for the swarm. Any piece
you get from your peers will likely already be had by most/all your peers
(because they're already cross-linked). This results in bad download
speeds because you cannot obtain any pieces your peers are interested in.
In fact it is quite likely your peers won't have much at all anyway
because they can't get anything either. Pretty much you end up with the
swarm divided into bunches of independant islands/cliches, completely
destroying the effectiveness of BitTorrent.

Then there are the secondary effects of this. These cliches will greatly
increase the redundancy of those peer lists. I'd be surprised if more
than a third of those 2,500 were unique. By the time you've got a mere
1,000 clients in your swarm things are definitely breaking down (this is
an optimistic estimate, even a swarm of 250 might have problems) By the
time you've got 10,000 clients in your swarm you're totally dead.

With the tracker not only will you get a list of 50 unique peers, but
those will also be widely spaced across the network. This *cannot* be
done by peers telling each other about other peers.


>From: apoipoi <[email protected]>
> 
> --- In [email protected], Elliott Mitchell <ehem@m...> wrote:
> > >From: apoipoi <apoipoi@y...>
> > > The only necessary change to the torrent format is the list of
> > > addresses. Which can be affixed at the end. The creator of torrent
> > > would be the first in the list of course to ensure anyone can connect
> > > and start downloading. As the "network" grows, this addresses can be
> > > added to the torrent. The updated torrent file can be uploaded on the
> > > web. Instead of every new clients connecting to A, they can
> > > (randomly?) try B, C, or D. This will reduce the workload on A.
> > 
> > Two fatal weaknesses. First, you've replaced a tracker getting bombarded
> > with requests with the client A/torrent source getting bombarded with
> > requests.
> 
> A does not replace the role of tracker by being the (sole) keeper of
> address list. There are only two events that may constitute as
> "request" to A (or to any other peers):
> (1) When a new peer (let's say) connects to A, where A givest it the
> list of other address(es), and
> (2) (Optional?) After the connection has been made, A informs others
> in the network of a new address/peer. This is optional though, as B
> now has the list may contact them individually.
> 
> All in all, the amount of "request" is far less than what is made to
> trackers.

Really? Sounds like precisely the same amount of data as that sent to a
tracker. Worse, you're now trusting peer A to be running and giving
correct information (suddenly DoSes become *really* trivial). Peer A will
likely also have a limitation of the maximum number of peers it will talk
to, so with standard peers they'll start refusing connections once
they've hit 55.

> Trackers are designed as webserver where there is no active connection
> to any peer. It is not possible to send information about new peers to
> currently connected ones. Thus -- correctly if I'm wrong -- it is
> necessary to let peers request information periodically. With this new
> method, since there is already an active connection why not use it.
> And only sends information when needed, reducing the frequency to minimum.

Once you've got sufficient peers the standard client will stop talking to
the tracker to request more peers, it will merely say "I'm still here" at
a frequency specified by the tracker (so that new peers can connect to
it). There could be a small gain of allowing a client to specify it
doesn't need more peers, or specify that it is full (will refuse new
connections). Perhaps you should be looking towards increasing the
rerequest interval of your trackers or decreasing the number of peers
returned? (particularly with large torrents that will take hours to
download, why tell the tracker every 5 minutes you're still alive)

> The torrent file now also contains the list of addresses. Not only the
> hashes. Peer A can then upload new torrent (with identical hash but
> additional peers addresses) and new peers can connect to either one of
> them. Yes, at the beginning new peers will recognize A since that's
> the only address contained in the torrent. After say 20 peers are
> connected to A and a new torrent make with total of 21 address
> including A, new peers can connect to any of the 21.
> 
> Or to take this even further, after each successful connection peers
> (e.g. A) may send the address of newly connected peer (e.g. B) to
> tracker. Tracker will store this address. When a client tries to
> connect it can request the peer the currently known list. Only the
> currently known, therefore one request only. The rest goes as usual,
> no more disturbing the tracker.

Gee, this is *exactly* what a tracker does! Only now it is *much* easier
to attack, and you've made the load on the original webserver much larger
as the torrent file is bigger. Yes, you have decreased the bandwidth
somewhat by batching things, yet the loses are very bad and those gains
are dubious anyway.

> > Second, you've missed an important effect of the first function
> > of the central tracker. That of keeping the network widely
> > interconnected. With this scenario a client will tend to connect with
> > nearby neighbors, forming cliches. With the tracker you'll get some
> > nearby connections, but also a bunch that span the network, a client
> > cannot do this.
> 
> New peer will be given list of addresses known to existing network (or
> at least to the peer it is connecting to). This has always been the
> case with trackers. The difference may be that trackers only send a
> limited number and chosen randomly. This is to reduce the total data
> size transfered and to cater to multiple requests. Ideally the
> complete list should be given. What cannot be done currently (because
> of sheer total request load) can be done with this new method as I've
> explained above.

Though limiting the size of the request might be the justification
mentioned in the protocol specification, it also has the *far* more
important effects mentioned above. Mainly you get peers widely spaced
across the network and therefore likely to have distinct sets of pieces.
If you get the full list from a peer, not only will you peers that are
close together, but you're just made the request much larger.

> The choice of which address to connect is up to clients. BitTorrent is
> a protocol. There is no way we can enforce a law of which address to
> choose (or to not choose).
> 
> Yes, we can specify that the list should contain limited number of
> address chose randomly. But this will cripple the network. Instead
> improving performance by introducing each peer to every other peers,
> we limit them to just a few. Besides, any developer can choose not to
> follow this particular rule.

Yes, a peer can be smarter and make a better choice about who to attempt
to connect to. Unfortunatly if the list is sub-optimal even the best
algorithm/heuristic to choose will be useless.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \   (    |         [email protected] PGP 8881EF59         |    )   /
  \_  \   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
    \___\_|_/82 04 A1 3C C7 B1 37 2A*E3 6E 84 DA 97 4C 40 E6\_|_/___/




------------------------ Yahoo! Groups Sponsor --------------------~--> 
Make a clean sweep of pop-up ads. Yahoo! Companion Toolbar.
Now with Pop-Up Blocker. Get it for free!
http://us.click.yahoo.com/L5YrjA/eSIIAA/yQLSAA/dkFolB/TM
--------------------------------------------------------------------~-> 

 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/BitTorrent/

<*> To unsubscribe from this group, send an email to:
    [email protected]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/