Re: Re: Request for protocol extension: get_info/info messages

On Wed, 2005-01-12 at 05:09 +0000, iain_wade wrote:
> 
> --- In [email protected], "Olaf van der Spek" <OvdSpek@L...> 
> > As follow-up to my .torrent-less download thread, I'd like two
> messages to
> > be added to the BT protocol: get_info and info.
> > get_info has no payload
> > info has has payload containing info (bencoded, from .torrent)
> > get_info could be send after the handshake and before other messages
> > info should be send after receiving a handshake and get_info, but
> before
> > receiving other messages
> 
> Hello Bram, and list members,
> 
> I would also like to see a get_info/info extension added.

Very interesting work.  This is also a very interesting problem.

I couldn't find the original message double-quoted above, so I'm taking
what's listed above out of context...

To use get_info/info for .torrent-less downloads, users would already
need to know where to find a tracker and the info_hash value.
Otherwise, they wouldn't know what peers to contact for the info, and
wouldn't be able to send a valid handshake.  Is there a scheme for
finding these?  In my proposed BT friends protocol extension
(btslave.sf.net/btfriends.html), I send a request_help message before
the handshake (I know, it sounds weird, but it simply works better
before the handshake, not after).  It's load provides just enough
information to build a basic torrent object and find the tracker.  Then,
the handshake can proceed as usual.

Another problem with the get_info/info extension is that the load of the
info message can be hundreds of KBytes.  That's a big message.  Even if
you download the message correctly, a user might kill the session early,
wasting the effort of sending the torrent's info in the first place.

In my BT friends extension, I do it differently.  After receiving a
piece, I compute it's SHA1, and then send a 'piece_info' message to a
peer that has the piece and supports the extension.  It's load is the
piece index and my computed SHA1 value for the piece.  The peer replies
with a 'piece_correct' or 'piece_incorrect' message.  Once I've received
a 'piece_correct' message, I send my HAVE messages.

This way, only small messages are sent, and the torrent info is only
sent as you need it.

> I work for an ISP and we are eager to cache bittorrent content to
> lighten the load on our network links as much as possible as well as
> accelerate the performance for our (and yours) users. win/win.
> 
> The intention of our caching peer is to have it passively listening
> for connections solely from our own customer base, not to participate
> in the general torrent distribution.

Reducing the external network traffic for an ISP will obviously save
them money, but you have to be careful not to upset either your
customers or the movie and music industries.

I'd suggest thinking of your program as a repeater, rather than a cache.
I assume that your cache acts just like a normal client: download as
fast as you can, and once you've got the whole thing start acting as a
seeder.  This has some real problems.  You'd download all kinds of
torrents, and probably wind up with lots of terrabytes of data.  The
small problem is that disk cache is expensive.  The big problem is that
the music/movie industry will probably have a nice chat with your
employer about seeding their works.  Another problem is that you're not
helping the torrent while you're downloading. 

A repeater, on the other hand, downloads pieces only when it sees a
chance to help it's peers.  If no connected peers are interested in the
pieces you currently have, go get another piece.  Otherwise, just seed
the pieces you have.  Once you've got a bunch of uninteresting pieces,
exit the torrent, delete the pieces, and re-enter the torrent.  I've
prototyped this scheme, and it seems to work quite well for torrents
that have at least a few downloading peers.  It helps with all of the
problems listed above.  The down-side is it can't help out in torrents
with only one downloading peer.  It would be interesting to know what
percentage of download traffic comes from very active torrents vs fairly
inactive torrents.

> To be able to do this, we need two things. The first is to get the
> clients to connect to us. We need that to happen by default as relying
> on user configuration would limit this feature to a small fraction of
> the user base. (I have a http proxy (for tracker communications)
> written which will add itself to the list of peers before returning,
> but getting a few hundred thousand people to change their settings is
> going to be a problem for us).
> 
> I would like the following patch to be integrated into the official
> BitTorrent client (as well as any other clients) for this reason. It
> does a single dns lookup each time the program starts for
> "btcache.p2p" and adds the ip addresses returned to the peer list:
> 
> https://habitue.net/projects/bt/btcache.patch

Is the idea that this would allow isp's to spoof the btchache.p2p
domain, and return their own bittorent server's address?  Then anyone
outside an ISP supporting this would get the lookup error, and thus no
additional peers?

Couldn't this feature be used to keep track of all user downloads?  I
know this is already possible for ISPs with basic traffic snooping, but
I think users might worry about an automatic feature that tells his ISP
that he's about to download a file set.  I'd like to think that my ISP
tries hard not to look too closely at my traffic.  It'd feel like a
small invasion of privacy otherwise.

Here's a potential alternative: with the BT friends extension, you're
client could go to the torrents where peers meet to make friends, and
only make friends with your customers.  When they need help downloading
from a torrent, they'll contact you with a request_help message, and you
can go from there.  They don't need to know that you're working for
their ISP, and hopefully you wont keep a log of the communication.

> The second feature would be the get_info/info extension so we can
> obtain the "pieces" and "piece length" fields needed for sensible
> torrent participation.
> 
> https://habitue.net/projects/bt/btgetinfo.patch
> 
> If people are against including the getinfo patch for any reason, I
> would still be eager to see the first patch included as I have some
> code written which will "probe" the block size (powers of 2) and not
> do any data checking if I don't have the pieces hashes.
> 
> I've got the .torrent-less caching server code written and freely
> available under a GPL license.
> 
> https://habitue.net/projects/bt/
> 
> If anyone has any feedback I'd be glad to hear it.
> 
> Regards,
> --Iain
> 
> Note// I understand JoltId (http://www.joltid.com) have been
> contacting bittorrent client authors and paying for them to implement
> a caching proxy solution locked in to their proprietary PeerCache
> protocol. I would hope any agreements with them do not preclude the
> fruits of our own work (open source and freely available) being
> accepted.

Scary...  I'm not against a guy making a few bucks, or ISPs saving
money, but these guys are seriously into spy-ware.

IMO, this is a good reason for the open-source community to help solve
the caching problem for free.  So, here's my two cents on how to make it
actually happen...

To get support from users, BT client authors, and Bram, I think you need
a good reason.  Saving the ISP money sounds good, but let's face it: no
one cares (unless they're being bribed to care).

Here's a good reason for average Joe to care: you help him download
faster.  If you do that, I think the path to acceptance will be much
easier.

My proposed BT friends protocol can help average Joe download faster
even if he's one of two guys in the world using the protocol (or one guy
who runs it at home and at work).  I'm pretty confident that it will
catch on quickly when it's done.  People like downloading faster for
free.

Is there some way to cooperate?

Bill




 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/BitTorrent/

<*> To unsubscribe from this group, send an email to:
    [email protected]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/