RE: Tracker scrape extension proposal

 
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

The current scrape system allows multiple info_hash= values to be
passed to a tracker. It is up to individual tracker package
developers to support this.

Your proposal does allow for more torrents to be scraped in a given
pass, but it's probably not 100% necessary.

- -----Original Message-----
From: Nick Johnson [mailto:[email protected]] 
Sent: Wednesday, January 12, 2005 1:11 AM
To: [email protected]
Subject: [BitTorrent] Tracker scrape extension proposal


A project I'm working on at the moment requires fetching scrape data 
from trackers. Often, I have multiple torrents on the same tracker
that 
I want scrape data for, so fetching scrape data for each file 
individually is probably very inefficient. On the other hand, the 
tracker's full scrape might be huge (multiple megs!) depending on how
many torrents are on the tracker that I don't care about.

With this in mind, I have a couple of proposals for how to improve
this:
- -Add a GET parameter 'hashes', which is a string that's a multiple of
20 
bytes. Each 20 bytes corresponds to one info_hash that you want the 
tracker to return. If the tracker recognises the extension, it
returns 
only the requested keys, otherwise it returns all keys (since this
would 
be the default behaviour of trackers getting unknown get parameters 
anyway - hopefully!)
Due to limitations on GET string length, this is probably limited to 
about 100 torrents before problems start to crop up. URLs like this
are 
quite likely to result in messes in log files, too. :/

- -Somewhat more complicated: The client supplies a Bloom Filter 
(http://en.wikipedia.org/wiki/Bloom_filter) to the tracker (using a 
defined set of hash functions), with the appropriate entry in the
filter 
set for each torrent it wants. The nature of the Bloom Filter
guarantees 
all the requested entries will be returned, but extra entries may be 
returned as well. This way, an unlimited number of hashes can be 
requested with a static request size, just with an increasing 
false-positive rate.

Other suggestions are welcomed. It just seems to me that, at the
moment, 
for fetching 100 torrents from a tracker that's tracking 1000, either
option is rather inefficient.

- -Nick Johnson

- -- 
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.300 / Virus Database: 265.6.10 - Release Date: 1/10/2005
 

-----BEGIN PGP SIGNATURE-----
Version: PGP 8.0.3

iQEVAwUBQeS3+l8nceBm0DUaAQK62Af/fk/iuMWokwRfLwg0zaQ0CJrKRJj6RWJl
PIM8E1+We6fbKx+WFoj0ey2U+wNAhlxqg611cR2p+OBlpTaupmq13t4PkcDngxoj
52l0BGPf02W6ebeJALJEWazyQoxaMomst6ENBVCRQvg4UFMs9lCZ4F1TlzrsFrKK
+qe1dX1HV3lbwCsPJF8PIuGIwz2Ip3Mq63QLVJcIcEmKl9xBila8BRd7IeV/XmoY
BSaC3IwAupwzmwFUrigGr9DnMK0WJu8ZfKoxnoBquXSH6WxDzwBuso+aXH0iRZNc
WeIyCqJjekra50eIoExdsEQWIrXVnndB4R28wF21jpXyYCpOAfWxfg==
=05gg
-----END PGP SIGNATURE-----




 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/BitTorrent/

<*> To unsubscribe from this group, send an email to:
    [email protected]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/