RE: Re: Query Hit bitprint

The bad thing is that the X-Thex-URI header can't be discovered until you
have started downloading some fragment from a source, this means that you
can't predict where to start downloading fragments, notably for the first
fragment, by first querying the servents that have bitprints in their query
hits.

If you had a set of servents showing the same bitprint, and wanted to
initiate a swarmed download from 4 sources, you would perform 4 connections
in parallel for one fourth of the file size on each attempted connection.

For the first responding servent you would start a parallel download for the
TigerTree data, bout you would not need to abort a download in the middle
just because the end of the file is already downloaded in parallel from
another discovered working source. Swarmed downloads could be more finely
grained, exactly on position of the elementary Tiger Tree blocks (no need to
download two parts of the same Tiger Tree block from two sources, as these
blocks are small enough).

Having to break a download in the middle is just a waste of time, because
you need to break the connection and you may need that connection again to
download other blocks. Files could be downloaded non linearily, with the
downloader splitting the remaining missing parts as needed, depending on the
speed and availability of usable sources.

The way to avoid  this problem is to use persistent HTTP connections, and to
request downloads for small fragments instead of whole files, so that you
can avoid breaking the connection and get your next fragment request
possible faster, without wasting uploader resources. But uploaders need to
honor persistent connections, and allow multiple successive requests for
fragments, not necessarily in the linear successive order (because some
parts may be already downloaded by the downloader from other sources).

Persistent connections with small blocks also have an advantage here: the
downloader can communicate with the uploader to transmit (in both
directions, using the HTTP query and reply headers) the location of other
sources that he successfully uses or has discovered separately, and so this
allows faster consolidation of the download mesh between all competing
downloaders and available sources.

For highly wanted files from few sources, this really helps distributing the
work if the downloaders are also sharing their existing fragments as soon as
they have downloaded the TigerTree data (and then being able to verify the
fragments they already have downloaded).

For this reason, this will help making more sources available instead of
just expecting that just a few sources will complete the whole work for many
candidate downloaders of the same file (today, we can see too many sources
not responding to download requests despite they do have the file or usable
parts of it). This will work best and reliably if downloaded files are
verifed against the TigerTree during the download instead of just at end
when the file is complete (what LimeWire still currently does).

Another reason why bitprints would be useful is that this would allow
detecting spammers more easily, because they are spamming the same data just
modified with a few random garbage (just to avoid easy detection by the
simple SHA1 digest of the complete file, as the garbage specific to each
source will make a new SHA1): it's more difficult to disguise a TigerTree
data of spammed contents, because their TigerTree will contain common
sequences of blocks hashes (but the parade for spammers is to locate their
"useful" spammed parts in the shared files at random variable position in
the copy of the spammed file they have, moving it using padding without
effect at the start, such as in metadata extension records of MPEG files or
a garbage file at the start of a ZIP archive, so that the "useful" mobile
part will not match the existing TigerTree block boundaries).

> -----Message d'origine-----
> De : [email protected] [mailto:[email protected]] De la part
> de [email protected]
> Envoyé : mercredi 12 septembre 2007 16:05
> À : [email protected]
> Objet : [the_gdf] Re: Query Hit bitprint
> 
> Quoting mark W <[email protected]> from ml.gnutella.dev-forum:
> :Actually HUGE specs shows the presence of bitprint :
> :http://rfc-gnutella.sourceforge.net/src/draft-gdf-huge-0_94.txt
> :
> :format of it:
> :urn:bitprint:[32-character-SHA1].[39-character-TigerTree]
> 
> Bitprints occupy more space in packets, and SHA1 has been good enough
> for now.  That's why all the servents I know use SHA1 in query hits,
> but offer a X-Thex-URI header during HTTP exchanges, leading to the
> TTH data from which you can derive the root Tiger Tree Hash (TTH) and
> therefore the bitprint.
> 
> Raphael
> 
> 
> 
> Yahoo! Groups Links
> 
> 
> 
>