Re: Re: Query Hit bitprint

Your knowledge of LimeWire's download verification appears to be a
little outdated, Philippe.

LimeWire currently does everything you are describing, including
verifying pieces of the file as it downloads.

(The only thing it does not do is know the tiger tree ahead of time,
but that isn't necessary for any of the other steps.  Swarmed
downloads can make an intelligent guess about the tree size and then
change their guess once the actual size is discovered.)

Sam

On 9/12/07, Philippe Verdy <[email protected]> wrote:
> The bad thing is that the X-Thex-URI header can't be discovered until you
> have started downloading some fragment from a source, this means that you
> can't predict where to start downloading fragments, notably for the first
> fragment, by first querying the servents that have bitprints in their query
> hits.
>
> If you had a set of servents showing the same bitprint, and wanted to
> initiate a swarmed download from 4 sources, you would perform 4 connections
> in parallel for one fourth of the file size on each attempted connection.
>
> For the first responding servent you would start a parallel download for the
> TigerTree data, bout you would not need to abort a download in the middle
> just because the end of the file is already downloaded in parallel from
> another discovered working source. Swarmed downloads could be more finely
> grained, exactly on position of the elementary Tiger Tree blocks (no need to
> download two parts of the same Tiger Tree block from two sources, as these
> blocks are small enough).
>
> Having to break a download in the middle is just a waste of time, because
> you need to break the connection and you may need that connection again to
> download other blocks. Files could be downloaded non linearily, with the
> downloader splitting the remaining missing parts as needed, depending on the
> speed and availability of usable sources.
>
> The way to avoid  this problem is to use persistent HTTP connections, and to
> request downloads for small fragments instead of whole files, so that you
> can avoid breaking the connection and get your next fragment request
> possible faster, without wasting uploader resources. But uploaders need to
> honor persistent connections, and allow multiple successive requests for
> fragments, not necessarily in the linear successive order (because some
> parts may be already downloaded by the downloader from other sources).
>
> Persistent connections with small blocks also have an advantage here: the
> downloader can communicate with the uploader to transmit (in both
> directions, using the HTTP query and reply headers) the location of other
> sources that he successfully uses or has discovered separately, and so this
> allows faster consolidation of the download mesh between all competing
> downloaders and available sources.
>
> For highly wanted files from few sources, this really helps distributing the
> work if the downloaders are also sharing their existing fragments as soon as
> they have downloaded the TigerTree data (and then being able to verify the
> fragments they already have downloaded).
>
> For this reason, this will help making more sources available instead of
> just expecting that just a few sources will complete the whole work for many
> candidate downloaders of the same file (today, we can see too many sources
> not responding to download requests despite they do have the file or usable
> parts of it). This will work best and reliably if downloaded files are
> verifed against the TigerTree during the download instead of just at end
> when the file is complete (what LimeWire still currently does).
>
> Another reason why bitprints would be useful is that this would allow
> detecting spammers more easily, because they are spamming the same data just
> modified with a few random garbage (just to avoid easy detection by the
> simple SHA1 digest of the complete file, as the garbage specific to each
> source will make a new SHA1): it's more difficult to disguise a TigerTree
> data of spammed contents, because their TigerTree will contain common
> sequences of blocks hashes (but the parade for spammers is to locate their
> "useful" spammed parts in the shared files at random variable position in
> the copy of the spammed file they have, moving it using padding without
> effect at the start, such as in metadata extension records of MPEG files or
> a garbage file at the start of a ZIP archive, so that the "useful" mobile
> part will not match the existing TigerTree block boundaries).
>
> > -----Message d'origine-----
> > De: [email protected] [mailto:[email protected]] De la part
> > de [email protected]
> > Envoyé: mercredi 12 septembre 2007 16:05
> > À: [email protected]
> > Objet: [the_gdf] Re: Query Hit bitprint
> >
> > Quoting mark W <[email protected]> from ml.gnutella.dev-forum:
> > :Actually HUGE specs shows the presence of bitprint :
> > :http://rfc-gnutella.sourceforge.net/src/draft-gdf-huge-0_94.txt
> > :
> > :format of it:
> > :urn:bitprint:[32-character-SHA1].[39-character-TigerTree]
> >
> > Bitprints occupy more space in packets, and SHA1 has been good enough
> > for now.  That's why all the servents I know use SHA1 in query hits,
> > but offer a X-Thex-URI header during HTTP exchanges, leading to the
> > TTH data from which you can derive the root Tiger Tree Hash (TTH) and
> > therefore the bitprint.
> >
> > Raphael
> >
> >
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>
>
>
>
>
>
> Yahoo! Groups Links
>
>
>
>