Re: Using the DHT to find download sources

--- In [email protected], Arne Babenhauserheide <arne_bab@...>
wrote:
>
> Am Mittwoch 23 Juli 2008 16:48:03 schrieb pwang708:
> > What I meant was the anti-spam system may not stop spams on rare
files.
> 
> From my experience, rare files aren't really being spammed (except
with the 
> same default spam replies, and those are common, so they'd be
recognized). 
> 
> Spam, in the way it is used today, is a means to make it harder for
people to 
> reach some popular files for whoose "protection" the spammers get paid. 
> 
> That rare files aren't in our search horizon anymore due to spam is
just a 
> byproduct. 
> 

Good point! I think maybe we also need to add something like an attack
model to our requirement etc list. This model will help us to evaluate
anti-spam schemes, and other security scheme. For example, I think
Credence is designed to fight "traditional" spammers. I doubt if it
works well against the spammers you described above. Considering an
attack similar to the whitewashing attack, join many nodes to the
network, vote honestly on everything expect on the files the spammer
don't want people to download. Note that the spammer nodes can always
learn honest nodes' votes on other things from the network... 

BTW, The default spam replies can be recognized because they don't
change? Mutating spam replies so that they look different should not
be too hard... 

Are there any ideas we can borrow from anti-spam email schemes?


> > This thesis is very helpful for understanding the Kad network which I
> > think is the most active kademlia based DHT network.
> >
> > Daniel Stutzbach and Reza Rejaie
> > (http://www.cs.uoregon.edu/~reza/PUB/infocom06-kad.pdf) estimated KAD
> > had about 1 million "DHT" nodes and the number of hops was 2.7. (Some
> > other papers reported more nodes...) And there are several million
> > firewalled users using the DHT service provided by the 1 million DHT
> > nodes.
> 
> From what I see, these 2.7 hops don't mean 2.7 requests, but rather
a multiple 
> of that due to parallel requests (and 2.7 is the theoretical value). 
> 
> Without parallelism, their resulst show about 3.2 hops (which is
still very 
> good). 
> 
> That's a mean value, so it would be interesting to have it split by
popularity 
> of the hash. 
> 

Yes, 3.2 is good. We still need parallelism though, at least when we
don't want to wait for timeout due to dead next hop... So a fixable
lookup protocol makes sense here. For example, for lookups we don't
care about speed, parallelism=1, else parallelism=3...

> 
> For example searching for "mp3" on Gnutella has about 2 hops and a
parallelism 
> of 1: The leaf contacts only one of its UPs and the UP just replies
with the 
> results of its other leaves, and stops after that, since it got a
sufficient 
> number of results. 
> 

That should be count as one (iterative) hop (as in Kademlia), right? 

> > I agree with "how do you find out before searching that your search
> > will be a rare one?". This is a problem we need to solve. :)
> 
> Maybe it would be possible to always do a Gnutella search first, and
if that 
> generates less than 300 results, fire up a Kad search afterwards. 
> 


It works and we can find rare files. But it doesn't satisfy the first
requirement in our requirement list: 1) Fast search and low overhead.
Having a requirement list up front is useful. ;-)


Best,
Peng