Re: Re: Using the DHT to find download sources

Am Donnerstag 24 Juli 2008 18:14:54 schrieb pwang708:
> works well against the spammers you described above. Considering an
> attack similar to the whitewashing attack, join many nodes to the
> network, vote honestly on everything expect on the files the spammer
> don't want people to download. Note that the spammer nodes can always
> learn honest nodes' votes on other things from the network...

The Credence system has a safeguard against this which works as long as there 
are more than some very few spam files in the network: 

Credence creates a trust metric for every other user. 

If the a user votes (vastly) different from me, the metric can go as far as 
inversing his votes: What he thinks good is treated as bad. 

So, if a spammer wants to vote his own files up, he must not vote positively 
on other spam (which users might already have voted as spam), but he must 
vote positively on many non-spam items (only those count which others have 
voted on). 

Since there are different spammer sin the network, most will vote similar real 
files up (they have to give many good votes), but they will try to promote 
different spam files (because as soon as a spam file gets marked as spam by 
too many users, the spammers votes will be disregarded, and if they didn't 
vote on very many good files they might even be inversed, so that their 
effort to promote too many different spam files gets turned into its 
opposite). 

And so, since their votes on good files will overlap, but their spam votes 
won't, they will promote the good files stronger than their own spam and 
likely help the network recognize good files in the end. 

Some spam files can survive in this, but the number of possible unfound spam 
files is finite, and if the spammers don't coordinate, they will spoil each 
others efforts. 


> BTW, The default spam replies can be recognized because they don't
> change? Mutating spam replies so that they look different should not
> be too hard...

These won't be voted up, though. But they could be used to spoil rare files. 

So what is see is: 

Credence can: 
 * stop advertising spam and
 * stop popular file masking spam. 

it can't: 
 * Stop direct attacks against rare file searches. 

But these direct attacks against rare file searches aren't what troubles me 
most, because the damage to rare file searches currently mostly comes from 
advertising spam, and that could be stopped, destroying the "advertising 
spam" business model and that way also vastly reducing spam which hinders 
rare file searches. 

> Are there any ideas we can borrow from anti-spam email schemes?

I just browsed a bit on that. 
Results: 

- They do statistical analysis of the content. 
One example is Bogofilter: 
http://bogofilter.sourceforge.net/faq.shtml#bogo-how
It doesn't look like it will work for the short query replies we get in 
Gnutella, though. 
Training might work, if the search message is taken into account. 
Also it could weed out the obnoxious long filenames of porn-spam. 
- description: http://www.linuxjournal.com/article/6467

- They have spam-tracking servers. One example is pyzor: 
http://pyzor.sourceforge.net/
"It works by computing signatures on the body of the content and
checking these signatures against a database of known spam."
~ from the Razor page whoose protocol pyzor implemented and improved. 
more detailed: 
- http://pyzor.sourceforge.net/blocking.html
Also I read that they check, if multiples of messages are floating around. 
Doesn't look like it could be used efficiently, either. The spammers already 
change the query replies dynamically. 

- Additional stuff: http://wiki.apache.org/spamassassin/TrustNetNotes

Long story short: They don't seem very applyable to p2p spam. 

> That should be count as one (iterative) hop (as in Kademlia), right?

Uh, yes. 

That means, rare searches would be 3-4 hops instead of 4-5. 

> It works and we can find rare files. But it doesn't satisfy the first
> requirement in our requirement list: 1) Fast search and low overhead.
> Having a requirement list up front is useful. ;-)

Low overhead: Mostly, since the requests only travel where they might find 
results (QRP). 

Fast search sadly not, but that wouldn't work with searching again for rare 
searches, either. 

To realize "fast search", you'd have to search on both networks 
simultaneously, which would violate "low overhead". 

So, why do you need fast search for rare files? Wouldn't a "Only few results 
found on fast query, begin rare files query" message to the user be more 
useful? 

The user knows "I see it's scarce, but my program takes care of it" and so he 
won't have problems with waiting a bit longer. For me this approach even 
seems more satisfying to users. 

That message could also be shown as soon as Dynamic Querying goes into "ask 
everyone" mode. 

It should be phrased carefully, though, to ensure that it states clearly that 
rare file queries aren't efficient when they search for popular files (it 
doesn't need to say "not good for the network", just "not good for popular 
files" - I don't know if my message really attained that goal). 

So I'd rephrase that first requirement to 
1) A good user experience and low overhead. 

We're talking filesharing, after all.  

Checking the requirements list against real world examples (aka use cases) 
helps refining it :) 

Could you post an updated version of the requirements list? 

Best wishes, 
Arne

-- Weblog: http://blog.draketo.de
-- Infinite Hands: http://infinite-hands.draketo.de - singing a part of the 
history of free software. 
-- Ein Würfel System: http://1w6.org - einfach saubere (Rollenspiel-) Regeln

-- PGP/GnuPG: http://draketo.de/inhalt/ich/pubkey.txt


[Non-text portions of this message have been removed]