DHT expiration questions

[email protected] Sat, 16 Aug 2008 07:15:51 +0000 (UTC)
Newsgroups gmane.network.gnutella.devel
Organization Home, Grenoble, France
Message-ID <[email protected]>
Hello,

Now that gtk-gnutella joins the Gnutella DHT, some holes in the original
LimeWire specifications for the DHT are surfacing.

What matters most is data expiration.

Since a DHT Value does not carry the original creation time of that
particular value, how do LimeWire nodes decide what to expire and when?

My fear is that with all the replication that goes on between the k-closest
nodes, it is easy to have stale data being permanently carried by the DHT,
preventing new fresh data from being stored (all the keys being full).

I start to notice many STORE requests at my node being refused because the
keys are full, precisely, and this particular node has been up for less
than a day...

Also gtk-gnutella is validating that the creator's contact address is valid
and is refusing STORE requests coming from 127.0.0.1, for instance, with
STORE_SC_BAD_CREATOR. I understand this could be because the node is
firewalled, but given that DHT PONGS carry the requestor's external address,
firewalled nodes could at least use that for their contact address.

I see a lot of lines such as:

DHT traffic from new firewalled node cde668edac5f8391f0ac55af9b34270f8191db28 at 127.0.0.1:0 (LIME v0.0)

in my logs, and this shows that these nodes do not learn their address
correctly.  Which is a problem.

Anyway, what is the policy LimeWire is using to expire data?  What are
the republishing and replication parameters?  How do you prevent stale
data from being replicated over and over (e.g. if my node comes up and
is given STORE from its k-closest, how can it know if that data is about
to expire at the original node?).

I know there are going to be far less GTKG nodes in the DHT than there are
LIME nodes, but it is in everyone's interest that common specifications goals
be shared.  GTKG is a "fair" node in that it will accept to store any data
(even opaque BINARY forms), and has large quotas.  Currently they are set
to 16 values per key, with 64 Ki values max (no limit on the number of keys,
everything is driven by values).

Thanks for at least sharing high-level design parameters.

Raphael

P.S: I've seen STORE requests for BINARY data with a length of zero bytes?
What is the purpose of storing empty BINARY data? Is that a bug?