DHT Node Push-Proxy Endpoint value publishing
Raphael Manfredi <[email protected]> Fri, 22 Apr 2011 14:31:04 +0200
| Newsgroups | gmane.network.gnutella.devel |
|---|---|
| Organization | Home, Grenoble, France |
| Message-ID | <[email protected]> |
--U5G9OEwjdi9kXlSNb2BrOgfkaYlppQEMhevwDK2
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
Node Push-Proxy Endpoint
Version 0.0
Raphael Manfredi
<[email protected]>
April 22nd, 2011
1. INTRODUCTION
With the advent of the Distributed Hash Table in Gnutella, it has become
trivial to easily and quickly locate push-proxies of firewalled servents
who happen to also support the DHT. Indeed, DHT-enabled servents publish
a so-called PROX value listing the known push-proxies of the host. A non-
firewalled host also publishes PROX entries, listing itself as a push-proxy.
Each time a firewalled host changes its push-proxy set, a new PROX value is
generated and published in the DHT.
Unfortunately, there are still lots of old Gnutella servents (acting as
leaves on the network) that will never support the DHT and which therefore
cannot be easily located by modern DHT-enabled hosts.
However, the majority of ultrapeers in Gnutella is made of modern servents
(LimeWire derivatives, gtk-gnutella) supporting the DHT. These hosts can
therefore spot whether their leaves are firewalled and support the DHT and
publish that they act as push-proxies endpoints for these legacy leaves.
This publishing is made through NOPE values.
2. NOPE VALUE SPECIFICATION
A NOPE value is a DHT value of type "NOPE" whose payload contains the
following GGEP keys:
guid the servent's GUID as raw 16 bytes
port our listening port for push-proxy messages
tls if present, indicates that we support TLS
The "guid" key is the GUID of the leaf for which we act as a push-proxy.
3. PERIODIC PUBLISHING
After handshaking with a Gnutella leaf and exchanging the features via
a specific vendor message, the ultrapeer will know whether the leaf supports
the DHT. When it receives a push-proxy request, it will know the leaf is
firewalled and learn about its GUID.
Optionally, ultrapeers can also publish NOPE values for non-firewalled leaves
not supporting the DHT. Their GUID can be gathered by monitoring the query
hits these leaves will generate and send back through the ultrapeer.
Publishing should occur regularly, every 50 minutes (the expiration time
of a NOPE entry is 1 hour).
4. GUID LOOKUPS
A DHT node looking to contact a node for which only the GUID is known
used to request the lookup of a PROX value, hashing the binary GUID to
construct the SHA1 key.
Since we can't know a priori whether the host to whom this GUID belongs will
publish it in the DHT or not, we can now look for NOPE values as well.
Unfortunately, looking for a PROX first, then a NOPE as a fall-back is
inefficient.
Therefore, DHT nodes should instead look for "ANY" value, and prepare for
handling any PROX or NOPE values they could get back out of that lookup.
If they get back a PROX, they have the full set of current push-proxies
for the node.
If they get back a NOPE, the creator of that value is one push-proxy of
the host (and several different NOPE values can then be received, one for
each ultrapeer which is a neighbour of that host). The set of push-proxies
can therefore be re-constructed.
5. HTTP HEADER EXTENSION
Knowing the GUID of a remote host is valuable now, because if that host
is connected somewhere, firewalled or not and DHT-capable or not, you
can find it through a DHT lookup of its GUID. You'll send a PUSH and
will get connected.
During subsequent HTTP exchanges, you may notice the server is not firewalled
(if it sends an X-Node header in the HTTP reply) and will be able to contact
the servent directly next time.
The following HTTP header should therefore be sent back by all Gnutella
servers which are not firewalled:
X-GUID: 7d9af85c55f9543be4da8ae9d7b1944f
It will tell downloaders about the servent's GUID so that this host can be
located again at a later time, should it change its IP address.
Firewalled servents don't need to generate X-GUID because they already
emit another header (X-FW-Node-Info) which gives out their GUID along
with their current push-proxy set.
That header has the following form in LimeWire:
X-FW-Node-Info: 9DBC52EEEBCA2C8A79036D626B959900;fwt/1;
26252:85.182.49.3;
pptls=E;69.12.88.95:1085;64.53.20.48:804;66.17.23.159:343
However, gtk-gnutella continues to use the legacy X-Push-Proxies header
to send back its push-proxies, and uses a simpler X-FW-Node-Info to
hand out its GUID (when it is firewalled):
X-FW-Node-Info: 7d9af85c55f9543be4da8ae9d7b1944f; 26252:85.182.49.3
There is no "fwt/1" indication because gtk-gnutella does not support
transfers between firewalled hosts yet. The port:IP value following
is purely informative and give the local server known listening port and
its external IP. This allows direct connection attempts, in case the
servent mistakenly thinks it is firewalled. Giving the listening port
first is NOT a mistake -- it shows that is is not a push-proxy indication.
6. CONCLUSION
Support for NOPE is very simple, both on the server-side (simple
identification of non DHT-capable hosts + periodic NOPE publishing)
and on the client-side (just query for "ANY" value instead of explicit
PROX and be prepared to handle PROX or NOPE replies).
With NOPE support, a servent GUID becomes valuable information worth
exchanging, hence the addition of the X-GUID header to be emitted
by non-firewalled uploaders, firewalled ones emitting X-FW-Node-Info
as usual.
--U5G9OEwjdi9kXlSNb2BrOgfkaYlppQEMhevwDK2
Content-Type: text/html; charset=US-ASCII
Content-Transfer-Encoding: 7bit
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
</head>
<body style="background-color: #fff;">
<span style="display:none"> </span>
<!--~-|**|PrettyHtmlStartT|**|-~-->
<div id="ygrp-mlmsg" style="position:relative;">
<div id="ygrp-msg" style="z-index: 1;">
<!--~-|**|PrettyHtmlEndT|**|-~-->
<div id="ygrp-text" >
<p> Node Push-Proxy Endpoint<br>
Version 0.0<br>
<br>
Raphael Manfredi<br>
<<a href="mailto:Raphael_Manfredi%40pobox.com">[email protected]</a>><br>
<br>
April 22nd, 2011<br>
<br>
1. INTRODUCTION<br>
<br>
With the advent of the Distributed Hash Table in Gnutella, it has become<br>
trivial to easily and quickly locate push-proxies of firewalled servents<br>
who happen to also support the DHT. Indeed, DHT-enabled servents publish<br>
a so-called PROX value listing the known push-proxies of the host. A non-<br>
firewalled host also publishes PROX entries, listing itself as a push-proxy.<br>
Each time a firewalled host changes its push-proxy set, a new PROX value is<br>
generated and published in the DHT.<br>
<br>
Unfortunately, there are still lots of old Gnutella servents (acting as<br>
leaves on the network) that will never support the DHT and which therefore<br>
cannot be easily located by modern DHT-enabled hosts.<br>
<br>
However, the majority of ultrapeers in Gnutella is made of modern servents<br>
(LimeWire derivatives, gtk-gnutella) supporting the DHT. These hosts can<br>
therefore spot whether their leaves are firewalled and support the DHT and<br>
publish that they act as push-proxies endpoints for these legacy leaves.<br>
This publishing is made through NOPE values.<br>
<br>
2. NOPE VALUE SPECIFICATION<br>
<br>
A NOPE value is a DHT value of type "NOPE" whose payload contains the<br>
following GGEP keys:<br>
<br>
guid the servent's GUID as raw 16 bytes<br>
port our listening port for push-proxy messages<br>
tls if present, indicates that we support TLS<br>
<br>
The "guid" key is the GUID of the leaf for which we act as a push-proxy.<br>
<br>
3. PERIODIC PUBLISHING<br>
<br>
After handshaking with a Gnutella leaf and exchanging the features via<br>
a specific vendor message, the ultrapeer will know whether the leaf supports<br>
the DHT. When it receives a push-proxy request, it will know the leaf is<br>
firewalled and learn about its GUID.<br>
<br>
Optionally, ultrapeers can also publish NOPE values for non-firewalled leaves<br>
not supporting the DHT. Their GUID can be gathered by monitoring the query<br>
hits these leaves will generate and send back through the ultrapeer.<br>
<br>
Publishing should occur regularly, every 50 minutes (the expiration time<br>
of a NOPE entry is 1 hour).<br>
<br>
4. GUID LOOKUPS<br>
<br>
A DHT node looking to contact a node for which only the GUID is known<br>
used to request the lookup of a PROX value, hashing the binary GUID to<br>
construct the SHA1 key.<br>
<br>
Since we can't know a priori whether the host to whom this GUID belongs will<br>
publish it in the DHT or not, we can now look for NOPE values as well.<br>
Unfortunately, looking for a PROX first, then a NOPE as a fall-back is<br>
inefficient.<br>
<br>
Therefore, DHT nodes should instead look for "ANY" value, and prepare for<br>
handling any PROX or NOPE values they could get back out of that lookup.<br>
<br>
If they get back a PROX, they have the full set of current push-proxies<br>
for the node.<br>
<br>
If they get back a NOPE, the creator of that value is one push-proxy of<br>
the host (and several different NOPE values can then be received, one for<br>
each ultrapeer which is a neighbour of that host). The set of push-proxies<br>
can therefore be re-constructed.<br>
<br>
5. HTTP HEADER EXTENSION<br>
<br>
Knowing the GUID of a remote host is valuable now, because if that host<br>
is connected somewhere, firewalled or not and DHT-capable or not, you<br>
can find it through a DHT lookup of its GUID. You'll send a PUSH and<br>
will get connected.<br>
<br>
During subsequent HTTP exchanges, you may notice the server is not firewalled<br>
(if it sends an X-Node header in the HTTP reply) and will be able to contact<br>
the servent directly next time.<br>
<br>
The following HTTP header should therefore be sent back by all Gnutella<br>
servers which are not firewalled:<br>
<br>
X-GUID: 7d9af85c55f9543be4da8ae9d7b1944f<br>
<br>
It will tell downloaders about the servent's GUID so that this host can be<br>
located again at a later time, should it change its IP address.<br>
<br>
Firewalled servents don't need to generate X-GUID because they already<br>
emit another header (X-FW-Node-Info) which gives out their GUID along<br>
with their current push-proxy set.<br>
<br>
That header has the following form in LimeWire:<br>
<br>
X-FW-Node-Info: 9DBC52EEEBCA2C8A79036D626B959900;fwt/1;<br>
26252:85.182.49.3;<br>
pptls=E;69.12.88.95:1085;64.53.20.48:804;66.17.23.159:343<br>
<br>
However, gtk-gnutella continues to use the legacy X-Push-Proxies header<br>
to send back its push-proxies, and uses a simpler X-FW-Node-Info to<br>
hand out its GUID (when it is firewalled):<br>
<br>
X-FW-Node-Info: 7d9af85c55f9543be4da8ae9d7b1944f; 26252:85.182.49.3<br>
<br>
There is no "fwt/1" indication because gtk-gnutella does not support<br>
transfers between firewalled hosts yet. The port:IP value following<br>
is purely informative and give the local server known listening port and<br>
its external IP. This allows direct connection attempts, in case the<br>
servent mistakenly thinks it is firewalled. Giving the listening port<br>
first is NOT a mistake -- it shows that is is not a push-proxy indication.<br>
<br>
6. CONCLUSION<br>
<br>
Support for NOPE is very simple, both on the server-side (simple<br>
identification of non DHT-capable hosts + periodic NOPE publishing)<br>
and on the client-side (just query for "ANY" value instead of explicit<br>
PROX and be prepared to handle PROX or NOPE replies).<br>
<br>
With NOPE support, a servent GUID becomes valuable information worth<br>
exchanging, hence the addition of the X-GUID header to be emitted<br>
by non-firewalled uploaders, firewalled ones emitting X-FW-Node-Info<br>
as usual.<br>
</p>
</div>
<!--~-|**|PrettyHtmlStart|**|-~-->
<div style="color: #fff; height: 0;">__._,_.___</div>
<div id="ygrp-actbar" style="clear: both; margin-bottom: 10px; white-space: nowrap; color: #666; padding-top: 15px;">
<div>
<a href="mailto:[email protected]?subject=Re%3A%20DHT%20Node%20Push-Proxy%20Endpoint%20value%20publishing" style="margin-right: 0; padding-right: 0;">
Reply to <span style="font-weight: 700;">sender</span></a> |
<a href="mailto:[email protected]?subject=Re%3A%20DHT%20Node%20Push-Proxy%20Endpoint%20value%20publishing">
Reply to <span style="font-weight: 700;">group</span></a> |
<a href="http://groups.yahoo.com/group/the_gdf/post;_ylc=X3oDMTJxMDBwbjR1BF9TAzk3MzU5NzE0BGdycElkAzI2ODQyNTMEZ3Jwc3BJZAMxNzA1MDE2MDYxBG1zZ0lkAzIzNzc4BHNlYwNmdHIEc2xrA3JwbHkEc3RpbWUDMTMwMzQ3NTQ4Nw--?act=reply&messageNum=23778">Reply <span style="font-weight: 700;">via web post</span></a> |
<a href="http://groups.yahoo.com/group/the_gdf/post;_ylc=X3oDMTJlMmQ4bWEyBF9TAzk3MzU5NzE0BGdycElkAzI2ODQyNTMEZ3Jwc3BJZAMxNzA1MDE2MDYxBHNlYwNmdHIEc2xrA250cGMEc3RpbWUDMTMwMzQ3NTQ4Nw--" style="font-weight: 700;">Start a New Topic</a>
</div>
<a href="http://groups.yahoo.com/group/the_gdf/message/23778;_ylc=X3oDMTM2YjFvYzdqBF9TAzk3MzU5NzE0BGdycElkAzI2ODQyNTMEZ3Jwc3BJZAMxNzA1MDE2MDYxBG1zZ0lkAzIzNzc4BHNlYwNmdHIEc2xrA3Z0cGMEc3RpbWUDMTMwMzQ3NTQ4NwR0cGNJZAMyMzc3OA--">Messages in this topic</a>
(<span style="font-weight: 700;">1</span>)
</div>
<!------- Start Nav Bar ------>
<!-- |**|begin egp html banner|**| -->
<div id="ygrp-vital" style="background-color: #e0ecee; font-family: Verdana; font-size: 10px; margin-bottom: 10px; padding: 10px;">
<span id="vithd" style="font-weight: bold; color: #333; text-transform: uppercase; ">Recent Activity:</span>
<ul style="list-style-type: none; margin: 0; padding: 0; display: inline;">
</ul>
<div style="clear: both; padding-top: 2px; color: #1e66ae;">
<a href="http://groups.yahoo.com/group/the_gdf;_ylc=X3oDMTJlcjNsNzc2BF9TAzk3MzU5NzE0BGdycElkAzI2ODQyNTMEZ3Jwc3BJZAMxNzA1MDE2MDYxBHNlYwN2dGwEc2xrA3ZnaHAEc3RpbWUDMTMwMzQ3NTQ4Nw--" style="text-decoration: none;">Visit Your Group</a>
</div>
</div>
<div id="ft" style="font-family: Arial; font-size: 11px; margin-top: 5px; padding: 0 2px 0 0; clear: both;">
<a href="http://groups.yahoo.com/;_ylc=X3oDMTJkamtmNjVnBF9TAzk3MzU5NzE0BGdycElkAzI2ODQyNTMEZ3Jwc3BJZAMxNzA1MDE2MDYxBHNlYwNmdHIEc2xrA2dmcARzdGltZQMxMzAzNDc1NDg3" style="float: left;"><img src="http://l.yimg.com/a/i/us/yg/logo/us.gif" height="15" width="137" alt="Yahoo! Groups" style="border: 0;"/></a>
<div style="color: #747575; float: right;">Switch to: <a href="mailto:[email protected]?subject=Change Delivery Format: Traditional" style="text-decoration: none;">Text-Only</a>, <a href="mailto:[email protected]?subject=Email Delivery: Digest" class="margin-rt" style="text-decoration: none;">Daily Digest</a> • <a href="mailto:[email protected]?subject=Unsubscribe" style="text-decoration: none;">Unsubscribe</a> • <a href="http://docs.yahoo.com/info/terms/" style="text-decoration: none;">Terms of Use</a></div>
</div>
<!-- |**|end egp html banner|**| -->
</div> <!-- ygrp-msg -->
<!-- Sponsor -->
<!-- |**|begin egp html banner|**| -->
<div id="ygrp-sponsor" style="width:160px; float:right; clear:none; margin:0 0 25px 0; background: #fff;">
<!-- Start Recommendations -->
<div id="ygrp-reco">
</div>
<!-- End Recommendations -->
</div> <!-- |**|end egp html banner|**| -->
<div style="clear:both; color: #FFF; font-size:1px;">.</div>
</div>
<img src="http://geo.yahoo.com/serv?s=97359714/grpId=2684253/grpspId=1705016061/msgId=23778/stime=1303475487/nc1=1/nc2=2/nc3=3" width="1" height="1"> <br>
<div style="color: #fff; height: 0;">__,_._,___</div>
<!--~-|**|PrettyHtmlEnd|**|-~-->
</body>
<!--~-|**|PrettyHtmlStart|**|-~-->
<head>
<style type="text/css">
<!--
#ygrp-mkp {
border: 1px solid #d8d8d8;
font-family: Arial;
margin: 10px 0;
padding: 0 10px;
}
#ygrp-mkp hr {
border: 1px solid #d8d8d8;
}
#ygrp-mkp #hd {
color: #628c2a;
font-size: 85%;
font-weight: 700;
line-height: 122%;
margin: 10px 0;
}
#ygrp-mkp #ads {
margin-bottom: 10px;
}
#ygrp-mkp .ad {
padding: 0 0;
}
#ygrp-mkp .ad p {
margin: 0;
}
#ygrp-mkp .ad a {
color: #0000ff;
text-decoration: none;
}
#ygrp-sponsor #ygrp-lc {
font-family: Arial;
}
#ygrp-sponsor #ygrp-lc #hd {
margin: 10px 0px;
font-weight: 700;
font-size: 78%;
line-height: 122%;
}
#ygrp-sponsor #ygrp-lc .ad {
margin-bottom: 10px;
padding: 0 0;
}
a {
color: #1e66ae;
}
#actions {
font-family: Verdana;
font-size: 11px;
padding: 10px 0;
}
#activity {
background-color: #e0ecee;
float: left;
font-family: Verdana;
font-size: 10px;
padding: 10px;
}
#activity span {
font-weight: 700;
}
#activity span:first-child {
text-transform: uppercase;
}
#activity span a {
color: #5085b6;
text-decoration: none;
}
#activity span span {
color: #ff7900;
}
#activity span .underline {
text-decoration: underline;
}
.attach {
clear: both;
display: table;
font-family: Arial;
font-size: 12px;
padding: 10px 0;
width: 400px;
}
.attach div a {
text-decoration: none;
}
.attach img {
border: none;
padding-right: 5px;
}
.attach label {
display: block;
margin-bottom: 5px;
}
.attach label a {
text-decoration: none;
}
blockquote {
margin: 0 0 0 4px;
}
.bold {
font-family: Arial;
font-size: 13px;
font-weight: 700;
}
.bold a {
text-decoration: none;
}
dd.last p a {
font-family: Verdana;
font-weight: 700;
}
dd.last p span {
margin-right: 10px;
font-family: Verdana;
font-weight: 700;
}
dd.last p span.yshortcuts {
margin-right: 0;
}
div.attach-table div div a {
text-decoration: none;
}
div.attach-table {
width: 400px;
}
div.file-title a, div.file-title a:active, div.file-title a:hover, div.file-title a:visited {
text-decoration: none;
}
div.photo-title a, div.photo-title a:active, div.photo-title a:hover, div.photo-title a:visited {
text-decoration: none;
}
div#ygrp-mlmsg #ygrp-msg p a span.yshortcuts {
font-family: Verdana;
font-size: 10px;
font-weight: normal;
}
.green {
color: #628c2a;
}
.MsoNormal {
margin: 0 0 0 0;
}
o {
font-size: 0;
}
#photos div {
float: left;
width: 72px;
}
#photos div div {
border: 1px solid #666666;
height: 62px;
overflow: hidden;
width: 62px;
}
#photos div label {
color: #666666;
font-size: 10px;
overflow: hidden;
text-align: center;
white-space: nowrap;
width: 64px;
}
#reco-category {
font-size: 77%;
}
#reco-desc {
font-size: 77%;
}
.replbq {
margin: 4px;
}
#ygrp-actbar div a:first-child {
/* border-right: 0px solid #000;*/
margin-right: 2px;
padding-right: 5px;
}
#ygrp-mlmsg {
font-size: 13px;
font-family: Arial, helvetica,clean, sans-serif;
*font-size: small;
*font: x-small;
}
#ygrp-mlmsg table {
font-size: inherit;
font: 100%;
}
#ygrp-mlmsg select, input, textarea {
font: 99% Arial, Helvetica, clean, sans-serif;
}
#ygrp-mlmsg pre, code {
font:115% monospace;
*font-size:100%;
}
#ygrp-mlmsg * {
line-height: 1.22em;
}
#ygrp-mlmsg #logo {
padding-bottom: 10px;
}
#ygrp-mlmsg a {
color: #1E66AE;
}
#ygrp-msg p a {
font-family: Verdana;
}
#ygrp-msg p#attach-count span {
color: #1E66AE;
font-weight: 700;
}
#ygrp-reco #reco-head {
color: #ff7900;
font-weight: 700;
}
#ygrp-reco {
margin-bottom: 20px;
padding: 0px;
}
#ygrp-sponsor #ov li a {
font-size: 130%;
text-decoration: none;
}
#ygrp-sponsor #ov li {
font-size: 77%;
list-style-type: square;
padding: 6px 0;
}
#ygrp-sponsor #ov ul {
margin: 0;
padding: 0 0 0 8px;
}
#ygrp-text {
font-family: Georgia;
}
#ygrp-text p {
margin: 0 0 1em 0;
}
#ygrp-text tt {
font-size: 120%;
}
#ygrp-vital ul li:last-child {
border-right: none !important;
}
-->
</style>
</head>
<!--~-|**|PrettyHtmlEnd|**|-~-->
</html>
<!-- end group email -->
--U5G9OEwjdi9kXlSNb2BrOgfkaYlppQEMhevwDK2--