Implementing a Gnutella-structured network only with basic HTTP possible?

Hi, 

I've been thinking about ways to write networks with the structure of Gnutella 
using standards as much as possible (so programmers can mostly utilize 
existing libraries), and it got me to think about ways to implement the 
structures in plain HTTP. 

Currently Gnutella uses HTTP for the handshake and then switches to the binary 
stream - interspersed with rich XML. 

To make it easier to implement Gnutella servents, this could be switched to 
full HTTP, where queries from the client "upwards" would be simple POST 
requests (with the kind of query advertised in the headers) and queries from 
the server "downwards" would be put into the reply. 

To make the communication bidirectional, the server could require the client 
to send a ping (GET request) at least once every second, lest it would drop 
the connection. 

Messages could then be collated, so that each request (and reply) can contain 
more than one search or result, and request body compression can reap higher 
benefits. The exact maximum number of requests per second needs testing, but at 
most 5 collations per second should give fast enough connection speed (and 
automatic randomization of hosts to forward a message to). 

Aside from the basic network layer (transport of messages between servents) it 
should be possible to use the basic workings of Gnutella (Bootstrap, DQ, QRP, 
UP+leaf, …) mostly unchanged. UDP based transport (OOB, UHC) isn't touched. 


What this would gain compared to the current Gnutella network layer: 

* A Gnutella network could be implemented using only standard libraries. That 
way the barrier for entry for new developers would lower significantly 
(especially if they want to start with "thin servents" without ultrapeer 
capability). 

* "passive Gnutella" could be added on top of normal webservers. These would 
only communicate by sending out replies to client requests but wouldn't start 
any connections themselves. 

* The types of requests could be changed more easily, since they could be 
advertised in the header of every request. 


What this would cost: 

* Higher round-trip-time in "downward" direction. A request in downward 
direction would take a maximum mean time of 0.5 seconds (0.5*ping-time) per 
hop. Back when Gnutella was a pure broadcast network, that would have cost 
much response time, but since dynamic querying imposes wait times anyway (on 
sending), this shouldn't have too much impact. The main impact would be 
reduced response time from the direct leaves. The mean time per downward query 
would be reduced in areas where nodes have a high query throughput (since 
replies to queries can include downward queries). 

* Queries would travel a bit slower. With 32 in a connections (ultrapeer) in 
an active netowork (5 messages per second, evenly spaced) the average time 
till a message is forwarded to the first connection would be less than 10ms 
(per hop), and with 5 connections (leaf) it would still be only 40ms. 

* The full symmetry would be lost. The "upward" direction would have faster 
replies, since in a mostly inactive part of the network (one message per 
second) a client would send its own queries directly, but receive queries only 
when it sends. 

* The bandwidth overhead for connections would rise a bit, since every upward 
query would be a HTTP request (to at most 5 per second - otherwise collated 
with better compression) and every set of downward requests would be a 
HTTP request. Also the compression would be per request and not for the whole 
stream. 

* No compatibility to the current communication (but most parsing should be 
easily reusable). 


I'd love to hear your thoughts on the idea. 

Do you think it would be useful - and help getting more developers into 
Gnutella again?

Did I miss something which can't be done with basic HTTP? 

Best wishes, 
Arne

--- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- 
   - singing a part of the history of free software -
              http://infinite-hands.draketo.de



[Non-text portions of this message have been removed]