3 Milestones for "R5N (2021-02-038)"/C implementation reached

Dear GNUnet community,

I'm happy to announce that as of the very recent
14da59e43311204cadc883cd3688be08bd77b6a8

it is also believed that the C code correctly implements the updated 
LSD0004 specification, for the first three milestones. In addition to 
fixing stylistic issues in the protocol (like message layouts) that 
arose from trying to write a clean specification, there were also 
various minor and major bugfixes where the routing implementation did 
not quite do what it should have, or simply could do better as we 
discovered when trying to write down what the C code was actually doing.

Furthermore, a new DHT underlay mechanism was added. Now the DHT 
implementation can use both GNUnet-core and/or a new simplistic UDP 
protocol as 'underlays'. The UDP protocol should be great for DHT 
interoperability testing (no encryption, no complicated GNUnet 
core+transport protocols to support first). Moreover, the DHT underlay 
mechanism is *general* and allows for additional underlay plugins to be 
defined. This will make it easy to run R5N over other P2P networks (like 
I2P) in the future. Because a single DHT process can use multiple 
underlays concurrently, this even allows the creation of one big R5N DHT 
that spans multiple P2P networks, as long as there are (some) peers that 
run DHT instances and have configured underlays in both P2P networks.

Last but not least, the latest implementation adds signatures to the 
paths (if path tracking is enabled), such that if an application enables 
DHT path tracking, it can actually be assured that the message did 
traverse the peers on the path and hence that path through the topology 
did exist. This prevents certain types of attacks on the CADET layer.

The next (milestone 4) task for the C agenda is interoperability testing 
with other implementation(s), which we will do jointly as these other 
implementations progress.


Happy hacking!

Christian

On 6/11/22 20:29, Bernd Fix wrote:
> Dear GNUnet community,
> 
> I am happy to announce the completion of the first milestone of the "Go 
> implementation of the R5N DHT" for the NLnet-funded project "R5N 
> (2021-02-038)". The implementation is not based on the current GNUnet 
> DHT/Transport protocol, but on a new protocol defined in 
> https://lsd.gnunet.org/lsd0004. The new specification corrects some 
> design decisions in the old protocol and improves security in many 
> places (e.g. signed DHT-PATHes and HELLOs).
> 
> The source code is written for Go1.18+ (it makes use of generics in some 
> places); it can be found in the GNUnet Git-Repository at 
> "https://git.gnunet.org/gnunet-go.git/" with tag "v0.1.27" in the master 
> branch. The code for Milestone 1 covers the following areas (the main 
> source references are given in square brackets as a starting point):
> 
> # Defining and implementing base data structures:
> 
> ## Routing table:
> 
> ### K-Buckets:
>    The code provides a Kademlia-like bucket implementation.
>    [gnunet/dht/routingtable.go]
> 
> ### network addresses:
>    A generic, GNUnet-compatible address implementation based on URIs
>    with many helpers (like thread-safe maps). [gnunet/util/address.go]
> 
> ### distance metric:
>    PeerAddress type [gnunet/dht/routingtable.go]
> 
> ## Infrastructure messages
> 
> ### add/update peers:
>    A complete event signalling framework for transport layer events has
>    been implemented. Listeners can define filters for events they are
>    interested in. [gnunet/core/event.go]
> 
> # Defining and implementing base processes:
> 
> ## Bootstrapping a node:
>    Connect to the first node via HELLO URL or plain network address. The
>    code implements HELLO messages, blocks and URLs as defined in the
>    LSD0004 spec to send and receive HELLO messages (including signing
>    and verification).
>    [gnunet/core/core.go, gnunet/service/dht/blocks/hello.go]
> 
> ## Message transport:
>    The transport layer has been completely re-written from the previous
>    version. It supports the "ip+udp" protocol and has a three tier layout
>    (Endpoint/Transport/Core) to separate functionality. Currently only
>    PacketEndpoints (UDP) are implemented.  [gnunet/transport/*.go]
> 
> ## Persistence:
>    A key/value store API [gnunet/service/store.go] for DHT blocks and
>    stringed key/value pairs. DHT block storage is disk-based, whereas
>    for string key/value pairs multiple implementations are avaiable
>    (Redis,SQL). Eviction is based on expiry (mandatory) and the
>    "lifeSpan * size / usedCount" ratio for optional removals.
> 
> ## Block verification API:
>    Verify methods incorporated into DHT query instances and DHT blocks
>    [gnunet/service/dht/blocks/generic.go]
> 
> ## Routing table lookup:
>    The lookup methods as described in LSD0004 are implemented.
>    [gnunet/service/dht/routingtable.go]
> 
> 
> Please bear in mind that the code for this milestone compiles and run 
> the unit tests, but it is not complete to run as a fully functional 
> application. This is just the starting point for the next milestones; I 
> hope to complete the next milestone in a rather short time.
> 
> If you have any questions or suggestions, please get back to me via 
> email. I am happy to answer them in due time.
> 
> Happy hacking, Bernd.
>