Re: Multithreaded nDPI

Steve,

> On 14 Mar 2019, at 16:07, Steve Hill <[email protected]> wrote:
> 
> On 14/03/2019 09:31, Simone Mainardi wrote:
> 
>>> How does this impact the detection process?
>> Each thread performs the detection independently from the other. nDPI
>> detection is on a per-thread basis.
> 
> Thanks for your help so far, but I'm not sure this answers my question, so I'll try to give a more real world example:
> 
> The IRC protocol allows clients to set up peer to peer connections directly between two clients (DCC) to transmit files.  This works by one IRC client sending a private message to another client via the server, containing:
>  DCC SEND <filename> <my_address> <my_port>
> The receiving client then makes a connection directly to <my_address>:<my_port>.
> 
> 
> In single threaded operation:
> 
> 1. A client makes a connection to an IRC server.
> 2. nDPI creates a new flow for the connection (we'll call it flow[0]).
> 3. ndpi_search_irc_tcp() detects that this is an IRC flow and marks it as NDPI_PROTOCOL_IRC.
> 4. The client sends a DCC request request over flow[0].
> 5. ndpi_search_irc_tcp() examines the traffic, spots the DCC request and records <my_port> within flow[0]->src->irc_port and flow[0]->dst->irc_port.
> 6. The other party makes a connection to <my_address>:<my_port>.
> 7. nDPI creates a new flow for the connection (we'll call it flow[1]), and flow[1]->dst == flow[0]->src since one end is the client that we already know about.
> 8. ndpi_search_irc_tcp() finds that the destination port of flow[1] is recorded within flow[1]->dst->irc_port, and therefore infers that this is an IRC DCC flow and marks it as NDPI_PROTOCOL_IRC.
> 
> 
> Now compare to multi-threaded operation, using a 5-tuple hash to assign each flow to a different thread.  Each thread maintains separate flows and peers data to allow it to be lockless:
> 
> 1. A client makes a connection to an IRC server.
> 2. nDPI creates a new flow for the connection (we'll call it flow[0]) in thread 0.
> 3. ndpi_search_irc_tcp() detects that this is an IRC flow and marks it as NDPI_PROTOCOL_IRC.
> 4. The client sends a DCC request request over flow[0].
> 5. ndpi_search_irc_tcp() examines the traffic, spots the DCC request and records <my_port> within thread[0]->flow[0]->src->irc_port and thread[0]->flow[0]->dst->irc_port.
> 6. The other party makes a connection to <my_address>:<my_port>.
> 7. nDPI creates a new flow for the connection (we'll call it flow[1]) which could be in any thread since we're selecting thread based on a 5-tuple hash.  We'll assume it is handled by in thread 1.
> 8. Since thread[1]->flow[1]->dst and thread[0]->flow[0]->src are NOT the same data structure, ndpi_search_irc_tcp() DOES NOT find that the destination port of flow[1] is recorded within thread[1]->flow[1]->dst->irc_port, and therefore fails to detect this flow as IRC.
> 
> This kind of pattern is followed by quite a few dissectors, whereby data recorded in the peer structures when examining one flow is later used to detect another flow.  As far as I can see, this cannot work when the peer data is not shared between threads and the flows are being divided between the threads based on a 5-tuple hash.  I would certainly welcome some advice on this.
> 
> 
>>> ndpi_detection_module_struct appears to include various caches that
>>> are shared between flows, such as ookla_cache.  If different flows
>>> see different versions of those caches, is that not going to break
>>> some detections?
>> Nothing is going to break.
> 
> Can you explain how ookla_cache is used within this struct?  It looks to me as though this is used by the http dissector to record IP addresses that are associated with Ookla servers, and that the Ookla dissector then uses that IP address cache to detect other flows as NDPI_PROTOCOL_OOKLA.
> 
> If these flows do not all share the same ndpi_detection_module_struct, I cannot see how Ookla detection won't break?


The point is that, in general, packets (and, thus, flows) are NOT being balanced among threads using the 5-tuple hash, neither in ntopng nor in nDPI. All the traffic coming from an interface is processed by a single thread.

If you have to use an hash function for the balancing (e.g., when using RSS) then you have to pick a function that guarantees all the necessary packets for the detection of the applications you are interested into are delivered to the same thread. Typically, 5-tuple works OK as src->dst and dst->src packets are hashed to the same thread. 

nDPI structures have not been designed to be used concurrently by multiple threads to avoid costly synchronization mechanisms.


Simone

> 
> 
> Many thanks.
> 
> -- 
> - Steve Hill
>   Technical Director | Cyfarwyddwr Technegol
>   Opendium    Online Safety & Web Filtering     http://www.opendium.com
>               Diogelwch Ar-Lein a Hidlo Gwefan
> 
>   Enquiries | Ymholiadau:   [email protected]     +44-1792-824568
>   Support   | Cefnogi:      [email protected]   +44-1792-825748
> 
> ------------------------------------------------------------------------
> Opendium Limited is a company registered in England and Wales.
> Mae Opendium Limited yn gwmni sydd wedi'i gofrestru yn Lloegr a Chymru.
> 
> Company No. | Rhif Cwmni:   5465437
> Highfield House, 1 Brue Close, Bruton, Somerset, BA10 0HY, England.
> _______________________________________________
> Ntop-dev mailing list
> [email protected]
> http://listgateway.unipi.it/mailman/listinfo/ntop-dev