Multithreaded nDPI

I'm looking for some advice on using nDPI in multithreaded code.

Some Googling tells me that I need a separate 
ndpi_detection_module_struct for each thread, but beyond that I can't 
find a lot of information.  The example ndpiReader code seems to only 
use threads when processing multiple independent pcap files or NICs. 
That's not directly applicable to my situation, as I'm processing a 
stream of packets from a single NIC so don't have completely independent 
data sources like that.

So, I can use a 5-tuple hash to pick a consistent thread for each flow. 
That would mean that each flow would consistently be processed with the 
same ndpi_detection_module_struct.

The documentation isn't very clear, but presumably each endpoint should 
have a single ndpi_id_struct, irrespective of how many flows it has. 
Therefore each of the ndpi_id_structs are shared between multiple flows 
and may therefore be used by any of the threads in association with any 
of the ndpi_detection_module_structs.  I can obviously lock the 
ndpi_id_structs so that each can only be used by one thread at a time, 
but I'm not sure whether using the same endpoint with multiple 
ndpi_detection_module_structs is going to be a problem?

Finally, I'm not sure how much information nDPI shares between flows via 
the ndpi_detection_module_struct.  i.e. when a flow is processed, does 
nDPI store any information in ndpi_detection_module_struct about what it 
has learnt and then use that information when processing other flows? 
If so, splitting the flows between multiple 
ndpi_detection_module_structs is fundamentally problematic.


I've also got a quick query about the ndpi_id_structs that are passed to 
ndpi_detection_process_packet.  The documentation just says that these 
are "source" and "destination", but isn't clear on whether these are the 
source/destination of the packet or of the flow.  Obviously reply 
packets will have the source/destination the opposite way around to the 
flow's source/destination.


Many thanks.

-- 
  - Steve Hill
    Technical Director | Cyfarwyddwr Technegol
    Opendium    Online Safety & Web Filtering     http://www.opendium.com
                Diogelwch Ar-Lein a Hidlo Gwefan

    Enquiries | Ymholiadau:   [email protected]     +44-1792-824568
    Support   | Cefnogi:      [email protected]   +44-1792-825748

------------------------------------------------------------------------
Opendium Limited is a company registered in England and Wales.
Mae Opendium Limited yn gwmni sydd wedi'i gofrestru yn Lloegr a Chymru.

Company No. | Rhif Cwmni:   5465437
Highfield House, 1 Brue Close, Bruton, Somerset, BA10 0HY, England.