RE: gen_udp:recv()
Avinash Dhumane <[email protected]>
| Newsgroups | gmane.comp.lang.erlang.general |
|---|---|
| Message-ID | <[email protected]> |
Indeed! As Jesper too pointed out, a sub-10-microsecond level turnaround time seems impossible to achieve in Erlang; the best I got is 60 microseconds, and average is in the order of hundreds of microseconds (almost close to a millisecond) I am taking the “tick-to-order” processing (in stock-market parlance) out of Erlang and into Rust to meet this ultra-low latency requirement. Thanks all for confirming the overall approach. From: Max Lapshin Sent: 07 April 2021 06:32 PM To: Avinash Dhumane Cc: Łukasz Niemier ; Erlang Questions Subject: Re: gen_udp:recv() I suppose that you really need the same as me. You need to receive _one_ big binary with an extra field containing messages boundaries and other meta info. On Wed, Apr 7, 2021 at 1:25 PM Avinash Dhumane <[email protected]> wrote: > > Our case is similar. The stock exchange disseminates the market movements (called, ticks) as stream of variable length messages (datagrams or packets), each less than 50 bytes, and each stamped with sequence number, over multicast (udp) streams, each stream replicated so that if a packet is missed (dropped) – due to the nature of underlying udp – it may be received on the replicated stream (before fallback on a tcp-based recovery server). > > > > The message rate (without replication) exceeds 100K per second. Issuing as many calls, at Erlang-application level, viz. gen_udp:recv, is impractical, though I suppose, at underlying OS system-call level, each datagram must be received using a separate call. Therefore, I believed, that Erlang might provide an abstraction, which packs a number of datagrams (packets / messages) arrived so far and deliver to the application, for further handling. One-to-one call from app to Erlang, and further from Erlang to OS, for each datagram, would seem too much taxing. > > > > > > From: Max Lapshin > Sent: 06 April 2021 10:20 PM > To: Łukasz Niemier > Cc: Erlang Questions > Subject: Re: gen_udp:recv() > > > > Exactly. > > > > We receive MPEG-TS _stream_ in multicast messages and it is very > > painful to receive so many messages (it can be around 90K per second > > totally or 300-900 per one stream per second) one by one. Reducing > > this about by 10 decreases CPU usage a lot. > > > > > > > > > > On Tue, Apr 6, 2021 at 6:31 PM Łukasz Niemier <[email protected]> wrote: > > > > > > > Well, I like the idea to receive 20-200 UDP messages in a single > > > > binary with separate structure holding mapping of this data. > > > > > > Then it sounds like you want stream protocol instead of packet protocol. > > > The whole idea of packet protocol is to receive messages in packets. > > > > > > -- > > > > > > Łukasz Niemier > > > [email protected] > > > > >