Re: Performance of term_to_binary vs Bbinary_to_term

Hi Thomas, 

Thanks for your input. 

Indeed, parsing being slower that serialisation is not necessarily surprising. However, when a difference in performance is, well, two orders of magnitude, that it may be reasonable to ask if term_to_binary/1 does any actual work. Put differently, is there any difference between internal and external term presentation?


V/



> And you need to do it piecemeal, with a lot of information about current steps. This is far more complex than translating a particular memory setting that you know the size of into a binary stream.





> On 07 Jun 2021, at 14:38, Thomas Depierre <[email protected]> wrote:
> 
> Hi Valentin.
> 
> Yes there is a pretty simple answer :) Parsing is far harder than serialization ! for parsing, you have to read a bit of the binary stream, find what type it is, then translate the data to a data type, which means allocating memory. On top of that you need to validate that it is not a mangled binary stream. And you need to do it piecemeal, with a lot of information about current steps. This is far more complex than translating a particular memory setting that you know the size of into a binary stream.
> 
> Now is there space for speedup ? maybe. It is always an interesting area to explore. But fundamentally, parsing being slower than serialisation is not surprising,
> 
> Thomas Depierre
> 
> On Mon, 7 Jun 2021 at 14:25, Valentin Micic <[email protected] <mailto:[email protected]>> wrote:
> Hmmm… I realised that my previous email may be seen as a comment rather than a question, so let me ask the question clearly.
> 
> Given that binary_to_term/1 is about two orders of magnitude slower than term_to_binary/1, is there anyone out there that may have a reasonable explanation for that?
> 
> Kind regards
> 
> V/
> 
>> On 06 Jun 2021, at 02:07, Valentin Micic <[email protected] <mailto:[email protected]>> wrote:
>> 
>> Hi all,
>> 
>> I did some performance measurement recently that included conversion of an arbitrary erlang term to its external binary representation via term_to_binary/1, as well as reversing the result using binary_to_term/1.
>> 
>> I’ve noticed that term_to_binary/1 is significantly faster than binary_to_term/1.
>> 
>> Also, I’ve observed that binary_to_term/1 performance gets considerably worse as complexity of specified term increases, whilst term_to_binary/1 maintains (more-less) steady performance.
>> 
>> (cig@MacBook-Pro)40> tconvert:run( a, 10000000 ).
>> 
>> term_to_binary/1 RETURN VALUE:<<131,100,0,1,97>>
>> REQUEST COUNT:10000000
>> ELAPSED TIME (usec):97070
>> TIME PER REQUEST (usec): 0.009707
>> PROJECTED RATE (req/sec): 103018440.30081384
>> 
>> binary_to_term/1 RETURN VALUE:a
>> REQUEST COUNT:10000000
>> ELAPSED TIME (usec):3383483
>> TIME PER REQUEST (usec): 0.3383483
>> PROJECTED RATE (req/sec): 2955534.2822765773
>> ok
>> 
>> (cig@MacBook-Pro)41> tconvert:run( {a,<<1,2,3>>, b, [1,2,3], c, {1,2,3}, d, #{a=>1, b=>2, c=>3}}, 10000000 ).
>> 
>> term_to_binary/1 RETURN VALUE:<<131,104,8,100,0,1,97,109,0,0,0,3,1,2,3,100,0,1,
>>                                 98,107,0,3,1,2,3,100,0,1,99,104,3,97,1,97,2,97,
>>                                 3,100,0,1,100,116,0,0,0,3,100,0,1,97,97,1,100,
>>                                 0,1,98,97,2,100,0,1,99,97,3>>
>> REQUEST COUNT:10000000
>> ELAPSED TIME (usec):97307
>> TIME PER REQUEST (usec): 0.0097307
>> PROJECTED RATE (req/sec): 102767529.57135664
>> 
>> binary_to_term/1 RETURN VALUE:{a,<<1,2,3>>,
>>                                  b,
>>                                  [1,2,3],
>>                                  c,
>>                                  {1,2,3},
>>                                  d, 
>>                                  #{a => 1,b => 2,c => 3}}
>> REQUEST COUNT:10000000
>> ELAPSED TIME (usec):8747426
>> TIME PER REQUEST (usec): 0.8747426
>> PROJECTED RATE (req/sec): 1143193.4377038456
>> ok
>> 
>> 
>> 
>> I’ve performed testing on R21.1.
>> Any thoughts?
>> 
>> V/
>