Re: persistent_term to replace ETS for caching

Sounds like mutation of persistent terms has implications(across the
runtime, affecting everybody, even those who are not involved). For as long
as it is mostly read-only, it should be fine as the docs also notes

*Term lookup (using get/1
<https://erlang.org/doc/man/persistent_term.html#get-1>), is done in
constant time and without taking any locks, and the term is not copied to
the heap (as is the case with terms stored in ETS tables).*

It's the writes that should be cautiously and preferably controlled for the
most optimal usage.

Regards
Nalin Ranjan

On Sun, Dec 6, 2020 at 9:07 PM Zsolt Laky <[email protected]> wrote:

> Hi,
>
> You migh take a look at http://blog.erlang.org/persistent_term/
>
> Excellent article by Lukas, answered a lot of questions.
>
> Kind regards,
> Zsolt
>
> Sent from my iPad
>
> On 2020. Dec 6., at 15:30, Richard O'Keefe <[email protected]> wrote:
>
> 
> Thanks for that advice about persistent_term.
> From the documentation,
> <quote>
> When a persistent term is updated or deleted, a global garbage collection
> pass is run to scan all processes for the deleted term, and to copy it into
> each process that still uses it.
> </quote>
> Do I understand correctly that if three processes refer
> to a persistent-term, and one of them deletes it, it
> will be copied into both of the other processes, thus
> INCREASING the amount of memory used?
> This is sufficiently counter-intuitive that I am not
> sure I would dare to use this feature.
>
> Do I further understand correctly that *adding* a new
> persistent-term does NOT force garbage collection?
>
>
> On Mon, 7 Dec 2020 at 00:49, Nalin Ranjan <[email protected]> wrote:
>
>> It has the potential to trigger Global GC, and can affect responsiveness
>> as per the docs.
>>
>> https://erlang.org/doc/man/persistent_term.html
>>
>> Regards
>> Nalin Ranjan
>>
>> On Sun, Dec 6, 2020 at 5:16 AM Frank Muller <[email protected]>
>> wrote:
>>
>>> Hi guys,
>>>
>>> At work, we cache about 5.3 million entries in ETS. The system works
>>> perfectly, no issue so far (many years).
>>>
>>> During a brainstorming session, a colleague suggested to switch to
>>> persistent_term instead to avoid ETS term copying.
>>>
>>> Pretty simple: we check if the Key exists in persistent_term. If yes, we
>>> are done. If not, we get it from ETS, move it to persistent_term and send
>>> it back to the caller.
>>>
>>> Question: is there any limitation(s) on persistent_term usage? Stated
>>> otherwise, can we create 5.3 million persistent_term <K,V>?
>>>
>>> Any suggestion/idea/thought is very welcome.
>>>
>>> /Frank
>>>
>>