Re: Performance of mnesia:select/2

On 2/26/21 3:53 PM, Dan Gudmundsson wrote:
> Interesting, and times do you get for 500 ets lookup on that data?

Oh, I really should have tested that as well:

timer:tc(fun () -> lists:map(fun(K) -> ets:lookup(t, K) end, Keys) end).

* using lookups: [set, {keypos, 2}]:               3148 us
* using lookup: [ordered_set, {keypos, 2}]:  3423 us

I have repeated the select/pattern runs to have comparable results:

* using pattern, [ordered_set, {keypos, 2}]: 3173 us
* using pattern, [set, {keypos, 2}]:               8091 us

Note that there was quite a high jitter on the measurements so I have
compute the mean value of 3 measurements each.

So lookups and pattern/ordered_set were too close to really come to a
winner here. I had the impression that the variance was slightly higher
with lookup, but digging into this would require a better measurement
setup than what I have been using.


>  
> On Fri, Feb 26, 2021 at 3:12 PM Jacob <[email protected]
> <mailto:[email protected]>> wrote:
>
>     Hi,
>
>     assuming that the match spec compiler does clever things with
>     patterns,
>     I'd use select with the following
>
>        MatchExpression = [ {{'_', K, '_'}, [], ['$_']} || K <- Keys ]
>
>     If did some quick measurements with plain ETS, timer:tc and 500
>     keys out
>     of 1000000 table entries and got:
>
>        * using pattern, [ordered_set]: 4532605 us
>        * using pattern, [set]              :  4645525 us
>        * using pattern, [ordered_set, {keypos, 2}]: 3826 us (!!!)
>        * using pattern, [set, {keypos, 2}]: 5714 us (!!!)
>
>        * using guards, [ordered_set]: 12542928 us
>        * using guards, [set]:               12310452 us
>        * using guards, [ordered_set, {keypos, 2}]: 12365477 us
>        * using guards, [set, {keypos, 2}]: 12277839 us
>
>     I have initialised the DB with [ ets:insert(t, {N, N,
>     integer_to_list(N)}) || N <- lists:seq(1, 1000000) ].
>
>     I don't know though, how this will translate to Mnesia, but I'd give
>     select with pattern on the primary key a try.
>
>     /Jacob
>
>
>     On 2/26/21 11:03 AM, Vance Shipley wrote:
>     > If I need to lookup a list of keys which is the better approach?
>     Why?
>     >
>     > Fselect = fun(Keys) ->
>     >         MatchHead = {'_', '$1', '$2'},
>     >         F = fun(Key) ->
>     >                 {'=:=', '$1', Key}
>     >         end,
>     >         MatchConditions = [list_to_tuple(['or' | lists:map(F,
>     Keys)]),
>     >         MatchBody = ['$_'],
>     >         MatchFunction = {MatchHead, MatchConditions, MatchBody},
>     >         MatchExpression = [MatchFunction],
>     >         mnesia:select(Table, MatchExpression)
>     > end,
>     > mnesia:transaction(Fselect, [Keys]).
>     >
>     > Fread = fun F([Key | T], Acc) ->
>     >                 [R] = mnesia:read(Table, Key),
>     >                 F(T, [R | Acc]);
>     >         F([], Acc) ->
>     >                 lists:reverse(Acc)
>     > end,
>     > mnesia:transaction(Fread, [Keys. []]).
>     >
>     >
>     > --
>     >      -Vance
>