Re: ZODB as Timeseries

On 26 April 2018 at 11:22, Jim Fulton <[email protected]> wrote:

>
>
> On Tue, Apr 24, 2018 at 8:53 PM, Leonardo Rochael Almeida <
> [email protected]> wrote:
> [...]
>
> Perhaps, but if so, I doubt they guarantee ordering between clients. ZODB
> oids are monotonically increasing with holes, because they're preallocated,
> but they aren't necessarily added in order.
>
> I would love to hear about an algorithm for ordering TIDs without a lock.
>
>
>>  [...] I wonder if the first phase of ZODB commit could bypass the GTL if
>> we can guarantee all objects are new additions and we delegate the TID
>> creation to the database.
>>
>
> You keep talking about delegating TID creation to the database.  I'm not
> convinced that this does what you think it does, or, if it does, that it
> doesn't involve a lock.
>

I'm going by the described semantics of `nextval()`[1] in PostgreSQL:

[1] https://www.postgresql.org/docs/current/static/functions-sequence.html

Specifically: "This is done atomically: even if multiple sessions execute
`nextval` concurrently, each will safely receive a distinct sequence value".

Also: "To avoid blocking concurrent transactions that obtain numbers from
the same sequence, a nextval operation is never rolled back".

The implied semantics is that developers don't have to hold a lock
explicitly. It could be the case that one is held by the DB implicitly, but
considering the existence of atomic Fetch-And-Add[1] instructions in modern
CPUs since at least the 486, I'm pretty sure `nextval()` is consistent and
lock-free. It should be easy to check by calling `nextval()` in a
(database-side) loop in two simultaneous threads.

[2] https://en.wikipedia.org/wiki/Fetch-and-add

Of course, It's one thing having a lock-free, database generated and
database guaranteed unique TID per transaction. It's another thing entirely
to have this TID also represent a timestamp of the transaction directly.

If each incoming transaction is going to try to call `setval()` to a
timestamp, then we'd definitively need a lock. It might be better in this
case that the TID is just an arbitrarily increasing number and the
timestamp be stored as transaction metadata.

And if TID is just an arbitrarily increasing number, we could avoid the
whole trouble of creating a sequence and getting it's next value, and just
call `txid_current()`[3].

[3]
https://www.postgresql.org/docs/current/static/functions-info.html#FUNCTIONS-TXID-SNAPSHOT

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.