Re: UUIDs

Hi Michael

So Cayenne actually currently does support generating UUID PK's, if in the  
Cayenne Modeler you:

1. set your column type to BINARY
2. set it to a length of 16
3. check the PK flag, and
4. set the table's "PK Generation Strategy" to Cayenne

This will result in a UUID being generated via  
"org.apache.cayenne.util.IDUtil.pseudoUniqueSecureByteSequence(int)".

Unfortunately this UUID is currently a MD5 digest which is bad for  
indexing.

The reason for the digest is to anonymise the underlying UUID which is 16  
bytes long consisting of:
bytes 0..3 - incrementing #
bytes 4..11 - timestamp
bytes 12..15 - IP address

The above UUID generation could easily be changed to use Java's native  
UUID which is a Time-Based UUID that would be index friendly, if I'm not  
mistaken.

Regards
Jurgen


On Sat, 24 Aug 2024 04:42:39 +0200, Michael Gentry <[email protected]>  
wrote:

> Hi Andrus,
>
> Part of what I meant by adding UUID support to Cayenne was to include   
> UUID
> as a PK mechanism in Cayenne modeler and provide a corresponding PK
> generator class. Nothing currently stops you from manually setting a UUID
> yourself, but including support in the modeler would be a more natural  
> fit, I think.
>
> Thanks,
> mrg
>
>
> On Fri, Aug 23, 2024 at 4:33 PM Andrus Adamchik <[email protected]>  
> wrote:
>
>> I am actually glad we went on a tangent and started discussing UUIDs. I
>> just ran into a use-case of an idempotent PUT API endpoint that takes a  
>> mix
>> of new and existing objects, and there's no natural key in the entity to
>> check whether new (PK-less) objects are already in DB (so that we UPDATE
>> them instead of INSERT). UUID would come in handy in this situation :)
>>
>> (FWIW, the endpoint is running on Agrest with Cayenne underneath, and
>> Agrest is the layer that ensures idempotent semantics).
>>
>> Andrus
>>
>>
>> > On Aug 20, 2024, at 12:01 PM, Hugi Thordarson <[email protected]>  
>> wrote:
>> >
>> > Judging from some very, very basic experimentation, Cayenne seems to  
>> do
>> fine with UUID PKs.
>> >
>> > Db generated UUIDs really just work like serial integers with a
>> different generated value type:
>> >
>> >
>> https://github.com/hugithordarson/xx-c42/blob/main/src/main/java/family/MainUUIDDbGenerated.java
>> >
>> > …and the fun stuff, app generated UUID PKs (for all your cross- back-
>> and forth-referencing insertion needs) look fine as well:
>> >
>> >
>> https://github.com/hugithordarson/xx-c42/blob/main/src/main/java/family/MainUUIDAppGenerated.java
>> >
>> > …although I wouldn't vouch for that PK-generation method of exposing  
>> the
>> PK and populating it in a post-add hook.
>> >
>> > Unfortunately h2 doesn't appear to support deferred constraints, but I
>> tested this against postgres with the constraints present.
>> >
>> > Anyway, pardon this tangent, born from a joke. I won't really say this
>> really demonstrates much, but it was at least a fun experiment over  
>> lunch
>> and thought you might enjoy it:).
>> >
>> > Cheers,
>> > - hugi
>> >
>> >
>> >> On 16 Aug 2024, at 17:26, Michael Gentry <[email protected]> wrote:
>> >>
>> >> If UUID PKs are really going to be a thing, we should probably add  
>> them
>> to
>> >> Cayenne...
>> >>
>> >>
>> >> On Fri, Aug 16, 2024 at 9:44 AM Hugi Thordarson <[email protected]>
>> wrote:
>> >>
>> >>> Hi Michael!
>> >>>
>> >>> Sure, the UUID comment was meant as a bad joke, my world is all DB
>> >>> generated integer keys.
>> >>>
>> >>> That being said, I've wanted to try out UUID keys for a while. Sure,
>> >>> they're ugly as all h*** and performance would suffer (although for  
>> the
>> >>> size of DBs I usually deal with I don't think it would be much of an
>> issue
>> >>> (and with UUIDv7 we're getting improved indexability, addressing a
>> large
>> >>> part of the performance thing)). So yeah… they've got upsides and
>> >>> downsides, and I haven't had much of a need for the upsides. But  
>> I've
>> got a
>> >>> suspicion they might sneak into common use soon. Perhaps when
>> >>> openai.com/gptbot <http://openai.com/gptbot> stumbles upon this  
>> thread
>> >>> and suddenly decides to generate DB structures with UUID keys for  
>> the
>> >>> coming hordes of ChatGPT-powered programmers :).
>> >>>
>> >>> Cheers,
>> >>> - hugi
>> >>>
>> >>>
>> >>>> On 16 Aug 2024, at 14:20, Michael Gentry <[email protected]>  
>> wrote:
>> >>>>
>> >>>> Hi Hugi,
>> >>>>
>> >>>> From what I've read, UUID PKs have poor index performance and take  
>> up
>> >>> more
>> >>>> storage.
>> >>>>
>> >>>> Wouldn't it be better to use an integer sequence like PostgreSQL  
>> and
>> >>> Oracle
>> >>>> support? You can generate your PKs up front and Cayenne already  
>> knows
>> how
>> >>>> to deal with them.
>> >>>>
>> >>>> Thanks,
>> >>>> mrg
>> >>>>
>> >>>>
>> >>>> On Thu, Aug 15, 2024 at 6:49 AM Hugi Thordarson <[email protected]>
>> >>> wrote:
>> >>>>
>> >>>>> Hi Nikita,
>> >>>>>
>> >>>>> again, thanks for looking into this! And yeah, totally understand  
>> how
>> >>>>> we're not about to insert everything in one commit. Well, at least
>> until
>> >>>>> the universe decides it's time everyone move to app generated UUID
>> PKs
>> >>> and
>> >>>>> deferred constraint checks :).
>> >>>>>
>> >>>>> Cheers,
>> >>>>> - hugi
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>> On 14 Aug 2024, at 11:27, Nikita Timofeev <
>> [email protected]>
>> >>>>> wrote:
>> >>>>>>
>> >>>>>> In this case it seems like a true cycle, the Person entity has  
>> two
>> >>>>>> relationships to self. And that particular case Cayenne didn't
>> handle
>> >>>>> well
>> >>>>>> historically.
>> >>>>>> But looking at it, I want to try and tweak the new Graph-based
>> sorter,
>> >>>>>> because two updates generated shouldn't depend on each other. So
>> maybe
>> >>> it
>> >>>>>> could be fixed now.
>> >>>>>> It still won't be able to insert all the data in one go though.
>> >>>>>>
>> >>>>>> On Wed, Aug 14, 2024 at 11:33 AM Hugi Thordarson  
>> <[email protected]
>> >
>> >>>>> wrote:
>> >>>>>>
>> >>>>>>> Hi again Nikita!
>> >>>>>>>
>> >>>>>>> saw the fix you made yesterday and it works great for the test I
>> >>>>> created,
>> >>>>>>> so thanks for that!
>> >>>>>>>
>> >>>>>>> However, turns out that for the more complex case in our actual
>> >>> project,
>> >>>>>>> the operation still fails.
>> >>>>>>> I've added a new example to the test project that models that  
>> case
>> a
>> >>>>>>> little more closely:
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>
>> >>>
>> https://github.com/hugithordarson/xx-c42/blob/main/src/main/java/family/MainWithAddedBackReference.java
>> >>>>>>>
>> >>>>>>> Any thoughts?
>> >>>>>>>
>> >>>>>>> Cheers,
>> >>>>>>> - hugi
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>> On 12 Aug 2024, at 13:52, Nikita Timofeev <
>> [email protected]
>> >>>>
>> >>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>> Hi Hugi,
>> >>>>>>>>
>> >>>>>>>> Thanks for the perfect example, that's always my main problem.
>> >>>>>>>> I've found the issue with the new flush logic [1]. The last
>> operation
>> >>>>>>>> creates two logical changes (DbRowOps), and one of them is  
>> later
>> >>>>>>> discarded
>> >>>>>>>> as there's nothing to flush to the DB.
>> >>>>>>>> However it's discarded only after the sorting, so it fails.
>> >>>>>>>> I'm already testing a fix for that.
>> >>>>>>>>
>> >>>>>>>> Also wanted to mention that in this exact case
>> >>> GraphBasedDbRowOpSorter
>> >>>>>>>> helps, as it checks operation internals and ignores it.
>> >>>>>>>>
>> >>>>>>>> [1] https://issues.apache.org/jira/browse/CAY-2866
>> >>>>>>>>
>> >>>>>>>> On Fri, Aug 9, 2024 at 12:58 PM Hugi Thordarson <
>> [email protected]>
>> >>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> Hi Andrus,
>> >>>>>>>>> I've been taking a look at this with Maik, here's a runnable
>> example
>> >>>>>>>>> project containing a commit that works on v4.1 but fails in  
>> v4.2:
>> >>>>>>>>>
>> >>>>>>>>> https://github.com/hugithordarson/xx-c42/
>> >>>>>>>>>
>> >>>>>>>>> Quick link to the code actually demonstrating the failure:
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>
>> >>>>>
>> >>>
>> https://github.com/hugithordarson/xx-c42/blob/main/src/main/java/family/Main.java
>> >>>>>>>>>
>> >>>>>>>>> The last commit certainly results in a circular reference  
>> being
>> >>>>> present
>> >>>>>>> in
>> >>>>>>>>> the object graph, but it probably shouldn't be a problem for  
>> the
>> >>>>> actual
>> >>>>>>>>> operation since we're only updating a single row, right?
>> >>>>>>>>>
>> >>>>>>>>> Cheers,
>> >>>>>>>>> - hugi
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>> On 8 Aug 2024, at 18:10, Andrus Adamchik  
>> <[email protected]>
>> >>>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>> Hi Maik,
>> >>>>>>>>>>
>> >>>>>>>>>> Could you provide an example of a failing graph?
>> >>>>>>>>>>
>> >>>>>>>>>> Thanks,
>> >>>>>>>>>> Andrus
>> >>>>>>>>>>
>> >>>>>>>>>>> On Aug 7, 2024, at 7:31 AM, Maik Musall  
>> <[email protected]>
>> >>>>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>> Hi everyone,
>> >>>>>>>>>>>
>> >>>>>>>>>>> we upgraded an application from Cayenne 4.1.1 to 4.2.1, and  
>> now
>> >>>>> we’re
>> >>>>>>>>> getting more cyclic graph errors from AshwoodEntitySorter.  
>> Years
>> >>> back
>> >>>>> we
>> >>>>>>>>> already had a similar problem, but @SortWeight didn’t help and
>> >>>>>>>>> GraphBasedDbRowOpSorter wasn’t ready. The latter is now in 4.2
>> >>> stable
>> >>>>>>> but
>> >>>>>>>>> fails to save even simpler graphs, so unfortunately not a
>> solution.
>> >>> We
>> >>>>>>> had
>> >>>>>>>>> been able to get stable operation by fetching PK’s from
>> PostgreSQL
>> >>>>>>>>> sequences (Oracle-style) instead of having Cayenne generate  
>> them,
>> >>> and
>> >>>>>>> lived
>> >>>>>>>>> with the performance penalty associated with that, but the
>> problem
>> >>>>> came
>> >>>>>>>>> back with 4.2 despite that. Not reliably reproducible though,
>> >>> happens
>> >>>>>>> every
>> >>>>>>>>> now and then. Any thoughts?
>> >>>>>>>>>>>
>> >>>>>>>>>>> Thanks
>> >>>>>>>>>>> Maik
>> >>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>> --
>> >>>>>>>> Best regards,
>> >>>>>>>> Nikita Timofeev
>> >>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> Best regards,
>> >>>>>> Nikita Timofeev
>> >>>>>
>> >>>>>
>> >>>
>> >>>
>> >
>>
>>


-- 
Using Opera's mail client: http://www.opera.com/mail/