Re: Performance issue

On 1/24/25 14:50, Phil Stracchino wrote:
> On 1/24/25 14:43, Josh Fisher via Bacula-users wrote:
>> On 1/24/25 13:22, Bill Arlofski via Bacula-users wrote:
>>> On 1/24/25 10:17 AM, Phil Stracchino wrote:
>>>
>>>> this huge atomic write *also* makes it incompatible with Galera 3
>>>> clusters.)
>>
>> Are you sure about that? The only thing attribute spooling is doing is
>> delaying execution of the SQL statements generated by the job. Why would
>> Galera care when the SQL was executed?
>
> Josh,
> The problem is that except on small jobs, the current implementation 
> of attribute spooling in Bacula's MySQL driver results in a batch 
> write that exceeds Galera's hard limit of 128K rows in a single 
> transaction.

I see, although recent versions do not have that hard row limit. The 
global variable wsrep_max_ws_rows is used to set the write set row limit 
in newer versions and defaults to unlimited. This makes sense, because 
synchronization is "virtually" row-level anyway, regardless of 
transaction size. I assume that means that cluster replication is still 
occurring row by row, even during a transaction. From MariaDB 10.1.17 it 
should default to unlimited write set rows and a max 16 GB write set 
size (wsrep_max_ws_size).

>
> One *fix* for this would be to modify the current attribute spooling 
> code so that when despooling, it loops through its cache no more than, 
> say, 125K records at a time.  That would avoid the problem.
>
> I saw signs in the code when I last looked at it that *appeared* to be 
> an attempt to set such a maximum row batch limit, but that limit 
> doesn't appear to actually work.
>



_______________________________________________
Bacula-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-users