Re: Director 9.6.7 died on a duplicate Path entry

On 12/18/20 10:50 AM, Eric Bollengier via Bacula-devel wrote:
> Hello Phil,
> 
> I would recommend to test again, the batch mode is doing the insert of
> the File records 32 by 32 (with MySQL) in the batch table, and for the
> Path/File update, it's done with one query each. I hope that the
> limitation is not on the number of records written by a single query,
> (in something like INSERT INTO test1 SELECT * from test2), else whatever
> they advise, I would recommend to ask them to address this gotcha in
> their roadmap.

That is indeed exactly the case.  When I looked at this before, what I
recall finding was that the MySQL driver code creates a temporary table
into which it caches attribute records, but then at the completion of
the job it dumps the entire table into the main database all at once.  I
recall finding a setting for a batch size, I don't recall precisely what
it was, but I do remember that it ought to work fine if it was honored;
but as far as I can tell it is ignored by the MySQL driver.


> The non-batch mode is not intensively tested, however, I believe that we
> lock the SQL connection to insert Path, so we should not have this
> error. In this specific case, an INSERT IGNORE (not very standard I
> believe) can probably do the work (if it returns the correct PathId).
> You may also look if the bacula DB is properly locked and if you have
> only one connection open or one per running job.


I was not able to catch it in the act, so I can't say.  I can run some
test jobs while monitoring active connections.


-- 
  Phil Stracchino
  Babylon Communications
  [email protected]
  [email protected]
  Landline: +1.603.293.8485
  Mobile:   +1.603.998.6958