Re: postgres load at restore

On 8/29/24 13:39, Marcin Haba wrote:
> On Thu, 29 Aug 2024 at 18:42, Phil Stracchino <[email protected] 
> <mailto:[email protected]>> wrote:
>     The bvfs cache DOES greatly improve restore performance, but the one
>     showstopper drawback I found (beyond the runscript bug) was that if you
>     pre-populate the bvfs cache, dbcheck will refuse to clear orphaned Path
>     entries unless you clear the bvfs cache first.
> 
>     Marcin, do you have any insights on this issue?
> 
> 
> Hello Phil,
> 
> I think that it's due to the nature of the Bvfs cache. In the Bvfs 
> 'pathhierarchy' table is stored the directory tree structure in form 
> pathid -> parent pathid (ppathid). If some paths are orphaned, it means 
> that theoretically they could to be removed from Bvfs cache as well. But 
> it is hard to remove any path from the 'pathhierarchy' without 
> rebuilding the cache because otherwise the cache will be incomplete.
> 
> For example, we have the following path hierarchy relation written in 
> the 'pathhierarchy' table (I use pathids):
> 
> 1 => 2 => 3 => 4 => 5
> 
> If the path with pathid 3 is orphaned and removed, then in the Bvfs 
> cache we have the path relation:
> 
> 1 => 2 => ??? => 4 => 5
> 
> In this meaning the dbcheck behavior and block removing orphaned paths 
> for me is understandable to avoid rebuilding everything.

Oh yeah, the consistency problem is fully understandable, it just seems 
an oversight that dbcheck seems unable to fully handle the presence of a 
populated bvfs cache.  It seems like a part of dbcheck should be 
checking and updating the bvfs cache if present.


> BTW: When the cache is manually cleared by '.bvfs_clear_cache yes' it 
> will be completely (re)built anyway.


Gotcha.

If I do a .bvfs_update WITHOUT specifying a jobid, will it do a full 
rebuild?  I'm thinking what I'd LIKE to do is the following:

NIGHTLY
	Run all scheduled client backup jobs
	Incremental rebuild bvfs cache
	Run catalog backup

WEEKLY
	Clear bvfs cache
	Run dbcheck
	Full rebuild bvfs cache


-- 
   Phil Stracchino
   Fenian House Publishing
   [email protected]
   [email protected]
   Landline: +1.603.293.8485
   Mobile:   +1.603.998.6958