Rdiff-backup v2.2.6 list increments slowed by uid or gid name lookups

Hi,

Like so many others I want to express my thanks for all the recent 
development (and previous development) that has gone into this great 
piece of software!

I have noticed a slowdown that could be from a change in our system or 
from code changes in rdiff-backup, but either way I am wondering if 
performance could be improved.

Our situation is that we have a volume of user network home directories, 
which has over 50,000 top level directories in it (and millions of files 
below that). Each directory is owned by a different user. I've been 
backing it up for years with rdiff-backup without any problems.

As I am migrating to new storage servers, the "list increments" command 
is painfully slow, taking hours to complete. Tracing the system calls I 
see that it is doing a newfstatat() call on every one of those 50k top 
level directories. I don't know if this was done in previous versions or 
if this is new. Our previous storage systems used NIS to look up 
usernames/uids (which was very fast), while the new one uses Active 
Directory/SSSD. The calls to SSSD are what is causing the slowness, 
taking around half a second per call. (I'll happily entertain thoughts 
that this is a problem and too slow, but for now I need to accept it as 
a given.)

My question for the rdiff-backup developers, is why do we need to do 
stat all these top level directories to get the list of increments at 
all? Shouldn't that information all be in the rdiff-backup-data folder? 
If we do need to do some form of stat on these top directories, can it 
be done in such a way as to work with numberic uid/gid info rather than 
initiating a call to name services? If we did not have to hit Active 
Directory for each of those folders the speed would be drastically 
improved in this use case.

I have only spent a little time looking into this issue so I might not 
be seeing everything correctly, but I'd love to hear thoughts about this.

Thanks,

-Ty



-- 
-==============================================================-
   Ty Boyack
   NREL IT Engineer

   Please put all IT help requests though the ticketing system at:
   https://services.warnercnr.colostate.edu/
   or email to:
   [email protected]
-==============================================================-