Re: ext4 and extremely slow filesystem traversal

On 14/03/2013 02:05, Theodore Ts'o wrote:
> Just as a note, e2fsck -v can sometimes get this information much more
> quickly than other alternatives, since it can scan the file system in
> inode order, instead of the essentially random order.
> 
> Just as a side, if you just want to get a rough count of the number of
> directories, you can get that by grabbing the information out of
> dumpe2fs.

  Very useful. Global stats without having to scan the whole filesystems
are very precious...

  I was wondering : couldn't we use dumpe2fs or something based on
libext2fs to quickly extract a snapshot of all inodes from a given
filesystem ? For incremental backups, simply checking the mtime on
millions of inodes and discovering that only a handful of them were
updated since the previous pass looks very inefficient with
readdir()+lstat(). So mnay syscalls, so man spoonfed bits of
information. When I had a peek, I tought I'd got a list of inodes but
would not be able to link them back to their name(s) without inducing
the same cost as a regular find-like filesystem traversal. Does it make
sense ?

  AFAIK I would be better served with block-level snapshot solutions,
but LVM snapshots are supposed to double your writes if I got it right,
and I'm not sure there's something else in the Linux and free software
world. Plus I'd love to not migrate away from my ext3/4's without a
compelling reason. Btrfs is not (yet) and option and ZFS doesn't fit
legally with Linux.