Re: namei caching of newly created files?

On Wed, Jan 19, 2005 at 10:53:49PM +0100, Havard Eidnes wrote:
> Hi,
> 
> I've been doing some simple testing using postmark using various
> hardware configurations.  The postmark test is always run with "set
> number 20000" and "set transactions 50000".  This will create 20000
> files in a single directory, and perform various transactions (read/
> append and create/delete) on those files before removing them all.
> 
> My latest test was with a large FFSv2 file system using soft
> dependencies, and on identical hardware running recentish Linux, ext3
> with (supposedly) btree directory data structure, we are being
> trounched -- where Linux gets 3000 transactions/s and 15-20 MB/s
> read/write, we get at most around 800 t/s and 2-3 MB/s read/write.

I brought FreeBSD's ufs_dirhash into my -current tree and have
seen some good improvements.

Without dirhash, on a FFSv2 filesystem with softdep and the same
postmark settings as you, my laptop achieved 297 transactions/sec,
705kb/s reads and 1.30mb/sec writes. With dirhash this turns into
3846 transactions/sec, 8mb/sec writes and 15.1mb/sec reads. Your
namei tweak to ufs_vsnops.c appeared to add a few hundred to the
transactions/sec, .4mb/sec to read and 1mb/sec to write.

However, with many more files (I quickly tried 100,000)
performance degrades considerably. I thought it might have to do
with the hash pool maxing out, but raising the ceiling didn't
appear to help. I'll slap in the appropriate sysctls and try
playing around with it more sometime this weekend.

If you're interested in the patches I'll send them along, but
they're still a bit rough.

Cheers,
Steve

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (NetBSD)

iQEVAwUBQfK7O8uCaQyb9VOkAQKaDwf/WfgxykNCdBPMbrGVV4m8nUv4p8eLDAXy
4+15ZcbIexSHbEHEBV5DWa1p7Krr0bAQ6hz0RoMGVH9yH0ellTqMhdJJQ0ZkGwXS
qZ8iZFYD6L3PGvsbAqIjVhPxT2hLT4YyrQLXoNgXHRyRnkmsLvrv3tuN63vBu47I
wTBaz4YclawbZvXEYcTiK7iE1uxd7Cth/hXjLKX2UugyWViAx86lGP5VAHqMr95Z
woiKp2b2nNhB93gZuTZneEyaXxoWWpET8+7cPIeSPdp+EmbvohMlj9QCF1MIvHVQ
CmI4X1sSDsssrQ/WgglRUaUxnDwkMSWAjdPm5qVQmuPpMbT/MBeiuQ==
=nf14
-----END PGP SIGNATURE-----