[PATCH RFC/RFT v2 00/18] Fix easy bits of the negative dentry problem

Greetings all,
 following a v2 - which has grown a bit.  Thanks a log to Miklos for
 some great review - though I would still be happy for me (and maybe
 sashiko will manage to apply this time).
 I'd also be very happy to hear of testing in anyone is able to
 trigger the reported problems with excess negative dentries.

 There are no significant changes here, just a few bug fixes and
 clarification and splitting up of some patches.  The last three
 patches come from Miklos suggesting that positive_after() in
 autofs could be in common code - in fact it already was :-)

 Original comment, though without the per-patch highlights, is below.

Thanks,
NeilBrown

 I was reading about the negative dentry problem in
 https://lwn.net/Articles/1079407/ and thought I would have ago at the
 easy bits.  Given how involved some of that was I'm glad I didn't try
 the tricky bits!

 So I didn't try to reduce the number of negative dentries or handle
 them differently to positive dentries.  I just tried to address
 specific symptoms.

 This series addresses two reported symptoms.
  1/ refcounts on parents can potentially overflow if there are billions
     of (negative) children.  This was the easiest easy bit.  The first
     patch is all that is needed.

  2/ Walking the ->d_children list can take arbitrarily long and can
     trigger soft lockups.  I haven't addresses the "arbitrarily long"
     part, but the "soft lockups" only takes another 11 patches.
     With these in place I would feel more confident addressing
     the "arbitrarily long" part.

 The core idea is to use a common helper for all walking of
 ->d_children, and teach it to drop the lock and schedule when needed,
 inserting a cursor to keep its place.
 This simple idea requires:
   - nothing should be holding any other lock while walking ->d_children
   - dentries shouldn't be moved around needlessly, else things might
     get missed.
   - quite a bit of refactoring in libfs for readdir code.

 so I have probably missed some bits and would be very happy to be told
 what I have missed.
 I would also be very happy if someone who can trigger the soft lockups
 can try this code and confirm that the locks are gone, and nothing bad
 happens.
 I have done some basic testing including the "notify" parts of
 ltp, but that is all.

 I haven't addressed d_walk() in this series - I thought it was big
 enough for now.  Most of d_walk() already schedules as needed, but I
 think there is still room for improvement, but that can come later.


 [PATCH v2 01/18] VFS: don't count references through ->d_parent
 [PATCH v2 02/18] autofs: change positive_after() so it takes d_lock
 [PATCH v2 03/18] coda: don't take rcu_read_lock() in
 [PATCH v2 04/18] nfs: separate locked regions in
 [PATCH v2 05/18] Add and use d_for_each_positive_child family of
 [PATCH v2 06/18] fsnotify: don't hold a spin_lock across
 [PATCH v2 07/18] fsnotify: reduce i_lock hold time in
 [PATCH v2 08/18] libfs: simplify scan_positives()
 [PATCH v2 09/18] libfs: change scan_positives() to use
 [PATCH v2 10/18] libfs: allow scan_positives() to be called without a
 [PATCH v2 11/18] libfs: replace find_positive_dentry() with
 [PATCH v2 12/18] autofs: don't hold ->lookup_lock in
 [PATCH v2 13/18] VFS: don't move dentries in d_sib list when they
 [PATCH v2 14/18] Call cond_reshed() as needed in
 [PATCH v2 15/18] libfs: remove cond_resched() from scan_positives()
 [PATCH v2 16/18] libfs: rename and export scan_positives()
 [PATCH v2 17/18] autofs: replace positive_after() with
 [PATCH v2 18/18] autofs: change get_next_positive_dentry() to NOT