Re: Fix potential data loss and corruption due to Incorrect BIO Chain Handling

On Fri, Nov 21, 2025 at 04:17:39PM +0800, zhangshida wrote:
> We have captured four instances of this corruption in our production
> environment.
> In each case, we observed a distinct pattern:
>     The corruption starts at an offset that aligns with the beginning of
>     an XFS extent.
>     The corruption ends at an offset that is aligned to the system's
>     `PAGE_SIZE` (64KB in our case).
> 
> Corruption Instances:
> 1.  Start:`0x73be000`, **End:** `0x73c0000` (Length: 8KB)
> 2.  Start:`0x10791a000`, **End:** `0x107920000` (Length: 24KB)
> 3.  Start:`0x14535a000`, **End:** `0x145b70000` (Length: 8280KB)
> 4.  Start:`0x370d000`, **End:** `0x3710000` (Length: 12KB)

Do you have a somwhat isolate reproducer for this?

> After analysis, we believe the root cause is in the handling of chained
> bios, specifically related to out-of-order io completion.
> 
> Consider a bio chain where `bi_remaining` is decremented as each bio in 
> the chain completes.
> For example,
> if a chain consists of three bios (bio1 -> bio2 -> bio3) with
> bi_remaining count:
> 1->2->2
> if the bio completes in the reverse order, there will be a problem. 
> if bio 3 completes first, it will become:
> 1->2->1
> then bio 2 completes:
> 1->1->0
> 
> Because `bi_remaining` has reached zero, the final `end_io` callback
> for the entire chain is triggered, even though not all bios in the
> chain have actually finished processing. This premature completion can
> lead to stale data being exposed, as seen in our case.

It sounds like there is a problem because bi_remaining is only
incremented after already submittin a bio.  Which code path do you
see this with?  iomap doesn't chain bios, so is this the buffer cache
or log code?  Or is there a remapping driver involved?