Re: [PATCH 1/1] bcache: fix stale data race between read cache miss and bypass write

> 2026=E5=B9=B45=E6=9C=8827=E6=97=A5 21:41=EF=BC=8CAnkit Kapoor <ankitkap@g=
oogle.com> =E5=86=99=E9=81=93=EF=BC=9A
>=20
> Hi Coly,
>=20
> Thank you for the feedback, for confirming the issue, and for the guidanc=
e.
>=20
>> Hi Ankit,
>>=20
>> Yes, I confirm this is an issue that must be solved. Nice catch!
>>=20
>> On Thu, May 21, 2026 at 04:39:25PM +0800, Ankit Kapoor wrote:
>>> A race condition exists between a read cache miss and a bypass write
>>> due to either congestion or sequential bypass, that causes stale data
>>> to be cached when the read cache miss runs concurrently with a bypass
>>> write targeting the same sectors.
>>=20
>> This patch fixes the stale data issue in run time, but if power failure
>> happens inside the race window, after boot up again, the stale data
>> still exists in cache for following read hits.
>>=20
>> And your fix invalidate the key after on-disk bio completed, which makes
>> such stale data window by power failure longer.
>=20
> While I initially hoped that serializing the operations would suffice, I
> completely agree with your point regarding the power-failure risk
> which shall be addressed.
>=20
>> To solve all the stale data race both for run time and power failure
>> condition, could you please consider the following proposal.
>>=20
>> Maintain a data structure to hold all invalidate range from by-pass
>> write, record/insert the invalidation range before bch_data_insert(),
>> and after cached_dev_write_complete(), clear/remove the invalidation
>> range.
>>=20
>> For a cache-miss read, if there is any invalidation range refcount
>> exists, check all non-zero refcount ranges, if any range overlaps with
>> the cache-miss read range, do NOT update the missing bkey back to btree
>> and only read data from backing device.
>=20
> I am now working on a new implementation to track the in-flight=20
> sectors currently being written, exactly as you suggested here.
>=20
>> Here you need to design a efficient data structure both for performance
>> and memory consumption. I would sugguest to maintain chunk refcounts
>> which mapping multiple 32MB ranges on cache device (current max key size
>> if I remember correctly) range. You may look at how md raid maintains
>> the legacy bitmap refcount, hope that code can give you any hint.
>=20
> Thanks, I will look into the md raid legacy bitmap reference implementati=
on for
> hints. In the meantime, could you please recommend any specific fio
> configurations or workloads you prefer for evaluating the memory
> overhead and performance impact of this change?

Maybe you can use a large and fast SSD as backing device, and do full rando=
m I/O with write around mode.
Then try to setup the race windows, that the in-memory refcount may occupy =
a more memory.

I don=E2=80=99t suggest to use a tree-like structures. Just use a refcount =
to cover 32MB range on backing device, it can be faster.
If a cache-miss read overlay a refcount covered range, change it to read-wi=
thout-refill-cache.  To avoid the refcounts
Occupy too much memory, if a page=E2=80=99s refcounts are all zero, you may=
 think of releasing this page. This is what I mentioned
how md bitmap manages the pages of bits. Maybe the idea may help a little b=
it.


>=20
> I will send a v2 patch series as soon as the tracking mechanism is ready
> and thoroughly tested.

Thank you, for catch this issue and work on the fix.

Coly Li