Re: [PATCH 0/1] bcache: reduce front IO latency during GC

Hi Robert,

Thanks for the fix up :-)

> 2025年4月15日 06:44，Robert Pang <[email protected]> 写道：
> 
> In performance benchmarks on disks with bcache using the Linux 6.6 kernel, we
> observe noticeable IO latency increase during btree garbage collection. The
> increase ranges from high tens to hundreds of milliseconds, depending on the
> size of the cache device. Further investigation reveals that it is the same
> issue reported in [1], where the large number of nodes processed in each
> incremental GC cycle causes the front IO latency.
> 
> Building upon the approach suggested in [1], this patch decomposes the
> incremental GC process into more but smaller cycles. In contrast to [1], this
> implementation adopts a simpler strategy by setting a lower limit of 10 nodes
> per cycle to reduce front IO delay and introducing a fixed 10ms sleep per cycle
> when front IO is in progress. Furthermore, when garbage collection statistics
> are available, the number of nodes processed per cycle is dynamically rescaled
> based on the average GC frequency to ensure GC completes well within the next
> subsequent scheduled interval.
> 
> Testing with a 750GB NVMe cache and 256KB bucket size using the following fio
> configuration demonstrates that our patch reduces front IO latency during GC
> without significantly increasing GC duration.
> 
> ioengine=libaio
> direct=1
> bs=4k
> size=900G
> iodepth=10
> readwrite=randwrite
> log_avg_msec=10
> 
> Before:
> 
> time-ms,latency-ns,,,
> 
> 12170, 285016, 1, 0, 0
> 12183, 296581, 1, 0, 0
> 12207, 6542725, 1, 0, 0
> 12242, 24483604, 1, 0, 0
> 12250, 1895628, 1, 0, 0
> 12260, 284854, 1, 0, 0
> 12270, 275513, 1, 0, 0
> 
> /sys/block/bcache0/bcache/cache/internal/btree_gc_average_duration_ms:2880
> /sys/block/bcache0/bcache/cache/internal/btree_gc_average_frequency_sec:133
> /sys/block/bcache0/bcache/cache/internal/btree_gc_last_sec:121
> /sys/block/bcache0/bcache/cache/internal/btree_gc_max_duration_ms:3456
> 
> After:
> 
> 12690, 378494, 1, 0, 0
> 12700, 413934, 1, 0, 0
> 12710, 661217, 1, 0, 0
> 12727, 354510, 1, 0, 0
> 12730, 1100768, 1, 0, 0
> 12742, 382484, 1, 0, 0
> 12750, 532679, 1, 0, 0
> 12760, 572758, 1, 0, 0
> 12773, 283416, 1, 0, 0
> 
> /sys/block/bcache0/bcache/cache/internal/btree_gc_average_duration_ms:3619
> /sys/block/bcache0/bcache/cache/internal/btree_gc_average_frequency_sec:58
> /sys/block/bcache0/bcache/cache/internal/btree_gc_last_sec:23
> /sys/block/bcache0/bcache/cache/internal/btree_gc_max_duration_ms:3866
> 
> [1] https://lore.kernel.org/all/[email protected]/


I see the data, it makes sense. I’d like to see more testing data, e.g.
1) Larger SSD (4T or 8T)
2) Higher I/O pressure (randomwrite)
3) Much longer time with high I/O pressure (24 hours+)

Then it can help me to understand the optimization better and make decision easier. Of course it will save a lot of testing time from my side.

Thank you in advance.

Coly Li