Re: [PATCH RFC 09/15] mm/mglru: frequency guided workingset promotion (MGLRU-FG)
Kairui Song <[email protected]>
| Newsgroups | gmane.linux.kernel.cgroups,gmane.linux.kernel.mm,gmane.linux.kernel |
|---|---|
| Message-ID | <CAMgjq7C1nFb92FGE+uSapZZS+14R95OG9w8ji+ZBkF=fOQ2Eag@mail.gmail.com> |
On Tue, Aug 18, 2026 at 3:12 PM Baoquan He <[email protected]> wrote: > > On 08/04/26 at 03:47am, Kairui Song via B4 Relay wrote: > ...snip... > > +int folio_inc_lru_refs(struct folio *folio, bool is_fault, bool is_exec) > > +{ > > + int max_gen, min_gen; > > + int type, refs, gen, new_gen; > > + unsigned long new_flags, old_flags, max_seq; > > + struct lru_gen_folio *lrugen; > > + struct lruvec *lruvec; > > + > > + type = folio_is_file_lru(folio); > > + lruvec = folio_lruvec_live_get(folio); > > + lrugen = &lruvec->lrugen; > > + > > + old_flags = READ_ONCE(*folio_flags(folio, 0)); > > + do { > > + new_flags = old_flags; > > + gen = lru_gen_from_flags(old_flags); > > + refs = lru_refs_from_flags(old_flags) + 1; > > + new_gen = gen; > > + if (!(old_flags & BIT(PG_lru)) || gen < 0) > > + goto out; > > + > > + max_seq = READ_ONCE(lrugen->max_seq); > > + max_gen = lru_gen_from_seq(max_seq); > > + min_gen = lru_gen_from_seq(READ_ONCE(lrugen->min_seq[type])); > > + if (gen == max_gen) > > + goto out; > > + > > I am a little confused about the new mechanism. In the current mglru, it > does have the issue both mm walk and fd read set PG_referenced at the > 1st access, this is a obvious drawback. Now with the change, the ref > count is clearer, while the mm walk and fd read accessing is still mixed. > Imagine the cases below: > - one fd read; then mm walk; directly move to max_gen; > - one mm walk; then several times fd read; promote to next gen; > > Can I understand the final effect as: > 1) explicti ref count; > 2) more drastically promote mm walk based on the mixing ref counting; > - compared with the old behaviour: move to next gen when 2nd mm walk Hi Baoquan, Thanks a lot for the review! The old behavior is move to max_gen on walk access (unless it's the first access of a folio), and only protect non-mapped folios with PID refaults. Now, the non-mapped folios are also promoted after multiple accesses. The promotion is more proactive but not that drastic; it still requires 8 accesses (2^3, which overflows tier 4) to force promote one folio. For anonymous pages, this also provides a more consistent and explicit reference count mechanism, and you will see folios distributed fairly among tiers now, as repeated page table access will increase the refs above LRU_REFS_PROTECTED. Previously, all anon folios were stuck at tiers 0, 1, and mostly 3, skipping tier 2 completely. Also we reduced the bit usage by 1. This new mechanism is supposed to work as a whole, This new mechanism is supposed to work as a whole; it is documented and described in the chunk of comments in mmzone.h (maybe not the best place but old MGLRU's comments are there so I just updated it inplace). I tried to split it into smaller parts, which doesn't seem doable. And I think the three folio_inc_lru_refs* helper are also kind of self explaining on this. > I can only see one benefit and one significant change. Do I understand > it correctly, and is it worth? I think it's definitely worth it, if you look at the results in the cover letter :), especially the zipf access test (1.2, 1.1, 0.9, 0.8 have similar results, zipf is commonly used standard to simulate realworld access patterns) and a few other cases; 90% of the performance gain comes from this design. I tweaked the refs & promotion rule many times to double check and ensure it fits well for different workloads. And note that this is not an anon/file reclaim balance tradeoff issue, both anon refaults, file refaults, and pgpgins are reduced, meaning MGLRU is doing better at protecting both hotter anon and hotter files. It's widely complaines that MGLRU was actually performing poor on unmapped folio protection, worse than classical LRU in many workloads (Not the LRU overhead, MGLRU had lower overhead, but the actual ability to protect the hotter cache is not good, provable with the zipf test in the cover letter). After this change, its performance is obviously better than that of classical LRU. Attachment: a typical tier layout of a workload before this change: node 0 35 3910 37524 0x 0 238r 15596e 0p 0x 0x 0x 1 409r 6429e 6211p 0x 0x 0x 2 0r 0e 0p 0x 0x 0x 3 895r 39182e 8588p 0x 0x 0x 0x 0x 0x 0x 36 3244 84684 0 0 0x 0x 0x 2882r 6059e 0p 1 0x 0x 0x 448r 964e 0p 2 0x 0x 0x 298r 439e 0p 3 0x 0x 0x 416r 721e 0p 0x 0x 0x 0x 37 2858 221580 4269 0 0x 0x 0x 0x 0x 0x 1 0x 0x 0x 0x 0x 0x 2 0x 0x 0x 0x 0x 0x 3 0x 0x 0x 0x 0x 0x 0x 0x 0x 0x 38 2183 29167 242 0 12R 2499T 0x 2496R 4101T 0x 1 322R 29241T 0x 92R 156T 0x 2 0R 0T 0x 100R 144T 0x 3 150R 75764T 0x 741R 1161T 0x 20903930T 5439295Y 48822F 19595A And after: node 0 47 3227 197247 0x 0 967r 35231e 0p 0x 0x 0x 1 412r 6871e 1051p 0x 0x 0x 2 12r 639e 2319p 0x 0x 0x 3 0r 0e 49p 0x 0x 0x 0x 0x 0x 0x 48 1623 49290 0 0 0x 0x 0x 2065r 3586e 0p 1 0x 0x 0x 338r 836e 0p 2 0x 0x 0x 352r 817e 0p 3 0x 0x 0x 107r 139e 0p 0x 0x 0x 0x 49 1491 91927 4865 0 0x 0x 0x 0x 0x 0x 1 0x 0x 0x 0x 0x 0x 2 0x 0x 0x 0x 0x 0x 3 0x 0x 0x 0x 0x 0x 0x 0x 0x 0x 50 1305 10444 674 0 69R 6630T 0x 1082R 2586T 0x 1 21R 2509T 0x 55R 168T 0x 2 71R 2322T 0x 111R 288T 0x 3 0R 6T 0x 114R 619T 0x 31126090T 7780753Y 72142F 27409A For file heavy workloads, the file folios are also now fairly distributed amon gens instead of being stuck at tail gen. And it's not just looks prettier, the performance is indeed better.