Re: [PATCH v8 0/8] mm: Hot page tracking and promotion infrastructure
Bharata B Rao <[email protected]>
| Newsgroups | org.kvack.linux-mm,org.kernel.vger.linux-kernel |
|---|---|
| Message-ID | <[email protected]> |
On 06-Aug-26 7:14 PM, SJ Park wrote: > On Thu, 6 Aug 2026 11:19:22 +0530 Bharata B Rao <[email protected]> wrote: > >> On 28-Jul-26 11:44 PM, Andrew Morton wrote: >>> On Tue, 28 Jul 2026 11:13:48 +0530 Bharata B Rao <[email protected]> wrote: >>> >>>> This patchset introduces pghot, a subsystem for hot page tracking and >>>> promotion. >>> >>> Can DAMON be used to do this sort of thing adequately? >> >> Hi SJ, >> >> I started comparing hot page detection and promotion aspects of DAMON >> and pghot through benchmark runs. Here is the first set of results from >> pointer chase workload. >> >> Since I wasn't very familiar with DAMON tunables and settings, I let the >> AI to chose some default followed by other combinations. The test harness >> is developed and run by AI. Request you to go through this and let me know >> you feedback about the combinations and configurations tried here. Based on >> that I can adapt the settings for the future runs. > > This sounds like you asked AI to do the comparison because you don't know > DAMON. And you still don't know DAMON, so you cannot really explain the > results. So asking me to review what your AI did and guide your AI. Sigh. > > Correct me if I'm wrong. If I'm not wrong, no, I'm not an AI slop reviewer. I > will not review that until this is clarified. I call it AI slop regardless of > the output when it is something that generated and shared without full > understanding from the human user. > > If you want me to review something, please send me what you really understand. > To compare/discuss DAMON, please learn it first. Spend time and ask question. > I will be more than happy to help that. I was seeking feedback about the combinations and configurations that were tried so that I can adapt the setting for future runs. I spent time looking at the rich set of configuration possibilities [1] that DAMON provides and experimented with something very basic to start with, followed by a few changes in each step. Now I can see more meaningful numbers from DAMON without the need to enable migrate_cold. Metric notes ------------ - lat : steady access latency, ns/access, mean of last 25% of samples (lower is better). Baselines: base 337 ns, pghot 348 ns (within 3%, so absolute latency is broadly comparable here). - node0: end-of-run per-process placement on node 0 (numastat -p). - CPU : cumulative cputime of the monitor/promoter thread over the 240 s run, and its peak instantaneous %cpu (ps). This is the concentrated cost only; see caveat 2 on pghot's distributed cost. - promoted: bytes actually migrated (DAMON scheme sz_applied; pghot pgpromote_success). DAMON configuration method (default vs manually set) ---------------------------------------------------- The aim was to start from DAMON's own sysfs defaults and set only what a promote scheme needs, then change one thing at a time. Left at the DAMON default in every run below: sample=5ms, aggr=100ms, update=60s, apply_interval=0 (=aggr), intervals_goal off, watermarks none, quota goals none, filters none. min_nr_regions=10 (except the page-size runs). Set manually in every run (the minimum for migrate_hot to act): operations=paddr; one target region = node 2 physical range; action=migrate_hot; target_nid=0; access pattern nr_accesses>=1 (sz>=1 page, any age), i.e. "promote anything accessed at least once in the aggregation window". Per-run additions (the single variable that changes each row): D0 default : nothing extra (pure default + migrate_hot). D1 page-size : min=max nr_regions = 67,108,864 (one region per 4 KiB page over the 256 GiB target). D2 +quota : quota bytes=128 MiB, reset=1000 ms (~128 MB/s), weights sz/acc/age = 0/1000/0 (hottest first). D3 fine+quota : max_nr_regions=100,000 (min default 10) + D2 quota. D4 pagesz+q : page-size regions (as D1) + a quota (1 GiB/1000 ms) + the same weights. Table 1. DAMON, base kernel (each row changes one variable) ------------------------------------------------------------ id config regions quota lat node0 over kdamond (settled/pin) (MB/s) (ns) (GB) CPU -- ----------------- ------------- ------ ---- ----- ----- --------- B0 baseline - (no DAMON) none 337 0.0 - - D0 default (coarse) 11 none 121 64.0 +300% 25s/10% D1 page-size 64M pin none 129 33.0 +106% 277s/100% D2 default + quota 14 128 105 23.3 +46% 9s/3.7% D3 fine + quota 14-38 128 88 23.6 +47% 14s/5.6% D4 page-size + quota 64M pin ~1000 296 1.0 -94% 270s/100% B0 is the no-promotion control on the base kernel (numa_balancing=0, no DAMON), and is the reference for the lat column. The "over" column is measured against the 16 GiB HOT SET, not against B0: over = (node0 - 16 GiB) / 16 GiB. B0 promotes nothing, so its node0 is 0.0 and its over is n/a. Regions column: - A plain number (D0/D2/D3) is the SETTLED count DAMON adapted to, measured via the damon_aggregated tracepoint. These rows leave min_nr_regions at the default 10; D3 only raises the max_nr_regions ceiling to 100,000 (a cap, not a target), so DAMON splits/merges freely and settles at 14-38. - "64M pin" (D1/D4) means min_nr_regions = max_nr_regions = 67,108,864 (one 4 KiB region per page of the 256 GiB target). Setting min=max PINS the count: DAMON cannot merge/split, so it is forced, not settled. The actual count was unmeasurable (NA) because the pegged kdamond never completed an aggregation pass to emit the tracepoint. Table 2. pghot, pghot kernel ----------------------------- id config thr lat node0 over promoted kmigrated (ns) (GB) (GiB) CPU -- ---------------- --- --- ----- ---- -------- --------- P0 baseline - 348 0.0 - - 0s P2 hint-fault (def) 2 124 16.0 0% 16.0 6s/0.7% P1 hint-fault 1 112 16.0 0% 15.9 13s/1.1% Notes on Table 2: - numa_balancing=2, and pghot tunables with the default values (pghot_enabled_sources=1 (hint-fault source), pghot_target_nid=0 and pghot_promote_rate_limit_MBps) - Both thresholds place exactly the 16 GiB hot set (0% over): P2 pgpromote_success 4,194,304 = 16.0 GiB exactly; P1 pgpromote_success 4,157,158 (15.9 GiB); - thr=1 converges faster (first interval already 136 ns; it promotes during the load phase) and is slightly quicker (112 vs 124 ns) at marginally higher kmigrated CPU. thr=2 starts at 395 ns and converges over the run. Caveats ------- 1. Two kernels: DAMON on base, pghot on pghot. The baselines are within 3% (337 vs 348 ns), so absolute latency is broadly comparable, but this is not a single-kernel A/B. 2. The CPU columns are the concentrated monitor/promoter thread only. pghot's hint-fault source also imposes distributed overhead on the application threads (NUMA scanning + hint-fault handling: numa_pte_updates 10-15M, numa_hint_faults 3-8M), which is NOT in the kmigrated figure. DAMON's cost is concentrated in kdamond and is captured. A fair total-cost comparison must add pghot's NUMA- balancing tax. 3. Latency does not track placement linearly for this workload and the mechanism was not isolated; placement/over-promotion is the primary accuracy metric. 4. DAMON and pghot increment different counters (scheme sz_applied vs pgpromote_success); cross-mechanism promotion counts are not directly comparable, so placement (numastat) is used. [1] DAMON config directory - tree view -------------------------------------- root@host:/sys/kernel/mm/damon/admin# tree . └── kdamonds ├── 0 │ ├── contexts │ │ ├── 0 │ │ │ ├── addr_unit │ │ │ ├── avail_operations │ │ │ ├── monitoring_attrs │ │ │ │ ├── intervals │ │ │ │ │ ├── aggr_us │ │ │ │ │ ├── intervals_goal │ │ │ │ │ │ ├── access_bp │ │ │ │ │ │ ├── aggrs │ │ │ │ │ │ ├── max_sample_us │ │ │ │ │ │ └── min_sample_us │ │ │ │ │ ├── sample_us │ │ │ │ │ └── update_us │ │ │ │ └── nr_regions │ │ │ │ ├── max │ │ │ │ └── min │ │ │ ├── operations │ │ │ ├── schemes │ │ │ │ ├── 0 │ │ │ │ │ ├── access_pattern │ │ │ │ │ │ ├── age │ │ │ │ │ │ │ ├── max │ │ │ │ │ │ │ └── min │ │ │ │ │ │ ├── nr_accesses │ │ │ │ │ │ │ ├── max │ │ │ │ │ │ │ └── min │ │ │ │ │ │ └── sz │ │ │ │ │ │ ├── max │ │ │ │ │ │ └── min │ │ │ │ │ ├── action │ │ │ │ │ ├── apply_interval_us │ │ │ │ │ ├── core_filters │ │ │ │ │ │ └── nr_filters │ │ │ │ │ ├── dests │ │ │ │ │ │ └── nr_dests │ │ │ │ │ ├── filters │ │ │ │ │ │ └── nr_filters │ │ │ │ │ ├── ops_filters │ │ │ │ │ │ └── nr_filters │ │ │ │ │ ├── quotas │ │ │ │ │ │ ├── bytes │ │ │ │ │ │ ├── effective_bytes │ │ │ │ │ │ ├── goals │ │ │ │ │ │ │ └── nr_goals │ │ │ │ │ │ ├── goal_tuner │ │ │ │ │ │ ├── ms │ │ │ │ │ │ ├── reset_interval_ms │ │ │ │ │ │ └── weights │ │ │ │ │ │ ├── age_permil │ │ │ │ │ │ ├── nr_accesses_permil │ │ │ │ │ │ └── sz_permil │ │ │ │ │ ├── stats │ │ │ │ │ │ ├── max_nr_snapshots │ │ │ │ │ │ ├── nr_applied │ │ │ │ │ │ ├── nr_snapshots │ │ │ │ │ │ ├── nr_tried │ │ │ │ │ │ ├── qt_exceeds │ │ │ │ │ │ ├── sz_applied │ │ │ │ │ │ ├── sz_ops_filter_passed │ │ │ │ │ │ └── sz_tried │ │ │ │ │ ├── target_nid │ │ │ │ │ ├── tried_regions │ │ │ │ │ │ └── total_bytes │ │ │ │ │ └── watermarks │ │ │ │ │ ├── high │ │ │ │ │ ├── interval_us │ │ │ │ │ ├── low │ │ │ │ │ ├── metric │ │ │ │ │ └── mid │ │ │ │ └── nr_schemes │ │ │ └── targets │ │ │ ├── 0 │ │ │ │ ├── obsolete_target │ │ │ │ ├── pid_target │ │ │ │ └── regions │ │ │ │ ├── 0 │ │ │ │ │ ├── end │ │ │ │ │ └── start │ │ │ │ └── nr_regions │ │ │ └── nr_targets │ │ └── nr_contexts │ ├── pid │ ├── refresh_ms │ └── state └── nr_kdamonds 29 directories, 60 files