[RFC PATCH 0/1] RISC-V vector fault-only-first vl truncation under plugin memory callbacks

Hi,

This is an RFC for a RISC-V vector fault-only-first interaction with plugin
memory callbacks. I'd like feedback on whether this is considered a bug and
whether the fix is at the right layer before I invest in tests/etc.

The fault-only-first load (vle*ff, e.g. vle8ff.v) decides vl by probing each
element and truncating at the first fault. That probe goes through
probe_access_flags(), which -- when a plugin registers a vcpu_mem callback --
returns a non-zero "not plain RAM" flag for ordinary RAM: TLB_MMIO via
force_mmio in system mode (commit 6d03226b42), or TLB_FORCE_SLOW in user mode.

For a normal load these flags only select the fast/slow path and the data is
still read correctly. But vext_ldff() treats any flag other than TLB_WATCHPOINT
as "this element faults", so the first non-first element is reported faulting
and vl is truncated to 1. The loaded data stays correct, but a vectorized
strlen/scan degenerates from N bytes/iter to 1 byte/iter, inflating the
dynamic instruction count whenever a memory-observing plugin is attached.

We hit this in practice while generating SimPoint BBVs for SPECcpu2017
500.perlbench_r under a memory-observing plugin: the instruction stream
diverged from the plugin-disabled baseline at perl_parse's vectorized strlen.

Patch 1 routes vext_ldff()'s probes through probe_access_full_mmu()
(check_mem_cbs=false), leaving the plain-load path unchanged so plugin
observation of real loads/stores is preserved. A user-mode counterpart of
probe_access_full_mmu() is added (it previously only existed in system mode).

Reproduced on master (v11.1.0-rc3) with a minimal memory-callback plugin and
a vle8ff strlen guest: enabling the plugin inflates the vle8ff execution
count ~16x without the fix, and has no effect with it. Reproducer (plugin +
guest sources, ~100 lines total) available on request or in a follow-up.

Questions:

  1. Is the vl truncation under a memory-observing plugin a bug, or is
     reflecting force_mmio/TLB_FORCE_SLOW into the FOF probe intentional?
  2. Is probe_access_full_mmu() at the FOF call sites the preferred layer,
     or should probe_access_flags() itself not fold plugin-induced force-slow
     into its returned flags?
  3. I noticed target/i386/tcg/access.c already works around the same
     force-slow (the "No haddr means probe_access wants to force slow path"
     comment). Would a more general fix at the probe_access_flags() layer be
     preferred so each target doesn't paper over it independently?

Patch is compile-tested on riscv64 system and user modes; full functional
testing and a proper tests/tcg case will follow once the approach is agreed.

Thanks,
Liang Li

Liang Li (1):
  target/riscv: fix vector fault-only-first vl truncation under plugin
    memory callbacks

 accel/tcg/user-exec.c            | 35 ++++++++++++++++++++++----
 include/accel/tcg/probe.h        |  8 ++++--
 target/riscv/tcg/vector_helper.c | 42 ++++++++++++++++++++++++--------
 3 files changed, 68 insertions(+), 17 deletions(-)

--
2.51.1