Re: [PATCH v4 00/18] PCI/P2PDMA: Fix ACS egress control handling

> PCI P2PDMA treats any enabled ACS P2P Egress Control bit as an upstream
> redirect. PCIe r7.0, sec 6.12.3, table 6-11 says the Egress Control
> Vector bit for the target port decides instead: a clear bit routes a peer
> request directly, regardless of P2P Request Redirect. Firmware can
> therefore enable Egress Control with a permissive vector while Linux
> incorrectly rejects a valid direct P2P path.

I've never seen anyone use the egress control vector and broadly Linux
doesn't support it. The ACS command line shouldn't enable "P2P Egress
Control Enable" for this reason.

It is not a bad thing to accommodate the egress vector when improving
the ACS logic, but the main stream usage is the interaction of the
other bits along with ATS & RO in the TLP. See the comment I left a
long time ago:

https://elixir.bootlin.com/linux/v7.2/source/drivers/infiniband/hw/mlx5/mlx5_ib.h#L1649

So it would be nicer to read in the commit message how the mainstream
stuff is fixed up and just a little bit about egress control.

> [ ... 36 lines skipped ... ]
> A pre-existing gap comes first. The routing analysis covers only Requests
> carrying an Untranslated address; ACS Direct Translated P2P overrides
> those controls, so that scope is now written down rather than implied.

What I talked about with Thomas is we probably need the P2P subsystem
to know what kind of TLP the driver intends to put here when doing the
evaluation: strict order, relaxed order and translated all have
different possible routing options, and real system configure things
so each one takes a different path :\

Currently I think the P2P subsystem is assuming strict order
non-translated TLPs when it makes its calculations. Which is fine, but
as we go toward enhancing this each of the different paths should be
kept seperate.

I don't know how the driver facing API should work, but at least real
devices have options to use ATS or not, use RO or not, and can make
use of information from the P2P subsytem to make the right choice.

Further, when we get to things like an ACPI description of this stuff,
it would be nice to still discover these differences as well.

-- 
Jason