Re: XDP_REDIRECT not working in XDP_DRV_MODE with Intel xgbe driver

> Is it possible the packet is looping? I.e., it goes out eth0, you
> redirect it to egress on eth1, it goes all the way up the stack and the
> kernel ends up forwarding it back out eth0?

I don't believe so, as I am redirecting from the egress on eth0 to the
ingress on the same eth0 interface.

> >> So this has nothing to do with the support in the driver; XDP is simply
> >> not suitable for what you're trying to do. Why are you trying to do this
> >> in the first place? I.e., what's the higher-level use case here?

> > The higher-level design goal (simplified to what's relevant) is that I
> > am trying to perform a TCP 3WHS between the kernel network stack and
> > my xdp ingress/tc egress hooks, where some trigger on xdp ingress
> > sends a SYN packet to the network stack, and when the network stack
> > returns a SYN-ACK packet the egress program redirects the packet to
> > the ingress interface, where the XDP program converts the packet to an
> > ACK packet, then completing the handshake with the network stack.

> It's trying to handshake with itself (on the same box)? Why? Is this a
> real use case, or a test for something else? If the latter, have you
> considered using a veth pair in going to a different namespace (that
> gets the "direction" right)?

This is a real use case as part of a larger design I am working on.
However, given that redirect from TC egress to XDP_DRV_MODE ingress
fundamentally doesn't work, I am rethinking other options to the
design. Alternatively, it seems that support for XDP_REDIRECT from
egress to ingress is currently in the works, although I don't believe
the patch has yet been applied to the mainstream kernel. So I will
also try building the kernel myself with the patch and hopefully get
the egress to ingress redirect working with 2 XDP programs attached in
driver mode. Thank you very much for all your help!


On Mon, Feb 14, 2022 at 5:28 PM Toke Høiland-Jørgensen <[email protected]> wrote:
>
> Sophia Yoo <[email protected]> writes:
>
> > On Mon, Feb 14, 2022 at 1:28 PM Toke Høiland-Jørgensen <[email protected]> wrote:
> >>
> >> Sophia Yoo <[email protected]> writes:
> >>
> >> > Hello,
> >> >
> >> > I am relatively new to BPF/XDP, and I’m currently trying to use a TC
> >> > egress program to redirect a packet back to ingress, where I have an
> >> > XDP program attached. When the XDP program is attached in generic mode
> >> > (XDP_SKB_MODE), the redirect occurs properly and the packet is seen on
> >> > the ingress of the interface, but when the program is attached in
> >> > native mode (XDP_DRV_MODE), the packet never redirects and is just
> >> > seen exiting the interface, even though the return code of the
> >> > redirect function is “success”.
> >>
> >> The difference between XDP generic and driver mode is that the generic
> >> mode is hooked into the core networking stack whereas driver mode runs
> >> in the driver as the very first thing after packets are physically
> >> received from the network.
> >>
> >> This also explains why what you're trying to do doesn't work: the TC
> >> hook runs in the core networking stack, and when it does a redirect, the
> >> packet does not actually pass through the network driver, it appears
> >> further up in the stack, where only the generic XDP hook will see it.
> >>
> > Thanks for your quick response! I see, I didn't realize that the
> > packet was being redirected before it passed through the network
> > driver. However, in this scenario, shouldn't the packets "disappear"
> > after being redirected from tc egress to xdp ingress? In other words,
> > even if the packet doesn't actually show up on the xdp ingress hook in
> > driver mode, it should at least not be seen exiting on the egress path
> > (e.g., on tcpdump)?
>
> Is it possible the packet is looping? I.e., it goes out eth0, you
> redirect it to egress on eth1, it goes all the way up the stack and the
> kernel ends up forwarding it back out eth0?
>
> >> So this has nothing to do with the support in the driver; XDP is simply
> >> not suitable for what you're trying to do. Why are you trying to do this
> >> in the first place? I.e., what's the higher-level use case here?
> >
> > The higher-level design goal (simplified to what's relevant) is that I
> > am trying to perform a TCP 3WHS between the kernel network stack and
> > my xdp ingress/tc egress hooks, where some trigger on xdp ingress
> > sends a SYN packet to the network stack, and when the network stack
> > returns a SYN-ACK packet the egress program redirects the packet to
> > the ingress interface, where the XDP program converts the packet to an
> > ACK packet, then completing the handshake with the network stack.
>
> It's trying to handshake with itself (on the same box)? Why? Is this a
> real use case, or a test for something else? If the latter, have you
> considered using a veth pair in going to a different namespace (that
> gets the "direction" right)?
>
> > Initially, I had wanted to use an XDP program instead of a TC program
> > on the egress path to do the redirect to ingress (which I believe
> > would bypass the current issue if both ingress and egress xdp programs
> > were attached in driver mode), but as of now it seems that redirect
> > from egress to ingress simply isn't supported in XDP, which is why I'm
> > using a TC program.
>
> Well, there's also an ingress hook for TC you could look at (or just
> stay with generic XDP).
>
> -Toke
>