NAT based on source IP?

I'm looking for a hand with m0n0 configuration, hopefully this is the right
place to ask - if not apologies in advance!

I have a single static IP address with a single network behind the firewall,
using NAT.

I need to route incoming requests on a fixed port to different servers
behind the firewall based on source IP. i.e.

I need to route incoming requests from a known source IP on port X to server
A on port Y, and requests from any other IP address on port X to server B on
port Y.

Whilst I can set up firewall rule to route by source IP (and port) to server
A and server B, I can't find a way to set up the NAT rules so that they are
applied on the basis of source IP.

Is what I am trying to do just "wrong" or otherwise impossible? Is there any
other way of approaching the problem? The obvious solution - changing the
clients so that the requests from a known source use a different port - is
not an option unfortunately.

Many thanks
Patrick