Routing traffic between identical IP addrs on different vlan's?

Hello,

I am hoping that someone might be able to give me some pointers on how
to use linux vlan tagging to solve a broadcast domain conflict. I'm
going around in circles on this one.

I have two servers with different ipv4 addresses on an 802.1Q-capable
ethernet switch.

Each server is running a VMware virtual machine running an apache web
server. However, both of the virtual machines have the same ip
address. Changing the ip addresses is not an option.

I want server A to be able to choose whether to make HTTP requests to
the virtual machine on server B or the virtual machine on server A.
Likewise, I want server B to be able to make HTTP requests to
whichever virtual machine it chooses. And, I want both server A and
server B to still be able to communicate with each other (ssh traffic,
rsync trafffic, etc...)

How can I use linux VLAN id tagging to solve this problem?

I know how to tag the traffic from each virtual machine with a unique
VLAN id. And I have both physical servers on the same VLAN id. The
switch is successfully routing the VLAN traffic. However, how do I get
the physical servers to selectively route traffic to two identical ip
addresses on different VLAN's?

Or, is using VLAN's the wrong approach?

Thanks in advance!