Strange screen blanking behavior with HDMI switch

I have a 3 to 1 HDMI switch (3 inputs to one output) and sometimes
(about 10% of the time) when the screen blanker/screen saver kicks in
(power save mode) the switch thinks the input actually turned off and
goes to another one.   Once I give the original one some
keyboard/mouse input and switch back the resolution has been reset to
3840x2160 from my preferred 2560x1440 and thus I have to switch back
using the XFCE4 settings panel.

In the past I have played around with every possible xrander setting I
can think of from the command line, my .xinitrc/.xsession, system boot
time, my .cshrc/.login nothing seems to turn this behaviour off (or
even let me disable power saving mode or the screen saver).

I am using X11/nvidia-driver on a GeForce GT 710 (I have a 1030 in the
machine but it is used for CUDA [via linux vm running on bhyve] but
the behavior was the same before I demoted the 1030 to CUDA only.

On host machine:
GPU 0: NVIDIA GeForce GT 710 (UUID: GPU-91031c64-bd2e-82ba-ee0a-db10353810b3)

On VM (proof that it is a different GPU since I use this for
TensorFlow which will not work on a 710)
GPU 0: NVIDIA GeForce GT 1030 (UUID: GPU-e29dac8e-ab72-a9d4-e7ef-9f3faa7ed55f)

To prove passthrough here is my /boot/loader.conf:
vmm_load="YES"
nmdm_load="YES"
pptdevs="37/0/0 37/0/1"
hw.vmm.amdvi.enable=1



-- 
Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org