GDB 16.3: is it possible to cross-debug an x64 core from ARM?

If I build an x86_64 GDB binary that has support for debugging ARM core
files, it works fine.  The build is invoked (on x86_64) like:

  $ ../gdb-6.3/configure ... --enable-targets=aarch64-linux-gnu
    ...
  $ make -j$(nproc)

There's no problem and the resulting GDB works both for native x86_64
cores and ARM cores.

But, if I try to build an aarch64 GDB (on an ARM system) that has
support for debugging x86_64 core files, the build fails like this:

  $ ../gdb-6.3/configure ... --enable-targets=x86_64-linux-gnu
    ...
  $ make -j$(nprocs)
    ...
  make[2]: *** No rule to make target '../sim/aarch64/libsim.a', needed by 'gdb'.  Stop.

If I use the same configure arguments but omitting --enable-targets
then I get a correctly built GDB that will work on ARM.

Comparing sim/config.log from the ARM build without --enable-targets
(which works) versus the failing one, shows nothing too interesting; I
get the same messages etc.  The only consequential difference is that
in the working version (no extra enabled targets) I see:

  SIM_ENABLE_ARCH_aarch64_FALSE='#'
  SIM_ENABLE_ARCH_aarch64_TRUE=''

while in the failing version (with x86_64 enabled target) I see:

  SIM_ENABLE_ARCH_aarch64_FALSE=''
  SIM_ENABLE_ARCH_aarch64_TRUE='#'

Which could have been guessed from the error message I suppose :).


Anyone have any further advice?  I guess the next step would be to run
configure with sh -x and see if I can deduce where it gets confused
(assuming this is a supported configuration).