SRV vs AAAA records

I've been troubleshooting why Exodus won't connect remotely to my jabber 
server. I use SRV records on the domain to locate the actual name of the 
server:

_xmpp-client._tcp.domainname SRV 0 0 5222 servername.domainname.

When I attempt to connect using Exodus to domainname, it does the SRV 
lookup correctly, but instead of looking up an A record for the server it 
only looks up AAAA and A6 records. It gets a name non-existent error 
(as I don't have them set up) and then does an A lookup on the domainname. 
This is a different IP address and so it fails.

Why is it skipping the A lookup?

-- 
Dr. Stephen Borrill