[Broken test case] Cryptic error message, fallback handling issue

Hello list,

Just reporting about a strange case of behavior difference between
Heimdal and MIT Kerberos implementations, that I stumbled upon. Of
course, this is assuming a somewhat broken scenario in the first place
(apache virtual hosts => inconsistent DNS & rDNS within a realm), but I
figured this might interest other people, and myself was looking for
further input.

-------------------------------

Environment specs :
- OS : FreeBSD 8.2-RELEASE / amd64
- Tested Kerberos versions :
 - Heimdal 1.1.0 (default, as confirmed from the version string embedded
in /usr/lib/libkrb5.so)
 - MIT 1.9 (Using security/krb5)
- KDC : Windows Server 2003 (default enc_type : ARCFOUR-HMAC-MD5)

-------------------------------

Scenario :
Single sign-on for a web application on Apache 2.2.17 + mod_auth_kerb
5.4 (Each time re-built with each different version of Kerberos)

The KDC is an AD domain controller. User accounts and keytab generation
process have all been tested and confirmed through other means (other
applications running seamlessly, with other OSes (OpenBSD, RedHat))

-------------------------------

Application configuration :
- Kerberos environment (besides realm definition) :
 - default_realm = KERBEROS.REALM
 - dns_lookup_realm = false
 - dns_lookup_kdc = false
- Application hosted on server "server.domain", DNS and rDNS set properly
- Application hostname "applicaiton.domain" is a CNAME to server.domain
- Virtualhost dedicated to the application, specified in Apache
 -> KrbServiceName set as HTTP/[email protected] (to
ensure the queried URL matches the used SPN)
 -> KrbLocalUserMapping on
 -> KrbSaveCredentials on
- HTTP service keytab was generated using ARCFOUR-HMAC-MD5

Note : For kicks, I tested DES-CBC-MD5, but this fails miserably with
decryption integrity check errors, as if it didn't acknowledge it, and
tried decrypting data as if it was ARCFOUR-HMAC-MD5... I found this
weird, but I didn't bother because this was as good an excuse as any
other to NOT use DES.

-------------------------------

Use cases (when accessing "application.domain") :
- Mozilla Firefox (latest) : No problem
- Google Chrome (latest) : No problem
- IE 7, IE 8 : IE tries unified authentication, which involves (as far
as what a tcpdump will reveal) confirming DNS and rDNS, then using the
final lookup result to do a Kerberos query (i.e, asking on behalf of
"HTTP/server.domain" instead of "HTTP/application.domain")


In the IE cases :
- Using Heimdal, authentication will blow up fatally, yielding either an
error 500 or 401 and the following cryptic error message (caught from
apache logs) :
   gss_display_name() failed:  An invalid name was supplied (, unknown
mech-code 0 for mech unknown)

- Using MIT, single sign-on will fail (as consistency requirements for a
trusted domain member are not met), but successive password dialogs
indicative of IE fumbling around and downgrading its requirements each
time, proceed, until we reach insecure basic authentication, which
ultimately succeeds.

-------------------------------

Of course, ultimately, the problem does not occur at all when giving a
dedicated IP to the application, and having a proper DNS setup for the
application hostname, however the difference in behavior between
implementations was jarring, to say the least.

So, I wanted to have a little more input if anyone was knowledgeable
about this topic. :)

Many thanks for your time,
-- 
Stephane LAPIE, EPITA SRS, Promo 2005
"Even when they have digital readouts, I can't understand them."
--MegaTokyo

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk79Y7YACgkQ24Ql8u6TF2PpOQCg6nO01XeCByq9jSs8lD7AFvb7
pZAAnRXkbrCw3cEf7IzohzCkez8/Y4Pe
=3RiP
-----END PGP SIGNATURE-----