Re: TLS 1.3 and SNI; test script included

Daniel Kahn Gillmor <[email protected]> wrote:
> >
> > And the server says "wtf, never heard of that domain, I'm
> > *mail*.isp.example.net" because that's the only TLS cert and domain it's
> > been configured for.  Server drops the connection with an error.
> 
> That itself smells like a bad decision on the server side (a reasonable
> server that doesn't have a matching certificate should probably go ahead and
> use the same certificate it would have used without any SNI).

Using that argument means the problem shouldn't have occurred in the first
place - breaking connections when TLS 1.3 is in use and SNI isn't supplied is
"a bad decision on the server side" for Gmail -- yet here we are.

I *could* just say "report this bug to Gmail", but we all know they don't
care, 900-pound-gorilla etc.

> And the user in this story is in an unsafe configuration in the first place
> because of the lack of certificate checking.

I'm not sure why you keep adding certificate checking into this argument.
Going from not supplying SNI at all to supplying SNI by default could break
things regardless of whether certificate checking is in use or not; that's
what I'm worried about.

I am *not* keen on adding another abstruse configuration option to getmail
(use_sni) to complicate its SSL configuration options even more which most
users will not understand and will just cargo-cult their way through based on
random forum postings ("try setting use_sni=True", "no, try setting it to
False").  But neither am I keen on breaking working configs for getmail users
because getmail suddenly starts sending an SNI value that their mail server
doesn't understand or doesn't like.

So if I'm going to make getmail send SNI whenever the underlying Python and
OpenSSL versions support it, I need to know it's NOT breaking connections with
previously-working configs.

As such, here's a test script.  Anyone who is interested in this issue, please
run it with the IMAP server to connect to as a first argument (and you can
supply port as a second argument if you don't want the default 993).  It will
make two connections to the IMAP server (without logging in), one using SNI
and one without, and then report on stdout/stderr:

  (a) what server it connected to
  (b) whether either or both connections failed
  (c) what TLS cipher/protocol was negotiated
  (d) whether it got different SSL certificates from the server between the
      two connections.

Example for imap.gmail.com:

  $ ./test-imap.py imap.gmail.com 
  Connecting to imap.gmail.com:993
  No SNI - got cipher ('TLS_AES_256_GCM_SHA384', 'TLSv1.3', 256)
  SNI - got cipher ('TLS_AES_256_GCM_SHA384', 'TLSv1.3', 256)
  Different certs for no-SNI and SNI: d5129635a050f63dd607ffa9271eefaab597c0975809765dad253973fc554d25 vs 1c971dad98db354df4c1b93aefd0098dbc17b62f11a9e7bdc0d438076889836d

(The script will report if TLS 1.3 or SNI is not supported by your
Python/OpenSSL combination).

I'd like to see results for a selection of IMAP servers current getmail users
are currently using, not just Gmail, which I can test from here ;)  Please
post the output to the list.

If there's a reasonable number of reports and it doesn't show connections
breaking when SNI is in use, I'll consider making the change.

> > Cue very unhappy getmail user, who had a working config, upgraded, and now
> > can't retrieve his mail.
> 
> iiuc, all gmail users are currently in exactly this position if they upgrade
> to OpenSSL 1.1.1 and leave TLS 1.3 enabled.

The difference there is that I didn't do anything to the users; *Google* broke
their working setup.  It's completely different if something *I* do breaks
working getmail configurations.

Charles

-- 
-----------------------------------------------------------------------
Charles Cazabon
GPL'ed software available at:               http://pyropus.ca/software/
-----------------------------------------------------------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

#!/usr/bin/env python2.7

import sys
import imaplib
import socket
import ssl
import hashlib

def wrap_socket(sock, keyfile=None, certfile=None,
                server_side=False, cert_reqs=ssl.CERT_NONE,
                ssl_version=ssl.PROTOCOL_TLS, ca_certs=None,
                do_handshake_on_connect=True,
                suppress_ragged_eofs=True,
                ciphers=None, server_hostname=None):
    return ssl.SSLSocket(sock=sock, keyfile=keyfile, certfile=certfile,
                     server_side=server_side, cert_reqs=cert_reqs,
                     ssl_version=ssl_version, ca_certs=ca_certs,
                     do_handshake_on_connect=do_handshake_on_connect,
                     suppress_ragged_eofs=suppress_ragged_eofs,
                     ciphers=ciphers, server_hostname=server_hostname)

ssl.wrap_socket = wrap_socket


#######################################
class IMAP4_SSL_EXTENDED(imaplib.IMAP4_SSL):
    # Similar to above, but with extended support for SSL certificate checking,
    # fingerprints, etc.
    def __init__(self, host, port=imaplib.IMAP4_SSL_PORT,
                 ssl_version=None, use_sni=False):
        self.ssl_version = ssl_version
        self.use_sni = use_sni
        imaplib.IMAP4_SSL.__init__(self, host, port, keyfile=None, certfile=None)

    def open(self, host='', port=imaplib.IMAP4_SSL_PORT):
        self.host = host
        self.port = port
        self.sock = socket.create_connection((host, port))
        extra_args = {}
        if self.ssl_version:
            extra_args['ssl_version'] = self.ssl_version
        if self.use_sni:
            extra_args['server_hostname'] = self.host

        self.sslobj = ssl.wrap_socket(self.sock, self.keyfile, self.certfile, 
                                     **extra_args)
        self.file = self.sslobj.makefile('rb')


has_sni = getattr(ssl, 'HAS_SNI', None)
if has_sni is None:
    raise SystemExit('no SNI support')
if not has_sni:
    raise SystemExit('OpenSSL - no SNI support')
has_tls13 = getattr(ssl, 'HAS_TLSv1_3', None)
if has_tls13 is None:
    raise SystemExit('no TLS1.3 support')
if not has_tls13:
    raise SystemExit('OpenSSL - no TLS1.3 support')

server = sys.argv[1]
port = imaplib.IMAP4_SSL_PORT
if len(sys.argv) == 3:
    port = int(sys.argv[2])

print 'Connecting to %s:%d' % (server, port)

certhash_nosni = None
certhash_sni = None
try:
    conn = IMAP4_SSL_EXTENDED(
        server, port, ssl_version=ssl.PROTOCOL_TLS, use_sni=False
    )
    sslobj = conn.ssl()
    peercert = sslobj.getpeercert(True)
    certhash_nosni = hashlib.sha256(peercert).hexdigest()
    ssl_cipher = sslobj.cipher()
    print 'No SNI - got cipher %s' % str(ssl_cipher)
except Exception as o:
    sys.stderr.write('TLS 1.3, no SNI - connection failed (%s)\n' % o)
else:
    conn.logout()

try:
    conn = IMAP4_SSL_EXTENDED(
        server, port, ssl_version=ssl.PROTOCOL_TLS, use_sni=True
    )
    sslobj = conn.ssl()
    peercert = sslobj.getpeercert(True)
    certhash_sni = hashlib.sha256(peercert).hexdigest()
    ssl_cipher = sslobj.cipher()
    print 'SNI - got cipher %s' % str(ssl_cipher)
except Exception as o:
    sys.stderr.write('TLS 1.3, with SNI - connection failed (%s)\n' % o)
else:
    conn.logout()

if certhash_nosni and certhash_sni and certhash_nosni != certhash_sni:
    sys.stderr.write('Different certs for no-SNI and SNI: %s vs %s\n' 
                     % (certhash_nosni, certhash_sni))