[pfx] OpenSSL 3.5.x TLS handshake overhead on AL2023 vs AL2 (OpenSSL 1.0.2) — any interest in AWS-LC as an al ternative TLS backend?
Rajesh Mishra via Postfix-users <[email protected]>
| Newsgroups | gmane.mail.postfix.user |
|---|---|
| Message-ID | <[email protected]> |
Hi all, We recently migrated a portion of our inbound smtp server from Amazon Linux 2 (Postfix 3.5.3, OpenSSL 1.0.2k) to Amazon Linux 2023 (Postfix 3.11.1, OpenSSL 3.5.7), same instance type, same postfix config, same traffic. We saw a large, reproducible increase in smtpd concurrency needed to handle the same connection volume, and traced it down to the STARTTLS handshake itself being meaningfully slower under OpenSSL 3.x. Sharing the numbers in case it's useful data for the list, and to ask whether AWS-LC support has come up before as an alternative TLS backend for Postfix — HAProxy already supports building against it and reports substantial improvements for the same class of workload (many short-lived connections doing one handshake each). Environment ----------- - Old: Amazon Linux 2, Postfix 3.5.3, OpenSSL 1.0.2k-fips (linked as libssl.so.10 / libcrypto.so.10) - New: Amazon Linux 2023, Postfix 3.11.1, OpenSSL 3.5.7 (linked as libssl.so.3 / libcrypto.so.3) - Identical instance type/vCPU/RAM on both, identical postfix TLS config What we measured ----------------- Isolated the STARTTLS stage specifically (raw-socket SMTP client, timing each protocol stage individually — connect, EHLO, STARTTLS, EHLO#2, MAIL, RCPT — against real postfix+postscreen, not a synthetic TLS-only benchmark) to rule out confounds from milters/DNS/content-filtering: Serial, one connection at a time (n=15-30 each): OpenSSL 1.0.2k: ~7ms mean per handshake OpenSSL 3.5.7: ~53ms mean per handshake (~7.8x) Under concurrent load (idle test box, 5 -> 300 simultaneous connections): 5 concurrent: 60ms mean 20 concurrent: 66ms mean 50 concurrent: 88ms mean 100 concurrent: 89ms mean 200 concurrent: 103ms mean 300 concurrent: 149ms mean, 459ms max The growth rate itself accelerates at higher concurrency (200->300, only 1.5x more connections, produced a 45% jump in mean) — looks like real lock/resource contention in the provider-fetch path, not simple queueing. Separately: enabling smtpd_tls_session_cache_database ----------------------------------------------------- Since the full-handshake-every-time issue above affects both generations, we're working on enabling session caching regardless of the AWS-LC question. On testing this we noticed tlsmgr appears to hard-route the cache database to data_directory no matter what path we configure — we even tried pointing smtpd_tls_session_cache_database explicitly at queue_directory (a real, legitimate postfix-owned directory, not an arbitrary path), and still got: tlsmgr: warning: request to update table btree:/var/spool/postfix/smtpd_scache in non-postfix directory /var/spool/postfix tlsmgr: warning: redirecting the request to postfix-owned data_directory /var/lib/postfix Given queue_directory itself got redirected, this looks like it's checking against data_directory specifically rather than general ownership/writability of the target path. Before we go create a custom directory with matching ownership/permissions and test whether that actually gets honored: is this intentionally hardcoded to data_directory only, or is there a supported way to point the session cache somewhere else (e.g. a separate volume/mount for this specific data)? Ask --- Has AWS-LC come up as a possible alternative TLS backend for Postfix, similar to how HAProxy supports building against it? We're aware this isn't a drop-in swap (different build/link target, not a config change, and would need real compatibility testing across the cipher/protocol matrix actual senders use, not just a perf benchmark) — mainly curious whether this has been discussed/attempted before, whether there's a known blocker specific to Postfix's TLS integration, or whether patches in this direction would be of interest. Thanks, Rajesh Mishra _______________________________________________ Postfix-users mailing list -- [email protected] To unsubscribe send an email to [email protected]