RE: [PATCH v2] fetch2/wget: reuse cached HTTPS  connections

Hi Fredrik,

> The checkstatus() path reuses cached connections for HTTP requests, but uses
> urllib's standard HTTPSHandler for HTTPS. Each HTTPS availability check
> therefore creates a new TCP connection and performs a new TLS handshake.
> 
> Add HTTPS equivalents of the existing cached connection and request handler.
> Preserve the SSL context selected by BB_CHECK_SSL_CERTS and extend
> FetchConnectionCache keys so HTTPS connections are kept separate from HTTP
> connections, different certificate policies and CA files, and different proxy
> tunnels.
> 
> Add a local TLS self-test which proves two HTTPS HEAD requests use one
> TCP/TLS connection. The test also verifies that a connection established with
> certificate checking disabled is not reused after checking is enabled.
> 
> In an ABBA benchmark against BitBake master, 4,283 remote sstate availability
> misses averaged 761.639 seconds without this change and
> 254.742 seconds with it. This reduced the check by 66.55%, a 2.990x speedup.
> The benchmark had no matching mirror objects, so it isolates availability checks
> rather than download and unpack time.

Funny enough, just last week I was also poking around in this area using viztracer to do some profiling. I was preparing to send a patch that just cached the SSL context creation, which on my system costs 100-200ms each time (since it has to load the certificate store from disk).

Thanks,
Chris