Re: OpenSSL 3.0 for 14.0-RELEASE: issues with 1.x/3.x symbol clashing, ports linking against base OpenSSL, ports that don't compile/link against OpenSSL 3, etc

> On May 3, 2023, at 16:10, Pierre Pronchery <[email protected]> wrote:
> 
>         Hi everyone,
> 
>> On 5/2/23 23:24, John Baldwin wrote:
>>> On 5/2/23 2:59 AM, Antoine Brodin wrote:
>>> On Tue, May 2, 2023 at 1:55 AM Enji Cooper <[email protected]> wrote:
>>>> 
>>>> Hello,
>>>> One of the must-haves for 14.0-RELEASE is the introduction of OpenSSL 3.0 into the base system. This is a must because, in short, OpenSSL 1.1 is no longer supported as of 09/26/2023 [1].
>>>> 
>>>> I am proposing OpenSSL be made private along with all dependent libraries, for the following reasons:
>>>> 1. More than a handful of core ports, e.g., security/py-cryptography [2] [3], still do not support OpenSSL 3.0.
>>>> i. If other dependent ports (like lang/python38, etc) move to OpenSSL 3, the distributed modules would break on load due to clashing symbols if the right mix of modules were dlopen’ed in a specific order (importing ssl, then importing hazmat’s crypto would fail).
>>>> ii. Such ports should be deprecated/marked broken as I’ve recommended on the 3.0 exp-run PR [4].
>>>> 2. OpenSSL 1.1 and 3.0 have clashing symbols, which makes linking in both libraries at runtime impossible without resorting to a number of linker tricks hiding the namespaces using symbol prefixing of public symbols, etc.
>>>> 
>>>> The libraries which would need to be made private are as follows:
>>>> - kerberos
>>>> - libarchive
>>>> - libbsnmp
>>>> - libfetch [5]
>>>> - libgeli
>>>> - libldns
>>>> - libmp
>>>> - libradius
>>>> - libunbound
>>> 
>>> In my opinion this is a huge amount of work a few weeks before the
>>> release.  Focusing on updating OpenSSL and those core ports may be
>>> simpler.
>> This is my view.  I think making OpenSSL private is a very huge task, and
>> fraught with peril in ways that haven't been thought about yet (e.g. PAM)
>> and that we can't hold up OpenSSL 3 while we wait for this.  Instead, I think
>> we need to be moving forward with OpenSSL 3 in base as-is.  We will have to
>> fix ports to work with OpenSSL 3 regardless (though this does make that pain
>> in ports happen sooner).  Moving libraries private can happen orthogonally
>> with getting base to work with OpensSL 3.
> 
> I have started to look at updating OpenSSL to version 3.0.8 in base, using the existing vendor/openssl-3.0 branch.
> 
> My progress can be found at https://github.com/khorben/freebsd-src/tree/khorben/openssl-3.0. I regularly force-push to keep a consistent and nice commit history, before possibly applying for a merge.
> 
> So far the status is:
> 
> - libssl, libcrypto build on amd64, i386, less sure about aarch64, other architectures not tested
> - libfetch builds, uses libmd in addition to OpenSSL
> - libradius builds, same thing
> - libarchive builds
> - libunbound builds, but not unbound
> - libmp builds
> 
> I used libmd to reach a buildable status faster, since the equivalent MD5_*() API is now deprecated in OpenSSL 3. If MD5 is still allowed in OpenSSL 3, we can avoid the dependency on libmd again. (anyone got sample code for this?)
> 
> Meanwhile I keep trying to build the rest of the system, hopefully in time for a possible inclusion in -14.
> 
> Reviews and tests on the whole thing will be more than welcome in any case!

I’ll take a look at your fork/branch and pitch in some of the areas you mentioned above where you switched to libmd, etc.

One thing that I noticed which was potentially a sticking point was the aarch64 support. I’m not sure if you ran into this as well, but someone with aarch64/arm64 expertise will need to help validate the branch/changes on that platform family.

Thanks!
-Enji