Simultaneous installations of several major versions of OpenSSL on the same machine

OpenSSL upstream declared that the interval between major releases will be decreased and will not be so long as it was between 3.0 and 4.0 releases. It means that the LTS versions of distributions will have to consider either having several versions of OpenSSL simultaneously or perform massive backports. 

This is an attempt to describe the problems that would happen on installing several versions of OpenSSL on the same Linux machine using standard distro packages mechanism. I’m not familiar with non-Linux ecosystems so I also want to encourage people who understand the situation for different platforms (Windows, MacOS) to share their opinions. Building OpenSSL from source is out of scope of this writeup.

Problem statement

Goal: we need to have 2 or more versions of OpenSSL on the same machine without breaking the system.
Rationale: OpenSSL is a low-level component so replacing an old version with the new one is unfeasible but IRL people may want to get some features (e.g. protocols, such as DTLS 1.3, or primitives, such as FN-DSA) available in newer versions of OpenSSL in some subset of applications. In case of LTS distro releases and shortening the upstream major versions life cycle it will happen.

Deadline: I thought it’s worth having it solved for 4.0 and anyway not later than we have an LTS in 4.0 series but now I'm inclined to take the 2nd thought on it.

Analysis

OpenSSL, being installed from packages, provides the following categories of files:

• Shared libraries (libcrypto, libssl)

• Providers built as a part of OpenSSL itself (native providers)

• Engines (irrelevant for 4.0+)

• openssl binary and corresponding man pages

• Configuration file openssl.cnf

• OpenSSL header files and developers’ man pages

Distros also ship 3rd-party providers that may have specific requirements (e.g. IRL they can be linked, implicitly or explicitly, to different versions of libcrypto etc.)

Below I describe the possible conflicts between 2 versions and proposed solutions.

• Shared libraries - no conflicts, they have different sonames. However, there is still an open question about symlinks (where should point libcrypto.so at - libcrypto.so.3.0.0 or libcrypto.so.4.0.0?)

• Native providers - both builds can potentially install providers with the same names (fips.so, legacy.so) in the same folder (MODULESDIR). It will create problems from the package management perspective as they should be dealt with in a special manner so having an option to separate module directories seems necessary despite the fact that the native providers can still be compatible with the different version.

• openssl binary and corresponding man pages - both will be named “openssl” by default. In our (Red Hat) practice there was never necessary to keep the old one though now we may need to keep both, the old ones for ENGINE support. So it could be a good idea to allow install the binary as “openssl{version_num}” and man pages also have a prefix like openssl{version_num}-smime etc - or not install man pages for the supplementary version at all. 

• Configuration file openssl.cnf also may cause problems in case of incompatible syntax or unsupported protocol values (e.g. 1.3 as the MaxVersion of DTLS will hardly be understood by 3.x series)

• OpenSSL header files and developers’ man pages can be (and usually are) separated to a dedicated subpackage that conflicts with the corresponding subpackage from the different version

The situation with 3rd party providers is more complicated but probably out of scope for OpenSSL as the project. If we have different directories by default for openssl3 and openssl4 providers, then that will solve the problem by itself; using a symlink to install the provider to both versions when it's compatible with both versions will be quite simple and serviceable.

While it’s possible to keep 2+ versions of OpenSSL in the same system simultaneously, it is hard enough.

Different scenarios of having 2 major versions of OpenSSL in system

We should distinguish 2 scenarios of having 2 OpenSSL versions in a system. The 1st is having a “compat” package to avoid breaking old/3rd-party apps altogether with a newer version of OpenSSL. The 2nd is introducing the newer version of OpenSSL in the middle of the distro life cycle. 

Compat package

This is a common practice. Compat package is shipped and allows old applications to run. No builds against compat packages are planned (so no devel packages). When 4.0 is released and 3.0 becomes a compat package, it is necessary to separate the providers directories to avoid file conflicts on file level, and also separate the config file. Note that no binary openssl utility is usually provided; just the library with its required modules and config file.

New version

In case of adding a new version in the middle of the life cycle of the distro it’s not feasible to update all the applications to refer to the new library versions. We will either have to package a new version of OpenSSL to a completely separate subtree (the simplest way) or separate all the installable components to avoid file conflicts. Note that depending on dependency chains building an application against the new openssl may fail or cause issues at runtime as some components may depend on the older openssl version development headers.