Enabling hybrid key exchange in FIPS mode for old FIPS providers
Anton Arapov
Public
Seen by 54
Cross-post from Distributions; Thread by @Dmitry Belyavsky:
OpenSSL 3.5 has enabled hybrid PQ key exchange with the new FIPS provider.
It's a great result but distributions (and I presume, big corporations) rely on previously certified versions of FIPS provider (e.g. 3.1 for upstream, 3.0.7 for RHEL).
According to a set of NIST documents (800-56Ar3) we can treat MLKEM as auxiliary data for EC key exchange.
Different orderings of the component data fields of FixedInfo may be used, and one or more of
the data fields may be combined (or omitted under certain circumstances). See Section 5 in SP
800-56C, and Sections 5, 7.4, 7.5 and 7.6 in SP 800-108 for details
We (Red Hat) has provided a proof-of-concept patch that works for our purpose and enables Hybrid kex in FIPS mode. It look a bit hackerish but resolves several obstacles:
pass propq down when generating MLKEM keys, so SHAKE would be fetched from default provider. Otherwise FIPS 3.0 provider doesn't have squeeze method before 3.2 (not certified upstream)
Expand EC pubkey when old (< 3.0.8) FIPS provider returns compressed one. We in RHEL have 3.0.7 so we came across this problem. I think it's acceptable upstream
I would like to see if this approach is of interest to the communities I represent (Distributions) or am involved in (Big corps)
Chris Brych Fri 20 Jun 2025 1:09PM
Oracle supports Red Hat's idea that enables hybrid KEX in FIPS mode.
Jeff Johnson 路 Wed 18 Jun 2025 1:22PM
@Dmitry Belyavsky we have enabled hybrid kex in FIPS mode internally. This is a way to keep FIPS capable products until FIPS PQ algs in a module are approved. However, it should be noted that NIAP and Common Criteria do not like nor accept hybrid kex. Typically, those requirements were subsets of FIPS so FIPS "mode" was required prior to Common Criteria restrictions being enabled (a CC mode). Their preference for pure PQ algs has changed the way to approach that. Basically, you cannot use hybrid kex to meet NIAP/CC requirements (and hence not FIPS hybrid) and there are no modules yet that have pure PQ algs. So...... what do they do. Those orgs have conceded that in the meantime they will accept ACVP alg certs listed on NIST/CMVP websites as an interim solution. Lots of fun during this transtion! 馃檪