OpenSSL Communities
Mon 16 Jun 2025 8:42AM

Enabling hybrid key exchange in FIPS mode for old FIPS providers

DB Dmitry Belyavsky Public Seen by 16

OpenSSL 3.5 has enabled hybrid PQ key exchange with the new FIPS provider.

It's a great result but distributions (and I presume, big corporations) rely on previously certified versions of FIPS provider (e.g. 3.1 for upstream, 3.0.7 for RHEL).

According to a set of NIST documents (800-56Ar3) we can treat MLKEM as auxiliary data for EC key exchange.

Different orderings of the component data fields of FixedInfo may be used, and one or more of

the data fields may be combined (or omitted under certain circumstances). See Section 5 in SP

800-56C, and Sections 5, 7.4, 7.5 and 7.6 in SP 800-108 for details


We (Red Hat) has provided a proof-of-concept patch that works for our purpose and enables Hybrid kex in FIPS mode. It look a bit hackerish but resolves several obstacles:

  • pass propq down when generating MLKEM keys, so SHAKE would be fetched from default provider. Otherwise FIPS 3.0 provider doesn't have squeeze method before 3.2 (not certified upstream)

  • Expand EC pubkey when old (< 3.0.8) FIPS provider returns compressed one. We in RHEL have 3.0.7 so we came across this problem. I think it's acceptable upstream


I would like to see if this approach is of interest to the communities I represent (Distributions) or am involved in (Big corps)

DB

Dmitry Belyavsky Tue 17 Jun 2025 1:47PM

@Anton Arapov could you also mark this post to make visible for large corporations?

AA

Anton Arapov Wed 18 Jun 2025 12:06PM

@Dmitry Belyavsky done: you can see it here