The State of OpenSSL for pyca/cryptography

IMO none of this is new. Practically all problems outlined in the article date back to decisions taken years ago. Substantial changes to OpenSSL --both code and organizational structures-- have occurred since addressing many of these issues. It would be worth while writing an equally clear response to this pyca article -- if not already done: Then a link there should be added here to give a balanced perspective.

That said, is everything good now? Again, imo, no: The code is complex, riddled with "backwards compatibility" workarounds supporting the many uses OpenSSL caters for.

And that is a core difference between OpenSSL and the forks you mention above: Those forks cater precisely for the needs of specific uses cases, some times single companies -- they expressly deny "general utility". The most clear one is BoringSSL: ["Although BoringSSL is an open source project, it is not intended for general use, as OpenSSL is."](https://github.com/google/boringssl?tab=readme-ov-file#boringssl).

Does this mean OpenSSL should not continue to improve? Clearly no. But I'd think it would be better to follow and support the many already on-going improvement activities (beyond discussions, also with code!). So, what about listing them here and seeing what every one can do to move them forward? If new ones are discovered, even better. But then, I'd encourage creating GH tickets so the people doing the real work get to see them and can begin chipping away at them. This discussion may be used to rank those by priority to draw attention to the most salient ones.

In my opinion, OpenSSL as a project needs to improve the way recent advances in the development gets known to users and developers. One common theme across pyca's article and earlier ones is that OpenSSL 3.x performance is a degradation compared to OpenSSL 1.1. While that was so dramatically true in 3.0 release, situation improved a lot by 3.6. It would be great to have that covered in a clear and reproducible way. Perhaps, this is a job for OpenSSL corporation marketing? There are few projects that already built performance testing frameworks around crypto primitives and TLS, like PQC-LEO.

Python-cryptography maintainers follow a thread with many other engineers who complain about OpenSSL 3.x API design. Since these API changes have been criticised for quite some time, perhaps it would be good to address some of the implications (performance is one but there are few more) -- through better documentation, design improvements, etc.

x.509 parsers slowness was reported multiple times. While pyca/cryptography is using its own code to parse x.509 certificates nowadays, addressing the performance here is probably the most concrete action item that could be planned. I'm sure BAC and other communities would support this need.

That an interpreted language like Python can outperform OpenSSL parsing data is something to be ashamed of.

@Paul Dale python-cryptography doesn't use Python code to parse X.509, they have their own parser written in Rust.

That's even worse 😎

Java's parsing outperforms OpenSSLs. Not quite interpreted though.

@Paul Dale I do not understand what's worse. That X.509 parser written from scratch in Rust without any particular API usage constraints outperforms X.509 parser in OpenSSL? I do not see anything too bad with that. It is fairly expectable and yeah, we are working on improving performance of X.509 parsing but that requires some non-trivial API changes.

This was discussed yesterday at the F-TAC meeting and we kept it as an agenda item for the next meeting as well.

Concretely, the plan was for @Jon Ericson to write up a post about the pyca and the haproxy critiques here, to try and channel the criticism through the OpenSSL communities platform and discuss with a broader audience the impact of those pain points and possible remediations.

As BAC and TAC we do need to do a better job to ensure these concerns are voiced and accounted for in our respective communities to leverage all the feedback for leading OpenSSL towards better serving our communities.

@Nicola Tuveri Thanks for pointing that it-- Glad to know that it was already discussed in yesterday's meet, I was unwell so couldn't join the meeting.

@Nicola Tuveri and if it hasn't been raised yet, I will raise it a s topic for the next C-BAC meeting as (as they pointed out in the blogpost) it has implications (not only) for distributions.