How to respond to recent critiques? (Big picture)

What OpenSSL (or each Community guardian) might do better is surface sticky points, lead discussions to conclusion, e.g., the formulation of concrete GH issues/tasks that may then be prominently tracked (say via a very visible "Top challenges" dashboard) & worked on with priority: Something for you to consider instituting maybe @Jon Ericson ?

I think something along these lines would be very helpful. For instance, a big goal of 4.0 is removing ENGINEs. It's something that was talked a lot about internally, but could have been more public. Probably even more public than blog posts (which are great, but don't necessarily get a lot of traffic). I'm thinking something on the GitHub repo, Library roadmap and/or the Library documentation.

A good deal of it is "just" documentation, though; referencing the performance dashboard or the OpenSSL3 design rationale(s) for example. Is anyone already doing that? Has this maybe already been done? If not, anyone volunteering?

Getting ready for FOSDEM is eating up my time right now, but I do feel like this is more or less up my alley.

I'd also go with a variant of "communicate the reasons" which is to let people know that there's a way forward, so it's not "we decided to arbitrarily change everything and now you're stuck with it forever". I'm not sure if it's possible to win a benchmarking war if you're competing with old forks of, ah, make-it-go above everything else code, but if you can explain that the changes were necessary for the future because the existing 1.0 architecture was on its last legs and now that you've got it stable it's time to tune it, that might give people some light at the end of the tunnel.

Possibly also publish a plan for profiling and addition of fastpath code for commonly-used stuff if you're not already doing that, with occasional updates to demonstrate progress is being made. For example I assume almost everything is going with SHA2-256 (they usually are) and fastpath that, so all the highly flexible configurable options get bypassed for the common case of SHA-256 which is a direct jump into the raw SHA2 code, bypassing backend selection, object creation, locking, and a lot of other things.

Thank you for your write-up!

My deep beliefs are that in such situations of controversy the only normal way that can improve the situation is direct communication with the most problematic peers (AFAIK, there were some efforts to deal with HAProxy feedback - don't know if they are the result of communication or a common sense). No, communication does not ensure the positive outcome but usually provides some way to move forward.

OTOH, looks like some design decisions made during 3.0 development were not great enough and should be reconsidered. Some improvements were already done. For example, I'm not sure that the logic of encoder/decoder lookup is perfect, and there was a sketch of design to improve it.

One more point I'm seriously consider, but it's quite controversial. Both my background related so support of national cryptography and recent PQ transition where we heavily related on the option to provide new algorithms via providers demonstrate me the importance of plugability and flexibility. But there are gazillion of scenarios where people don't need it but are limited with the choice of algorithms, and probably having alg-specific API targeting these use cases makes sense.

But there are gazillion of scenarios where people don't need it but are limited with the choice of algorithms, and probably having alg-specific API targeting these use cases makes sense.

In November @matt, @amyp, @nhorman and I visited the computer science department at NC State University. One suggestion we heard was to offer an opinionated library. I had to look this up: "Opinionated software means that there is basically one way (the right way™) to do things and trying to do it differently will be difficult and frustrating."—tvanfosson on Stack Overflow

It's pretty clear the OpenSSL Library is non-opinionated. As a (recovering?) Perl programmer, I feel the pain of having too many ways to shoot yourself in the foot do things. I like the idea of having a subset of algorithms that are secure, reasonably fool-proof and avoid complexity that many people don't need.

@jon, re OpenSSL Library being opinionated, depends on what you look at. Let me muse philosophically for a bit...

The design around the provider mechanisms made libcrypto un-opinionated, leaving it to providers to be the opinionated parties. But, the OpenSSL Library includes a set of providers, so its opinionatedness hasn't gone away, it just has been shuffled around a bit, and left some spaces for alternative opinions (through external providers).

@Jon Ericson Is it safe for something in OpenSSL's position to do that? My code is pretty opinionated because I'm pretty opinionated and so you get that thrown in for free, but in OpenSSL's case it's the swiss-army-chainsaw for everything. If you start removing algorithms and mechanisms I suspect you'd get a lot of pushback from the long tail of vociferous people who rely on it supporting obscure algorithm X.

@pgut001 To be clear, I'm suggestion the option to have an opinionated subset. I expect we are going to see some of what you are talking about in 4.0 when stuff is removed. We do need to support people who started using the Library years ago and have legitimate uses for algorithms that shouldn't be used for new code.

The idea would be to have an onramp for people who want to start using OpenSSL but are overwhelmed with the extra stuff. Maybe this is more of a documentation task, but I wonder if it would make sense to develop an "opinionated" or "minimal" provider that would make the library simpler for new users. (This is, perhaps, getting off-topic.)

I thought about this a bit overnight and a big problem is the fact that there are a million web pages full of recipes telling you how to do whatever the user is asking for with OpenSSL, so stripping out things is going to break those instructions. What about picking out the top 100 (or whatever) things that people want, which will keep 98% of users happy because everything will work as before, and making the rest a compile-time thing, OpenSSL standard vs. OpenSSL extended? I don't use stochastic parrots myself so this is speculation but this could be one thing that they're actually good at, "take all the information you've scraped and list the top 100 OpenSSL recipes from that, order by popularity".

Some more thoughts. There are 2 possible outcomes beyond such publications - either an invitation to make some bargain or, as opposite, is the final decision and no reasonable compromise is possible - but my qualification is definitely not enough to distinguish what's beyond particularly this one.