OpenSSL Communities
Fri 24 Oct 2025 10:42AM

Handling of generated files

IU Igor Ustinov Public Seen by 63

Hi individuals,

The Foundation TAC is discussing the following issue concerning OpenSSL codebase:

There are some .c and .h files which are committed to the Git repository, but are actually built automatically during the make process. To avoid manually editing them (which is pointless and will finally cause the loss of changes), these files are committed now as read-only. The pull request https://github.com/openssl/openssl/pull/28838 provides a further improvement to the handling of the generated files. In a nutshell, the idea is to revert some of the currently generated headers to be completely static again, and have an `#include` directive to include a `.inc` file which is instead generated. This is expected to minimize accidental writes to generated files, by isolating the "generated" part, and making the "main" headers/C files standalone and writable again. This reuses the `.inc` mechanism that was already part of openssl practices.
This pull request is likely going to be accepted and merged to `master`.

The Foundation TAC has been asked if it should be recommended (as an exception) backporting this approach also to 3.6, after it is merged in `master`.

As your representative in F-TAC, I should express not my own opinion, but the position of the community. So I ask everybody who cares about this issue to express your view.

NH

Neil Horman Fri 24 Oct 2025 1:44PM

I'm somewhat in favor of going back to static files (the read-only generated c files have bitten me several times already).

That said, playing devils advocate here, There is another issue with generators that I've personally been bitten by, and that is locating symbol definitions. for example, generators like our new param encoder/decoder, define several symbols that get referenced but not defined in a way that is easily searchable in a way that any ide (I use cscope for reference) can easily locate them. The saving grace here is that, given that generator calls in our template files are co-located with code that references them, allowing me to use regex matching to at least learn where those references might come from.

I'm concerned that the above approach will further complicate that process by moving those generator functions to alternate locations, separating references from definitions.

What (if anything) to do about that? I'm not sure, but just thinking out loud, I might propose the following alternate solution:

  1. Add a makefile target to identify template files and their corresponding generated file, such that the latter is added to the .gitignore list, creating an extra check at git add time to prevent inadvertent adding of generated files to the index

  2. Modify our generator perl functions to emit a comment in the generated files by symbol definition indicating the template file the generated symbol was derived from.

Its still something of a manual process (adding a template file would require running the make target in (1) , similar to how we have to run make update to update our ordinals after adding a new public api), but perhaps we could mitigate that with a CI check. This also doesn't address the similar problems that occur from our macros that expand to various function definitions, but perhaps that can be deferred to a separate conversation.

At any rate, I think the above approach might give us the benefit of the code generation that these templates provide while mitigating the pain points that they cause.

RL

Richard Levitte Mon 27 Oct 2025 9:11PM

Considering 3.6.0 has been released, I'm not sure how feasible a backport would be. I fear it would cause confusion and not be very helpful in the end.