OpenSSL Communities
Tue 26 Aug 2025 11:53AM

Dealing with autogenerated files

DB Dmitry Belyavsky Public Seen by 18

Dear colleagues,
I don't like the situation with autogenerated files.

Paul's efforts on dealing with parameters are much appreciated, but the result comes with a cost.

If we modify the autogenerated file by accident, we lose its content on `make clean` invocation (see https://github.com/openssl/openssl/discussions/28269). If we navigate to the autogenerated file by tag and don't know that it was expanded from the template,

The parameters are now not greppable until we make a real configure and generate the files. That is counter-intuitive and problematic for people who try to find smth by the constant corresponding to the param name (now we at least should have these constants in the generated code).

AFAIK, curl developers keep both the template file and generated file in the repo. Following this example and detecting the mismatch would save us both from losing our work by accident and problems with grep. However it will require implementing some mechanism to detect mismatch between the generated file and the template.

WDYT?

TM

Tomas Mraz Tue 26 Aug 2025 12:04PM

We actually use similar approach for other generated files (i.e. ERR_ code related) and we already have a CI job that checks these. So I think it would be fairly trivial to switch to this approach with other C sources generated with perl. (I would not use this approach for perlasm though as that output is build configuration specific.)

TS

Todd Short Tue 26 Aug 2025 1:41PM

Other repos I work in also have checked-in generated files. It's useful for the reasons you pointed out, especially being able to diff them to know what really changed prior to merge,

SL

Shane Lontis Wed 27 Aug 2025 12:27AM

The downside is that conflicts are more frequent if the file is frequently changed.

SN

Sasha Nedvedicky Thu 28 Aug 2025 6:53AM

I like the idea making the generated file read-only. error on save should raise level of awareness sufficiently. also having a comment at the beginning of the file saying it's been generated from path/to/template would help.

DB

Dmitry Belyavsky Wed 27 Aug 2025 6:23AM

Yes - but the threat of losing the results of work should overweight it, I think

RL

Richard Levitte Wed 27 Aug 2025 8:22AM

We should care, though, not to check in generated files that depend on configuration values.

PD

Paul Dale Wed 27 Aug 2025 8:50AM

Honestly, I don't suffer from this problem. I never need to look at the generated code. The line number mismatch is annoying but searching for the errant line of code almost always gets to the right place.

I do think that if we modify the Perl wrapper to include #line directives many of the concerns would vanish.

All we need to do is to make it more difficult to edit the generated code than not. The current "generated code" warnings do not seem to be sufficient.

RL

Richard Levitte Wed 27 Aug 2025 9:12AM

@Paul Dale 

I do think that if we modify the Perl wrapper to include #line directives many of the concerns would vanish.

You mean in the relevant template files that are used as input.

(we have template files that generate other than C source files, such as numerous .pod files)

PD

Paul Dale Wed 27 Aug 2025 9:41PM 

I had a thought overnight, we could use the Makefile to warn about the editing issue:

file.c.timestamp: file.c
        echo generated file.c modified
        exit 1

file.c: …
        …
        touch file.c.timestamp
DB

Dmitry Belyavsky Thu 28 Aug 2025 6:42AM

If these timestamps are removed on make clean, I'm not sure it saves

Load More