ElementsProject / ElementsProject/lightning

meta: Managing generated and derived files

Open
#8,329 13 comments 1 reaction 0 assignees View on GitHub
compilation docs open to suggestion Severity::Low Status::Needs Info
Dominant language
C
Stars
3.1k
Forks
1k
Avg merge
4d 10h
Merged PRs (30d)
13

Description

The Core Lightning project relies on a whole range of generated and derived files:

- Wire message files, generated from the protocol specification, for both C and Python source code
- Documentation files, and `schema.json` aggregating the docs, and metadata files from which they are generated
- `msggen.json` and similar to track the evolution and historic values over time
- Copies of files that are needed for python packages. These are built in isolation by the buidlers, and so they can't rely on files that aren't in their respective subdirectory.

So far we have defaulted mostly to just committing these files. However, they cause a lot of conflicts, since they aggregate things from a variety of places, causing issues even for changes that are not otherwise in conflict. They have a bit of a submarine effect, i.e., a change in one PR, that causes a later PR to fail, because the derived file was not updated, and not checked, forcing the latter PR's author to fix things they did not break. And finally of course, new contributors often stumble over this convention, making onboarding of new contributors difficult.

I'd like to propose that we stop committing the derived files wherever possible.

The open question I have is whether it is acceptable to ask contributors to call a setup, say `make sync` to setup all generated files and derived files, before being able to for example bundle msggen or similar packages?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.