hiero-ledger / hiero-ledger/hiero-consensus-node
Code Generation VS Reflection
- Dominant language
- Java
- Stars
- 406
- Forks
- 226
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 210
Description
I had several discussions in the past about code generation and/or reflection in our projects. Based on that I created this issue today. The issue should be used to discuss benefits and drawbacks of both solutions and should help us to define best practices and definitions for our projects.
# Why do we need it
We have several places in our code base that is based on reflection or code generation:
- The Config API uses reflection internally to provide the Config records and make them easy to use. To do so a classpath scan is used next to reflection.
- Our serialization framework uses a classpath scan and annotation, too.
- The services use Dagger to create code at build time for DI support.
- We plan to create code for the config Api to have constants for all property names (similar to JPA/Hibernate code generation).
- PBJ uses code generation to create the classes based on protobuff.
All that cases use different approaches and 3rdParty libraries. I assume that future features will introduce more code that might use code generation or reflection (see my PR for an annotation processor: https://github.com/hashgraph/hedera-services/pull/7289). Based on that it makes totally sense to discuss what patterns and libs we want to use in future for such use cases.
# Problems and benefits of reflection
Reflection happens at runtime and therefore cpu time is needed to handle the reflection calls. That can have an impact on the performance of the application. Based on that we currently try to do reflection calls only at boot time of the application. That will end in a longer boot time for the application but will not impact the performance at runtime. In general reflection calls can be done without the need of a special 3rdParty library. Currently we only need a 3rdParty library for class path scanning that happens always in combination with reflection. The class path scanning has even a much bigger negative impact on the application performance than the normal reflection calls.
# Problems and benefits of code generation
Code generation happens at build time and therefore it has no impact on the performance at runtime (as long as the generated code is performant). The code generation can be triggered in multiple ways: ANTLR for example is triggered by a custom gradle plugin to create Parser/Lexer classes based on a grammatic. Next to that you can use the annotation processor to trigger code generation. For code generation 3rdParty libs must be used or a custom generator must be written. Since we want to have deterministic artifacts we need to prove that a code generator that is used create deterministic code and will create deterministic code in future (by future versions of the generator). Having the date of the creation in the comment of the created code would be enough to create non-deterministic code.
# Next steps
As you can see both patterns have positive and negativ outcomes and I do not know if we can find a solution that fits all. In general we need to discuss the different solutions and find a safe but pragmatic way to define what we want to use.
There are some low hanging fruits that we can achieve quite fast:
- Do not use the ANTLR gradle plugin to create code. Since the grammars that we use change not that often it will be ok to generate the code once by hand and commit it in the repository.
In addition to the low hanging fruits the following questions need to be answered:
- Define how code generation at build time should be triggered (for example only by annotation processor)
- Define if we want to continue to use class path scanning. Another solution would be annotation processor based code generation in combination with Java SPI.
- Define how we ensure that 3rdParty libs that we use for code generation (like Dagger) always create deterministic code. Maybe the library maintainers have an answer for that point.
- Define if we want to use a 3rdParty lib for code generation (for example https://github.com/square/javapoet) or create our own generator.
# Additional resources
The following issues and PRs are related to that topic:
- https://github.com/hashgraph/hedera-services/pull/7289
- https://github.com/google/dagger/issues/4005
- https://github.com/hashgraph/pbj/issues/82
- https://github.com/hashgraph/hedera-services/issues/5168
- https://github.com/hashgraph/hedera-services/pull/7920
```[tasklist]
### Tasks
- [ ] https://github.com/hashgraph/hedera-services/issues/8177
- [ ] https://github.com/hashgraph/hedera-services/issues/8178
- [ ] https://github.com/hashgraph/hedera-services/issues/8179
- [ ] https://github.com/hashgraph/hedera-services/issues/10001
- [ ] https://github.com/hashgraph/hedera-services/issues/6293
```
Contributor guide
Assessment
This issue has not been assessed yet.