Improve mapping from Pkl values to org.pkl.core.Value in Java
- Dominant language
- Java
- Stars
- 11.5k
- Forks
- 402
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 20
Description
Currently, Pkl has somewhat unconventional structure of the mapping between Pkl values and Java types. Specifically, some of the Pkl-specific types like `Mapping` or `Listing` are mapped to built-in Java types, while other Pkl-specific types, like `Duration`, do get their custom counterpart, like `org.pkl.core.Duration`.
There are two problems here. First, this mapping feels somewhat inconsistent, because there is a mix of Pkl-specific and native non-primitive types. This is subjective, of course, so some may not think it is a problem; the second, bigger problem here is that this mapping is not injective: some Pkl types are mapped to the same Java types. This gives marginal decrease in complexity, but as a trade-off, evaluation of Pkl values loses type information: there is no way to determine whether the given `Map` value in Java was originally a Pkl `Mapping` or a Pkl `Map`.
This type information may be valuable in some use case, for example, for lossless binary conversion: currently there is `Config.fromPklBinary` method which allows decoding pkl-binary values to `Config` objects, but there is no way to go from `Config` back to pkl-binary because there is no way to know if the given `Map` was originally a Pkl `Map` or a `Mapping`.
If we follow the typical approach for defining structured values like JSON, it would make sense to actually provide a sealed interface hierarchy which describes every value, without exception:
```java
public sealed interface PklValue {
record String(java.lang.String value) implements PklValue {}
record Boolean(java.lang.Value value) implements PklValue {}
record Int(java.lang.Integer value) implements PklValue {}
record Regex(java.util.regex.Pattern value) implements PklValue {}
record Duration(org.pkl.core.Duration value) implements PklValue {}
...
record Mapping(java.util.Map properties) implements PklValue {}
record Map(java.util.Map properties) implements PklValue {}
record Listing(java.util.List values) implements PklValue {}
record List(java.util.List values) implements PklValue {}
...
record Module(URI moduleUri, String moduleName, Object object) implements PklValue {}
record Object(PClassInfo classInfo, Map properties) implements PklValue {}
}
```
Note that mapping/listing types require their contents to be `PklValues` as well, and Module/Object classes contain their structures directly inside them.
If we target lower Java version, then non-sealed hierarchy is also okay, I guess, but this reflects the general idea.
This approach gives nice type safety on the Java side, allows compiler-enforced exhaustiveness validations (e.g. in code which process PklValue you will be forced to handle every possible variant of a value), and makes Java representation more consistent, at the cost of having wrappers around values and therefore more complex structure of classes.
This might be a too radical change, so as the first step, which at least solves the ambiguity problem, we can just define Pkl-specific classes for Mapping and Listing:
```java
package org.pkl.core;
public class PMapping implements Map {
private final Map properties;
// delegate Map methods to the internal map
}
```
```java
package org.pkl.core;
public class PListing implements List {
private final List values;
// delegate List methods to the internal list
}
```
Then, make the evaluator return `PMapping` and `PListing` when evaluating Pkl structures to Java.
This will make evaluation injective and unambiguous, and enable patterns like `Config` -> binary serialization.
If you think this needs SPICE, I can try working on one, I think.
Contributor guide
Research direction
Start by tracing the evaluator's mapping from Pkl structures to Java values, then inspect Config.fromPklBinary and the org.pkl.core.Value API for the type information it currently preserves. Define the scope decision between PMapping/PListing and the broader sealed hierarchy, and consider the evaluator and affected public APIs as done only when Mapping and Listing remain distinguishable without breaking existing behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100