☂️ JNIgen transformations and possibly YAML config removal
- Dominant language
- Dart
- Stars
- 275
- Forks
- 144
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 47
Description
# Note
All of this could be used for `package:jni` itself, as it already contains a lot of hand-written code for `List`, `Map`, `ByteBuffer`, … I plan to add support for exceptions and it's really useful to write transformers for common exception types and have them either in `package:jni` or even another `package:java_library`.
# Use Cases
### Differences between Java and Dart styles
For example, Java naming convention recommends constants to be `SCREAMING_SNAKE_CASE`, while Dart uses `lowerCamelCase`.
_=> A way to easily change the case._
### Some words are keywords in Dart
For example, the word `in` is a keyword in Dart and cannot be used as an argument name in a method, but it can be in Java. The current behavior is to add `0` to the end of these reserved names. So `in` becomes `in0`.
_=> A way to specify certain words to rename._
### Some methods and fields are used in `Object`
Like `hashCode` and `toString`. `toString` is also used in Java, however the return type of the method in Java is `JString`. The current behavior is to add `1` to the end of the existing methods. A prettier renaming would be to change `toString` to `toJString` or `toJavaString`.
_=> A way to change method names, this of course needs to take the subclasses into account as well._
### Some names have a different meaning in Dart semantics
In Dart, `Error`s are classes that we normally don't `catch`, in contrast to `Exception`s which we do `catch`. This is largely the same in Java, however the exceptions that are subclasses of `RuntimeException` are more similar to Dart's `Error`s as they're "unchecked exceptions". One might want to rename classes such as `NullPointerException` to `NullPointerError` to better align with Dart semantics.
_=> A way to change class names._
### Some classes have the same names as `dart:core` classes
Even though it's technically possible to either `import dart:core as core`, or `import <generated> as gen`, it's probably easier to have a different name for classes like `Object` and `String` that exist both in Dart and Java. For example, renaming `Object` to `JObject`.
_=> A way to change class names._
### Purely for aesthetics
Self explanatory!
#### Java supports method overloading, but Dart does not
This means that `int foo()`, `int foo(int a)`, and `int foo(int a, int b)` will turn into `foo`, `foo1`, and `foo2` in Dart. An alternative for this would be to let users rename each `foo` or to write a custom logic with optional parameters.
_=> A way to change method names._
### Dart supports operator overloading, but Java does not
A good example of this is `java.lang.BigInteger`. Since Java doesn't support operator overloading, it has methods like `add`, `multiply`, `divide`, … These methods could be changed to their respective operators in Dart to match the language better. `List`'s `get` and `put` also correlate with operators `[]` and `[]=`.
_=> A way to convert methods to operators._
### Dart has getters and setters, but Java does not
Java is full of `getFoo`s and `setFoo`s. We can change these methods to getters and setters in Dart. We might only want to do this if we have a private `foo` field. Similarly some methods are better suited to be getters in Dart, an obvious example of this is `hashCode()` that can be converted to `hashCode` getter.
_=> A way to convert methods to setters/getters._
### We might not want to include certain classes/methods/fields
So, we need to have an option to exclude elements from the generated bindings.
_=> A way to exclude elements._
### Ability to add methods/fields to classes
For example, a `toDartString` could be a good method to add to `JString`, which is not already available. Of course, users could use `extension`s but those have their own limitations for example in case of `static` methods or constructors. This can be as simple as the ability to add any user-defined string to any class.
_=> A way to add custom code._
### Argument names are not available in the bytecode
When parsing from the bytecode, the argument names in methods are not available. This means that a method like `int foo(int bar, double baz)` gets turned into `int foo(int i, double d)`.
_=> A way to rename argument names._
### Ability to implement a Dart class
For example we might want `java.util.List` to implement or mixin `ListMixin` as it is already the case for `package:jni`'s `JList`.
_=> A way to add custom code and to modify the class definition._
# Approaches
## Yaml config with regex support
This is the approach already used by `ffigen`. The syntax is something like:
The point here is not to specify an exact syntax for the yaml, but to give you a general idea of how it can exclude and rename.
### Pros
* Relatively easy to do easy things.
* Declarative.
### Cons
* Impossible to do complicated things, such as checking for some classes' structure or superclass when deciding about renaming. One such example is when trying to change all `(.*)Exception` classes to `$1Error` classes when these classes extend `RuntimeException`. This and many other such examples are simply not possible to do with a yaml config.
* Regex is notoriously difficult to debug.
* It's not obvious in which order the changes happen. Does only one change happen like a switch statement? Or will they happen in a chain from top to bottom?
* Fully qualified names in Java include periods and dollar signs (for inner classes) like `a.b.Foo$Bar` which need to be escaped since we allow regex: `a\.b\.Foo\$Bar` which makes it ugly (and error-prone) to do simple things.
Overall, I think it's not a good idea to have all of the transformation logic in yaml.
## Passing functions to Dart config object
Another option would be to ditch the yaml configuration for more complex tasks and write Dart code. We already support this for normal use cases, and we can extend it to do transformations as well.
We could consider exposing a `Method` object instead of only the name to access things such as the name of the enclosing class.
This is better, we wrap the actual internal `Method` into a wrapper (called `Method` again!) with only the useful information and actions to be exposed to the user. This way any changes to the internal representation of `Method` will not break users' config code.
### Pros
* Still easy to do easy things.
* Possible to do more complicated things.
### Cons
* Having a single method for transforming methods, could lead to a messy code with a bunch of branchings.
## Using transformer visitors
This is basically the same as passing functions to Dart config but could be chained together and can suggest better code organization habits.
### Pros
* Almost the same as passing functions, just packaged differently
### Cons
* Could be a bit unfamiliar for users who are not used to the visitor pattern.
# Another Question, is yaml useful?
Suppose we go with the transformer option, will yaml be still useful? It would be weird to start off with yaml, and transfer everything to a Dart config to add a transformer. Another solution would be to "import" a dart file within the Yaml config like
But what do we gain from keeping the yaml around? Is it that much simpler than having a Dart file for beginners? Is JNIgen ever going to be used by beginners?
I would argue that most actual JNIgen use cases are going to be complicated enough to need some form of transformation.
# Things to keep in mind
## Imported bindings
Users cannot modify the imported bindings as they are already generated. This will impose certain limitations on what they can and cannot do. For example, one cannot rename an imported class, so we probably will have two classes `Class` and `MutableClass`, where `Class` is not renamable. Or have an `imported` boolean getter and throw when trying to modify an imported class.
What about fields and methods? The same method could be used in an imported class, and we cannot change its name if we're extending or implementing that class.
## Reserved Names
Users might want to add certain names to the reserved namespace of a class. These names should not be reused in the subclasses. This happens when a package author adds custom code to introduce useful fields/methods.
## Issues to tackle
- [x] https://github.com/dart-lang/native/issues/612
- [x] https://github.com/dart-lang/native/issues/614
- [x] https://github.com/dart-lang/native/issues/571
- [x] https://github.com/dart-lang/native/issues/569
- [ ] Remove yaml in favor of dart scripts for binding generation
- [x] https://github.com/dart-lang/native/issues/664
- [x] https://github.com/dart-lang/native/issues/694
- [x] https://github.com/dart-lang/native/issues/703
- [x] https://github.com/dart-lang/native/issues/726
- [x] https://github.com/dart-lang/native/issues/751
- [ ] https://github.com/dart-lang/native/issues/746
- [x] https://github.com/dart-lang/native/issues/787
Contributor guide
Assessment
This issue has not been assessed yet.