Kotlin / Kotlin/dataframe

"generic interpretable" functions for compiler plugin

Open
#1,347 0 comments 2 reactions 2 assignees Claimed by @koperagen View on GitHub
Compiler plugin enhancement research
Dominant language
Kotlin
Stars
1.1k
Forks
83
Avg merge
4d 12h
Merged PRs (30d)
30

Description

I had an idea for the compiler plugin which could make working with it a bit easier. I already discussed it with @koperagen some time ago, but let's put it here so we have something to keep track of:

The idea is this: Let's say you have a large function like this that consists solely of interpretable dataframe operations:

```kt
fun DataFrame.convertDuckDbTypes() = this
.convert { colsOf() }.with(infer = Infer.Type) { (it.array as Array).toList() }
.convert { colsOf() }.with { (it as DuckDBStruct).map }
.convert { colsOf>() }.with { it.mapValues { listOf(it.value) }.toDataFrame().single() }
.convert { colsOf() }.with { DataRow.readJsonStr(it.toString()) }
.convert { colsOf() }.toLocalTime()
.convert { colsOf() }.toLocalDate()
.convert { colsOf() }.with { it.toInstant().toKotlinInstant() }
.convert { colsOf() }.with { it.toLocalDateTime().toKotlinLocalDateTime() }
```

Since the compiler plugin can reason about all function invocations inside the body, it should be able to infer the combined effect of the entire `convertDuckDbTypes()` function.

We might need to introduce a new annotation, like `@GenericInterpretable` or simply `@Interpretable` without arguments, such that the compiler plugin becomes aware of this type of function. This also allows us to provide errors when a user puts something inside that the compiler plugin cannot reason about.

Alternatively, it may be possible to recognize these types of functions automatically without annotations, but I'm not entirely sure. They would have in common that they have a DataFrame type in the input, as well as the output.

The return type of this function would also need to be different than `DataFrameType_123`, as it would be a mapping from an original dataframe type to a new one.

Something like this:

```kt
@Interpretable
@Refine
fun DataFrame.doSomething(): DataFrame = renameToCamelCase().convert { id }.toLong()

val df1: DataFrame = ...
val myId: Int = df1.ID

val df2: DataFrame = df1.doSomething()
val myId2: Long = df2.id
```

Function arguments would need special treatment as well, but they could likely be treated similarly to `const` values. This could be added later, though.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.