Kotlin / Kotlin/dataframe

Add `replace { }.by { }` API for multi-column replacement using AddDsl

Open
#1,749 10 comments 0 reactions 0 assignees View on GitHub
API Compiler plugin enhancement
Dominant language
Kotlin
Stars
1.1k
Forks
83
Avg merge
4d 12h
Merged PRs (30d)
30

Description

Converting our "movies" example to the compiler plugin, I came across the following notation:

```kt
.split { title }.by {
listOf(
"""\s*\(\d{4}\)\s*$""".toRegex().replace(title, ""),
"\\d{4}".toRegex().findAll(title).lastOrNull()?.value?.toIntOrNull() ?: -1,
)
}.into("title", "year")
```

This creates the columns `title: DataColumn` and `year: DataColumn`. The problem with this `list` notation is that the compiler plugin cannot read the types easily. It can only see them as `Any`, because that's the type of the list.

I tried several alternatives, but they all boiled down to multiple steps:
- add `title2` and `year`, remove the old `title`, rename `title2` to `title`
- replace or convert the `title` column to a column group of `title`/`year`, ungroup it again
- use `split` and then `cast` or `requireColumn`

But all seem to be way too complicated for what we want here: simply replacing a column with two new ones.

A new API could look like:
```kt
df.replace { title }.by { // AddDsl
"title" from {
"""\s*\(\d{4}\)\s*$""".toRegex().replace(title, "")
}
"year" from {
"\\d{4}".toRegex().findAll(title).lastOrNull()?.value?.toIntOrNull() ?: -1
}
}
```

or more generally:
```kt
df.replace { name and firstName and age }.by {
"name" from {
"$firstName $name ($age)"
}
"welcomeMessage" from {
"Hi, $firstName!"
}
}
```
We could explore other notations or names, of course, but it would narrow down to:
- remove some columns
- add some new ones

in one operation

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the existing split and into operations and the AddDsl notation shown in the issue. Compare the proposed replace { }.by { } forms and determine a consistent API for removing selected columns and adding typed replacement columns in one operation. Done means the API design is settled and the multi-column replacement use cases are supported.

Written by the indexing model from the issue text.

Assessment

Tech stack
kotlin
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.