Kotlin / Kotlin/dataframe

Improve `replace` documentation, KDocs, and test coverage

Open
#1,979 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

documentation KDocs tests
Dominant language
Kotlin
Stars
1.1k
Forks
83
Avg merge
4d 12h
Merged PRs (30d)
30

Description

Improve replace documentation, KDocs, and test coverage

Motivation

replace is a core column-replacement operation, but its current documentation coverage is incomplete across the website, KDocs, and unit tests.

Current source:

core/src/main/kotlin/org/jetbrains/kotlinx/dataframe/api/replace.kt

Current tests:

core/src/test/kotlin/org/jetbrains/kotlinx/dataframe/api/replace.kt

Current website page:

https://kotlin.github.io/dataframe/replace.html

The website explains the basic replace { ... }.with(...) flow, but misses several important API behaviors. KDocs are almost absent, and unit tests currently cover only one narrow scenario: replacing a renamed selected column and converting Int to Double.

This is not only a KDocs issue. The operation needs aligned improvements in API documentation, website documentation, and tests.

Problems

1. Public APIs have little or no KDoc

The following APIs need concise KDocs:

DataFrame<T>.replace(columns: ColumnsSelector<T, C>)
DataFrame<T>.replace(vararg columns: String)
DataFrame<T>.replaceAll(...)
ReplaceClause<T, C>
ReplaceClause<T, C>.with(vararg columns: AnyCol)
ReplaceClause<T, C>.with(newColumns: List<AnyCol>)
ReplaceClause<T, C>.with(transform: ColumnsContainer<T>.(DataColumn<C>) -> BaseColumn<*>)

The only existing KDoc on the main with(transform) overload says:

For an alternative supported in the compiler plugin use Convert.asColumn

but it does not explain what replace(...).with { ... } itself does.

2. Website page does not document replaceAll

replaceAll is a public API, but it is not covered on the replace page.

Its default behavior is important:

columns: ColumnsSelector<T, *> = { colsAtAnyDepth().filter { !it.isColumnGroup() } }

So by default it replaces matching values in all non-column-group columns at any depth.

3. Website misses important replacement semantics

The page should explain:

  • replace replaces selected columns with new columns.
  • Selected columns are removed first, then replacement columns are inserted.
  • Replacement column names matter: a renamed replacement column can change the resulting column name/path.
  • with(vararg) and with(List) expect enough replacement columns for the selected columns.
  • If there are fewer replacement columns than selected columns, an exception is thrown.
  • with(transform) receives the original DataFrame as receiver and each selected DataColumn as argument.
  • There is a known limitation/TODO for replacing a ColumnGroup and its child at the same time.
4. Website contains an outdated recommendation

The page currently says that:

replace { columns }.with { columnExpression }

is equivalent to:

convert { columns }.to { columnExpression }

But Convert.to(columnConverter) is deprecated with error level, and replace.kt points users to Convert.asColumn as the compiler-plugin-compatible alternative.

This wording should be updated to avoid recommending an obsolete/deprecated API shape.

5. Unit tests are too narrow

Current tests cover only:

  • selecting a renamed column;
  • replacing it via with { ... };
  • checking the resulting name and type.

They do not cover most public overloads or edge cases.

Website Recommendations

Update replace.html / docs/StardustDocs/topics/replace.md to include:

Basic replace behavior

Explain that replace selects columns and then replaces them using either:

replace { columns }.with(newColumns)

or:

replace { columns }.with { column -> newColumn }
Replacement by new columns

Document both forms:

df.replace { oldCol }.with(newCol)
df.replace { colA and colB }.with(listOf(newA, newB))

Mention that the number of replacement columns must be sufficient for the selected columns.

Replacement by transform

Explain that the lambda:

ColumnsContainer<T>.(DataColumn<C>) -> BaseColumn<*>

receives:

  • the original DataFrame as receiver;
  • the selected column as the lambda argument.
Column naming semantics

Show that replacement columns may keep or change names:

df.replace { age }.with { it.convertToString() }
df.replace { age }.with { it.convertToString().rename("ageText") }

or using named, if that is the preferred style in docs.

replaceAll

Add a separate section for replaceAll, explaining that it replaces values rather than whole columns:

df.replaceAll(null to "N/A")
df.replaceAll("unknown" to null, columns = { name and city })

Clarify default column selection: all non-column-group columns at any depth.

Relationship to convert

Replace the outdated recommendation with something like:

For compiler-plugin-friendly column conversion, prefer `convert { columns }.asColumn { ... }`.

Avoid recommending deprecated convert { ... }.to { ... }.

KDoc Recommendations

Add concise KDocs for public non-deprecated APIs in replace.kt.

KDocs should cover:

  • what the operation replaces;
  • whether it replaces columns or values;
  • what the function returns;
  • whether names and positions are preserved or can change;
  • how selected columns are matched to replacement columns;
  • what happens when not enough replacement columns are provided;
  • default behavior of replaceAll;
  • meaning of the columns selector in replaceAll;
  • relationship to Convert.asColumn for compiler-plugin-compatible column conversion.

Deprecated access-API overloads may remain undocumented unless project policy requires otherwise.

Unit Test Recommendations

Expand core/src/test/kotlin/org/jetbrains/kotlinx/dataframe/api/replace.kt.

Add tests for:

replace
  • Replacing one column while preserving the original name.
  • Replacing one column with a new name.
  • Replacing multiple columns with with(vararg columns).
  • Replacing multiple columns with with(List<AnyCol>).
  • Throwing an exception when fewer replacement columns are provided than selected columns.
  • Replacing columns selected by name.
  • Replacing columns selected through a selector DSL.
  • Replacing a nested value column.
  • Replacing with a transform that uses the original DataFrame receiver.
  • Resulting column order after replacement.
replaceAll
  • Replacing one value pair in the whole DataFrame.
  • Replacing multiple value pairs.
  • Replacing values only in selected columns.
  • Replacing values in nested non-column-group columns by default.
  • Leaving values unchanged when there are no matches.
  • Behavior when duplicate keys are passed in valuePairs; because implementation uses toMap(), the last pair for the same key should win if that is intended behavior.
  • Replacing null values, if supported by current behavior.
Known limitation

Add either:

  • a regression test documenting the current limitation around replacing a ColumnGroup and its child at the same time, if the current behavior is stable enough to assert; or
  • a pending/ignored test or issue reference for the existing TODO:
// TODO: Issue #418: breaks if running on ColumnGroup and its child

Acceptance Criteria

  • Website documentation covers both replace and replaceAll.
  • Website explains the difference between replacing columns and replacing values.
  • Website no longer recommends deprecated convert { ... }.to { ... } usage.
  • Public non-deprecated APIs in replace.kt have concise KDocs.
  • KDocs document replacement-column matching and insufficient-column behavior.
  • KDocs document replaceAll default column selection.
  • Unit tests cover replace overloads using transform, vararg columns, and list columns.
  • Unit tests cover replaceAll with default and explicit column selectors.
  • Unit tests cover nested columns where relevant.
  • Unit tests cover error behavior for insufficient replacement columns.
  • Existing replace named test remains valid or is folded into broader coverage.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with core/src/main/kotlin/org/jetbrains/kotlinx/dataframe/api/replace.kt and core/src/test/kotlin/org/jetbrains/kotlinx/dataframe/api/replace.kt, then read docs/StardustDocs/topics/replace.md and the published replace.html page. Run the existing replace tests first; done means aligned non-deprecated website and KDocs plus coverage for replace/replaceAll overloads, selectors, nesting, naming, errors, and the stated limitation.

Written by the indexing model from the issue text.

Assessment

Tech stack
kotlin
Domain
documentation, testing
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
66/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.