Kotlin / Kotlin/dataframe

Spring JDBC integration for Kotlin DataFrame (Gradle projects)

Open
#1,651 0 comments 0 reactions 1 assignee Claimed by @zaleslaw View on GitHub
examples research
Dominant language
Kotlin
Stars
1.1k
Forks
83
Avg merge
4d 12h
Merged PRs (30d)
30

Description

This issue tracks research on extending Kotlin DataFrame with first-class integration support for Spring JDBC in Gradle-based projects.

The goal is to explore and define:

- a recommended integration pattern between Spring JDBC and Kotlin DataFrame,
- a minimal, idiomatic example for users,
- a short guide describing how DataFrame fits into Spring JDBC–based architectures.

The focus is on read-oriented use cases (analytics, reporting, data processing), not ORM or CRUD replacement.

**Proposed pattern:**

Use Spring JDBC as a thin SQL execution layer (JdbcTemplate / NamedParameterJdbcTemplate), and convert query results directly into Kotlin DataFrame for in-memory transformations, aggregation, and analysis. Spring manages infrastructure, while DataFrame handles tabular data logic.

Benefits

- Seamless adoption of DataFrame in existing Spring JDBC projects
- Clear separation between SQL access and data transformation logic
- No ORM or Spring Data dependency required
- Idiomatic Kotlin API for analytics and reporting
- Lower entry barrier for users already using Spring JDBC

**Best practice**

```kt
@Component
class SalesAnalyticsRepository(
private val jdbcTemplate: JdbcTemplate
) {

fun salesDf(from: LocalDate, to: LocalDate): DataFrame =
jdbcTemplate.query(
"""
select date, region, amount
from sales
where date between ? and ?
""",
from, to
) { rs ->
rs.toDataFrame()
}
}

data class SaleRow(
val date: LocalDate,
val region: String,
val amount: BigDecimal
)

@Service
class SalesReportService(
private val repo: SalesAnalyticsRepository
) {

fun revenueByRegion(from: LocalDate, to: LocalDate): DataFrame<*> =
repo.salesDf(from, to)
.groupBy { region }
.aggregate {
sum { amount }
}
}

```

Anti-patterns

```kt
@Service
class SalesService(
private val jdbcTemplate: JdbcTemplate
) {

fun revenueByRegion(): DataFrame<*> {
val df = jdbcTemplate.query(
"select * from sales"
) { rs ->
rs.toDataFrame()
}

// all is mixed here
return df
.filter { amount > BigDecimal.ZERO }
.groupBy("region")
.aggregate {
sum("amount")
}
}
}

or

interface SalesRepository {
fun findAll(): DataFrame<*> // bery bad
}

```

Possible API changes:

```kt
inline fun JdbcTemplate.readDataFrame(
sql: String,
limit: Int? = null,
inferNullability: Boolean = true,
dbType: DbType? = null
): DataFrame

fun NamedParameterJdbcTemplate.readDataFrame(
sql: String,
params: Map,
limit: Int? = null,
inferNullability: Boolean = true,
dbType: DbType? = null
): AnyFrame

inline fun NamedParameterJdbcTemplate.readDataFrame(
sql: String,
params: Map,
limit: Int? = null,
inferNullability: Boolean = true,
dbType: DbType? = null
): DataFrame

fun JdbcTemplate.readDataFrame(
sql: String,
limit: Int?,
inferNullability: Boolean,
dbType: DbType?
): AnyFrame =
query(sql) { rs ->
DataFrame.readResultSet(
resultSet = rs,
dbType = dbType ?: DbType.from(rs.metaData),
limit = limit,
inferNullability = inferNullability
)
}

```

Architecture notes:

Spring manages:
- DataSource
- Connection
- Transactions

DataFrame manages:
- Schema inference
- Type mapping
- In-memory tabular operations

Spring JDBC extensions:
- Glue code only

My thoughts on why it should be done after possible Spring Integration

**Top-5 UX Problems for Spring Users**

_Duplicate configuration_
Spring applications already define DataSource, but KDF requires a separate DbConnectionConfig.

_Connection lifecycle mismatch_
Spring manages connections and transactions, while KDF expects users to reason about connection handling explicitly.

_Extra cognitive load (DbType)_
Spring users must manually think about DbType, even though the database is already known and configured.

_Unclear best practices_
Users are unsure whether they are integrating KDF with Spring in a correct and idiomatic way.

_Perception mismatch_
The lack of Spring-friendly entry points creates the impression that KDF is not designed for Spring-based applications.

Inspired by added functionality https://x.com/intellijidea/status/2000566592384422334

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.