apache / apache/beam

[Bug]: Unable to infer a coder for JdbcIO.read() transform when using a custom class with @DefaultCoder annotation

Open
#26,003 5 comments 0 reactions 0 assignees View on GitHub
bug io java jdbc P3
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

### What happened?

I am using Apache Beam's JdbcIO to read data from a PostgreSQL database and log the retrieved data. I'm using a custom UserData class and have applied the @DefaultCoder(AvroCoder.class) annotation to it. However, I am still encountering an error related to the coder.

Error message:
```
java.lang.IllegalStateException: Unable to infer a coder for JdbcIO.read() transform.
```

Here's the UserData class:
```java
package me.jics;

import lombok.Builder;
import lombok.EqualsAndHashCode;
import lombok.Getter;
import lombok.ToString;
import lombok.Value;
import lombok.extern.jackson.Jacksonized;
import org.apache.beam.sdk.coders.AvroCoder;
import org.apache.beam.sdk.coders.DefaultCoder;

@Jacksonized
@Builder
@Getter
@Value
@EqualsAndHashCode
@ToString
@DefaultCoder(AvroCoder.class)
public class UserData {
String name;
String lastname;
}

```
And here's the main part of the pipeline in the `AppCommand` class:

```java
public void run() {
AppOptions options = PipelineOptionsFactory
.fromArgs("--runner=DirectRunner")
.as(AppOptions.class);
Pipeline p = Pipeline.create(options);
p.apply("Selecting", JdbcIO.read()
.withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(
"org.postgresql.Driver",
"jdbc:postgres://localhost:5432/mydb")
.withUsername("myuser")
.withPassword("mypassword"))
.withQuery("select name, lastname from user where name = 'john'")
.withRowMapper(resultSet -> UserData.builder()
.name(resultSet.getString(1))
.lastname(resultSet.getString(2))
.build())
.withOutputParallelization(false)
)
.apply("nex step", MapElements.via(new SimpleFunction() {
@Override
public Integer apply(UserData input) {
log.info(input.toString());
return 1;
}
}));

p.run();
}

```
I would appreciate any guidance on the recommended way to set the coder for JdbcIO in this scenario, considering I've already applied the `@DefaultCoder` annotation to my custom `UserData` class.

here sample project: https://github.com/j1cs/coder-error-beam
mail where the discussion started: https://lists.apache.org/thread/7wxmr3s7vcrll3mvb07rmj5cmco4wtn8

### Issue Priority

Priority: 3 (minor)

### Issue Components

- [ ] Component: Python SDK
- [X] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.