apache / apache/beam

PAssert ignore the Schema fields names for testing

Open
#21,174 0 comments 0 reactions 0 assignees View on GitHub
bug core java P3
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

Found this bug while testing Select operator that FieldName gets ignored by Passert here, this code passes

beam version 2.26.0.8

```
```

private static final Schema APP_SCHEMA = Schema.builder()
.addInt32Field("appId")
.addStringField("description")

.addFloatField("rating")
.build();

@Test
public void testProjectOperator(){
PCollection
projectedOutput = generateTestRow(pipeline).apply(Select.fieldNames("appId", "description"));

//
Modified schema with renamed field
Schema modifiedSchema = Schema.builder()
.addInt32Field("appId")

.addStringField("randomName")// this should ideally break
.build();

PAssert.that(projectedOutput).containsInAnyOrder(

Row.withSchema(modifiedSchema).addValues(-8, "Invalid").build(),
Row.withSchema(modifiedSchema).addValues(0,
"Invalid").build(),
Row.withSchema(modifiedSchema).addValues(1, "Recruiter").build(),
Row.withSchema(modifiedSchema).addValues(2,
"Hirein").build(),
Row.withSchema(modifiedSchema).addValues(1, "Workplace").build()
);

pipeline.run().waitUntilFinish();
}

public
static PCollection generateTestRow(Pipeline pipeline) {
// Create a concrete row with that type.

return PBegin
.in(pipeline)
.apply(Create.of(
Row.withSchema(APP_SCHEMA).addValues(-8,
"Invalid", 0f).build(),
Row.withSchema(APP_SCHEMA).addValues(0, "Invalid", -1.1f).build(),

Row.withSchema(APP_SCHEMA).addValues(1, "Recruiter", 4.2f).build(),
Row.withSchema(APP_SCHEMA).addValues(2,
"Hirein", 3.5f).build(),
Row.withSchema(APP_SCHEMA).addValues(1, "Workplace", 3f).build())

.withCoder(RowCoder.of(APP_SCHEMA)));
}
```

 

 

Imported from Jira [BEAM-12921](https://issues.apache.org/jira/browse/BEAM-12921). Original Jira may contain additional context.
Reported by: saniljain15.

Contributor guide

Open the contributing guide

Research direction

Start with the PAssert assertion in testProjectOperator and the Select.fieldNames("appId", "description") output shown in the report. Compare the expected Row schema field names with the actual projectedOutput schema, then run this regression test or an equivalent PAssert test. Done means a renamed field such as "randomName" causes the assertion to fail.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.