PAssert ignore the Schema fields names for testing
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
Found this bug while testing Select operator that FieldName gets ignored by Passert here, this code passes
beam version 2.26.0.8
```
```
private static final Schema APP_SCHEMA = Schema.builder()
.addInt32Field("appId")
.addStringField("description")
.addFloatField("rating")
.build();
@Test
public void testProjectOperator(){
PCollection
projectedOutput = generateTestRow(pipeline).apply(Select.fieldNames("appId", "description"));
//
Modified schema with renamed field
Schema modifiedSchema = Schema.builder()
.addInt32Field("appId")
.addStringField("randomName")// this should ideally break
.build();
PAssert.that(projectedOutput).containsInAnyOrder(
Row.withSchema(modifiedSchema).addValues(-8, "Invalid").build(),
Row.withSchema(modifiedSchema).addValues(0,
"Invalid").build(),
Row.withSchema(modifiedSchema).addValues(1, "Recruiter").build(),
Row.withSchema(modifiedSchema).addValues(2,
"Hirein").build(),
Row.withSchema(modifiedSchema).addValues(1, "Workplace").build()
);
pipeline.run().waitUntilFinish();
}
public
static PCollection generateTestRow(Pipeline pipeline) {
// Create a concrete row with that type.
return PBegin
.in(pipeline)
.apply(Create.of(
Row.withSchema(APP_SCHEMA).addValues(-8,
"Invalid", 0f).build(),
Row.withSchema(APP_SCHEMA).addValues(0, "Invalid", -1.1f).build(),
Row.withSchema(APP_SCHEMA).addValues(1, "Recruiter", 4.2f).build(),
Row.withSchema(APP_SCHEMA).addValues(2,
"Hirein", 3.5f).build(),
Row.withSchema(APP_SCHEMA).addValues(1, "Workplace", 3f).build())
.withCoder(RowCoder.of(APP_SCHEMA)));
}
```
Imported from Jira [BEAM-12921](https://issues.apache.org/jira/browse/BEAM-12921). Original Jira may contain additional context.
Reported by: saniljain15.
Contributor guide
Research direction
Start with the PAssert assertion in testProjectOperator and the Select.fieldNames("appId", "description") output shown in the report. Compare the expected Row schema field names with the actual projectedOutput schema, then run this regression test or an equivalent PAssert test. Done means a renamed field such as "randomName" causes the assertion to fail.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100