CanonicalSchema() is non-deterministic when a field name matches a record type name in the same namespace
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 1.1k
- Forks
- 232
- PR merge metrics
- No merged PRs in 30d
Description
## Summary
`CanonicalSchema()` produces two different output strings from the same schema input, non-deterministically (~14% vs ~86% split), when a **field name** and a **record type name** share the same local name within the same inherited namespace.
## Reproduction
```go
package main
import (
"fmt"
"github.com/linkedin/goavro/v2"
)
const schema = `{
"type": "record",
"name": "Parent",
"namespace": "com.example",
"fields": [{
"name": "items",
"type": ["null", {
"type": "array",
"items": {
"type": "record",
"name": "items",
"fields": [{"name": "id", "type": "int"}]
}
}],
"default": null
}]
}`
func main() {
seen := map[string]int{}
for i := 0; i < 1000; i++ {
codec, _ := goavro.NewCodec(schema)
seen[codec.CanonicalSchema()]++
}
fmt.Printf("distinct canonical forms: %d\n", len(seen))
for k, v := range seen {
fmt.Printf(" count=%-4d %s\n", v, k)
}
}
```
**Output (example run):**
```
distinct canonical forms: 2
count=862 {"name":"com.example.Parent","type":"record","fields":[{"name":"items","type":["null",{"type":"array","items":{"name":"com.example.items","type":"record","fields":[{"name":"id","type":"int"}]}}]}]}
count=138 {"name":"com.example.Parent","type":"record","fields":[{"name":"com.example.items","type":["null",{"type":"array","items":{"name":"com.example.items","type":"record","fields":[{"name":"id","type":"int"}]}}]}]}
```
The difference is the `"name"` of the `items` field on the outer `Parent` record:
- **Correct (~86%):** `"name":"items"` — the field's declared local name
- **Wrong (~14%):** `"name":"com.example.items"` — incorrectly namespace-qualified
## Expected behaviour
`CanonicalSchema()` should always produce the same output for the same input schema. Per the [Avro Parsing Canonical Form spec](https://avro.apache.org/docs/current/spec.html#Parsing+Canonical+Form+for+Schemas), field names are never namespace-qualified — the second form is incorrect.
## Root cause
goavro builds an internal map of named types during schema parsing. Go map iteration is non-deterministic. When computing the canonical form for a field, the code appears to consult this map to resolve whether the field name is a type reference, and non-deterministic iteration occasionally causes the field's local name to be replaced with the fully-qualified type name of the same-named record.
The field `"name": "items"` collides with the record type `com.example.items`. When goavro's named-type lookup happens to find that match during canonical form computation, it emits `"com.example.items"` as the field name instead of `"items"`.
## Impact
Any system that uses `CanonicalSchema()` output as a fingerprint (e.g. to compare schemas between a registry and a stored file) will see spurious mismatches ~14% of the time when this collision exists. We encountered this when validating exported Avro OCF files against a Pub/Sub schema registry — ~22 out of 135 daily files were flagged as mismatched despite containing identical schemas.
## Workaround
Rename the record type (or field) so they no longer share a local name. This makes `CanonicalSchema()` deterministic again.
## Environment
- goavro version: v2.15.0 (latest)
- Go version: 1.25.4
- OS: darwin/arm64 (also reproducible on linux/amd64 in CI)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at CanonicalSchema() and the schema-parsing code that builds the named-type map, using the supplied reproduction to trigger the field/type-name collision. Add a regression test showing repeated canonicalization is deterministic and that the outer field remains named "items"; done means every output matches the correct canonical form.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100