Aiven-Open / Aiven-Open/karapace
`/schemas/ids/{id}?format=serialized` returns 500 for protobuf schemas with custom references
- Langage dominant
- Python
- Étoiles
- 634
- Forks
- 110
- Merge moyen
- 4 j 7 h
- PR mergées (30 j)
- 4
Description
## Description
When a protobuf schema has custom references (imports other than well-known types like `google/protobuf/timestamp.proto`), the `/schemas/ids/{id}?format=serialized` endpoint returns HTTP 500 Internal Server Error.
The same schema works fine when fetched without `format=serialized` (returns raw proto text). Notably, the `/subjects/{subject}/versions/{version}?format=serialized` endpoint **works correctly** even with custom references — the bug is specific to the `/schemas/ids/{id}?format=serialized` endpoint.
## Steps to Reproduce
1. Start Karapace with Kafka:
```bash
docker run -d --name kafka -p 9092:9092 apache/kafka:latest
docker run -d --name karapace -p 8081:8081 ghcr.io/aiven-open/karapace:6.1.4
```
2. Register a protobuf schema with a custom reference:
```bash
# Register the referenced schema
curl -X POST http://localhost:8081/subjects/shared.proto/versions \
-H "Content-Type: application/vnd.schemaregistry.v1+json" \
-d '{
"schemaType": "PROTOBUF",
"schema": "syntax = \"proto3\";\npackage my.events;\nenum ProductEnum {\n A = 0;\n B = 1;\n}"
}'
# Register the main schema that imports it
curl -X POST http://localhost:8081/subjects/test-topic-value/versions \
-H "Content-Type: application/vnd.schemaregistry.v1+json" \
-d '{
"schemaType": "PROTOBUF",
"schema": "syntax = \"proto3\";\npackage my.events;\nimport \"shared.proto\";\nmessage UserCreatedEvent {\n int32 id = 1;\n ProductEnum product = 2;\n}",
"references": [{"name": "shared.proto", "subject": "shared.proto", "version": 1}]
}'
```
3. Fetch the main schema by ID without `format=serialized` (works):
```bash
curl http://localhost:8081/schemas/ids/2
# Returns 200 with raw proto text ✅
```
4. Fetch the main schema by ID with `format=serialized` (fails):
```bash
curl http://localhost:8081/schemas/ids/2?format=serialized
# Returns 500 ❌
```
5. Fetch the same schema via the **subjects** endpoint with `format=serialized` (works):
```bash
curl http://localhost:8081/subjects/test-topic-value/versions/1?format=serialized
# Returns 200 with base64-encoded FileDescriptorProto ✅
```
6. Register a schema **without** custom references and verify it works on both endpoints:
```bash
curl -X POST http://localhost:8081/subjects/simple-topic-value/versions \
-H "Content-Type: application/vnd.schemaregistry.v1+json" \
-d '{
"schemaType": "PROTOBUF",
"schema": "syntax = \"proto3\";\npackage my.events;\nmessage SimpleEvent {\n int32 id = 1;\n string name = 2;\n}"
}'
# Returns {"id":3}
curl http://localhost:8081/schemas/ids/3?format=serialized
# Returns 200 ✅
```
## Results Summary
| Endpoint | Has References | Result |
|----------|---------------|--------|
| `/schemas/ids/2` | Yes | 200 ✅ |
| `/schemas/ids/2?format=serialized` | Yes | **500 ❌** |
| `/subjects/test-topic-value/versions/1?format=serialized` | Yes | 200 ✅ |
| `/schemas/ids/3?format=serialized` | No | 200 ✅ |
## Expected Behavior
`/schemas/ids/{id}?format=serialized` should return the compiled `FileDescriptorProto` as a base64-encoded string, same as it does for schemas without custom references.
## Actual Behavior
Returns HTTP 500 Internal Server Error. Only the `/schemas/ids/{id}` code path is affected, and only when the schema has custom references (imports).
## Environment
- Karapace version: 6.1.4 (Docker image `ghcr.io/aiven-open/karapace:6.1.4`)
## Impact
This breaks any Kafka consumer that relies on `format=serialized` to get compiled protobuf descriptors for schemas with custom references. The Kafka wire format only embeds the schema ID (not subject/version), so consumers can only use the `/schemas/ids/{id}` endpoint — making this bug unavoidable. The fact that `/subjects/{subject}/versions/{version}?format=serialized` works fine suggests the serialization logic itself is correct, but the `/schemas/ids/{id}` code path fails to resolve references before serializing.
## Workaround
Fetch the raw proto text (without `format=serialized`) and parse it client-side.
Guide de contribution
Ouvrir le guide de contribution
Évaluation
Cette issue n'a pas encore été évaluée.