Aiven-Open / Aiven-Open/karapace

`/schemas/ids/{id}?format=serialized` returns 500 for protobuf schemas with custom references

Ouverte
#1,263 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Python
Étoiles
634
Forks
110
Merge moyen
4 j 7 h
PR mergées (30 j)
4

Description

## Description

When a protobuf schema has custom references (imports other than well-known types like `google/protobuf/timestamp.proto`), the `/schemas/ids/{id}?format=serialized` endpoint returns HTTP 500 Internal Server Error.

The same schema works fine when fetched without `format=serialized` (returns raw proto text). Notably, the `/subjects/{subject}/versions/{version}?format=serialized` endpoint **works correctly** even with custom references — the bug is specific to the `/schemas/ids/{id}?format=serialized` endpoint.

## Steps to Reproduce

1. Start Karapace with Kafka:

```bash
docker run -d --name kafka -p 9092:9092 apache/kafka:latest
docker run -d --name karapace -p 8081:8081 ghcr.io/aiven-open/karapace:6.1.4
```

2. Register a protobuf schema with a custom reference:

```bash
# Register the referenced schema
curl -X POST http://localhost:8081/subjects/shared.proto/versions \
-H "Content-Type: application/vnd.schemaregistry.v1+json" \
-d '{
"schemaType": "PROTOBUF",
"schema": "syntax = \"proto3\";\npackage my.events;\nenum ProductEnum {\n A = 0;\n B = 1;\n}"
}'

# Register the main schema that imports it
curl -X POST http://localhost:8081/subjects/test-topic-value/versions \
-H "Content-Type: application/vnd.schemaregistry.v1+json" \
-d '{
"schemaType": "PROTOBUF",
"schema": "syntax = \"proto3\";\npackage my.events;\nimport \"shared.proto\";\nmessage UserCreatedEvent {\n int32 id = 1;\n ProductEnum product = 2;\n}",
"references": [{"name": "shared.proto", "subject": "shared.proto", "version": 1}]
}'
```

3. Fetch the main schema by ID without `format=serialized` (works):

```bash
curl http://localhost:8081/schemas/ids/2
# Returns 200 with raw proto text ✅
```

4. Fetch the main schema by ID with `format=serialized` (fails):

```bash
curl http://localhost:8081/schemas/ids/2?format=serialized
# Returns 500 ❌
```

5. Fetch the same schema via the **subjects** endpoint with `format=serialized` (works):

```bash
curl http://localhost:8081/subjects/test-topic-value/versions/1?format=serialized
# Returns 200 with base64-encoded FileDescriptorProto ✅
```

6. Register a schema **without** custom references and verify it works on both endpoints:

```bash
curl -X POST http://localhost:8081/subjects/simple-topic-value/versions \
-H "Content-Type: application/vnd.schemaregistry.v1+json" \
-d '{
"schemaType": "PROTOBUF",
"schema": "syntax = \"proto3\";\npackage my.events;\nmessage SimpleEvent {\n int32 id = 1;\n string name = 2;\n}"
}'
# Returns {"id":3}

curl http://localhost:8081/schemas/ids/3?format=serialized
# Returns 200 ✅
```

## Results Summary

| Endpoint | Has References | Result |
|----------|---------------|--------|
| `/schemas/ids/2` | Yes | 200 ✅ |
| `/schemas/ids/2?format=serialized` | Yes | **500 ❌** |
| `/subjects/test-topic-value/versions/1?format=serialized` | Yes | 200 ✅ |
| `/schemas/ids/3?format=serialized` | No | 200 ✅ |

## Expected Behavior

`/schemas/ids/{id}?format=serialized` should return the compiled `FileDescriptorProto` as a base64-encoded string, same as it does for schemas without custom references.

## Actual Behavior

Returns HTTP 500 Internal Server Error. Only the `/schemas/ids/{id}` code path is affected, and only when the schema has custom references (imports).

## Environment

- Karapace version: 6.1.4 (Docker image `ghcr.io/aiven-open/karapace:6.1.4`)

## Impact

This breaks any Kafka consumer that relies on `format=serialized` to get compiled protobuf descriptors for schemas with custom references. The Kafka wire format only embeds the schema ID (not subject/version), so consumers can only use the `/schemas/ids/{id}` endpoint — making this bug unavoidable. The fact that `/subjects/{subject}/versions/{version}?format=serialized` works fine suggests the serialization logic itself is correct, but the `/schemas/ids/{id}` code path fails to resolve references before serializing.

## Workaround

Fetch the raw proto text (without `format=serialized`) and parse it client-side.

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.