Docs on protobufnative and pulsar-admin schema upload
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
**Is your feature request related to a problem? Please describe.**
it appears that the `pulsar-admin schemas upload` works with protobufnative
however there is little documentation on the subject
https://pulsar.apache.org/docs/en/schema-manage/
**Describe the solution you'd like**
- documentation on how to use `pulsar-admin schemas upload` with protobufnative
- guidance on how to create the `EXPECTED_SCHEMA_JSON`
referenced here https://github.com/apache/pulsar/blob/4128151412031f9a354d2296b69533d9533a9fbf/pulsar-client/src/test/java/org/apache/pulsar/client/impl/schema/ProtobufNativeSchemaTest.java#L41
- sidenote: I am aware I can generate the "schema" as such `protoc --include_imports --descriptor_set_out=/dev/stdout Test.proto | base64` just not aware of a way to generate the whole structure if possible, or if there is another preferred way
**Describe alternatives you've considered**
given this structure:
```
{
"type": "PROTOBUF_NATIVE",
"schema": "{\"fileDescriptorSet\":\"CtMDCgpUZXN0LnByb3RvEgVwcm90bxoSRXh0ZXJuYWxUZXN0LnByb3RvImUKClN1Yk1lc3NhZ2USCwoDZm9vGAEgASgJEgsKA2JhchgCIAEoARo9Cg1OZXN0ZWRNZXNzYWdlEgsKA3VybBgBIAEoCRINCgV0aXRsZRgCIAEoCRIQCghzbmlwcGV0cxgDIAMoCSLlAQoLVGVzdE1lc3NhZ2USEwoLc3RyaW5nRmllbGQYASABKAkSEwoLZG91YmxlRmllbGQYAiABKAESEAoIaW50RmllbGQYBiABKAUSIQoIdGVzdEVudW0YBCABKA4yDy5wcm90by5UZXN0RW51bRImCgtuZXN0ZWRGaWVsZBgFIAEoCzIRLnByb3RvLlN1Yk1lc3NhZ2USFQoNcmVwZWF0ZWRGaWVsZBgKIAMoCRI4Cg9leHRlcm5hbE1lc3NhZ2UYCyABKAsyHy5wcm90by5leHRlcm5hbC5FeHRlcm5hbE1lc3NhZ2UqJAoIVGVzdEVudW0SCgoGU0hBUkVEEAASDAoIRkFJTE9WRVIQAUItCiVvcmcuYXBhY2hlLnB1bHNhci5jbGllbnQuc2NoZW1hLnByb3RvQgRUZXN0YgZwcm90bzMKoAEKEkV4dGVybmFsVGVzdC5wcm90bxIOcHJvdG8uZXh0ZXJuYWwiOwoPRXh0ZXJuYWxNZXNzYWdlEhMKC3N0cmluZ0ZpZWxkGAEgASgJEhMKC2RvdWJsZUZpZWxkGAIgASgBQjUKJW9yZy5hcGFjaGUucHVsc2FyLmNsaWVudC5zY2hlbWEucHJvdG9CDEV4dGVybmFsVGVzdGIGcHJvdG8z\",\"rootMessageTypeName\":\"proto.TestMessage\",\"rootFileDescriptorName\":\"Test.proto\"}",
"properties": {}
}
```
`pulsar-admin schemas upload --filename $PWD/schema_proto_file_descript_set.json apache/pulsar/test-topic`
appeared to have worked
**Additional context**
When a schema is uploaded in an incorrect format pulsar provides no error information other than a 500 that I can tell
cli output:
```
14:24:15.038 [AsyncHttpClient-7-1] WARN org.apache.pulsar.client.admin.internal.BaseResource - [http://localhost:8080/admin/v2/schemas/apache/pulsar/test-topic/schema] Failed to perform http post request: javax.ws.rs.InternalServerErrorException: HTTP 500 Internal Server Error
HTTP 500 Internal Server Error
```
standalone server output:
```
14:24:14.959 [pulsar-30-5] INFO org.apache.pulsar.broker.namespace.OwnershipCache - Trying to acquire ownership of apache/pulsar/0x40000000_0x80000000
14:24:14.986 [pulsar-ordered-OrderedExecutor-6-0-EventThread] INFO org.apache.pulsar.broker.namespace.OwnershipCache - Successfully acquired ownership of /namespace/apache/pulsar/0x40000000_0x80000000
14:24:15.019 [ForkJoinPool.commonPool-worker-5] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [12/Jun/2021:14:24:14 -0600] "POST /admin/v2/schemas/apache/pulsar/test-topic/schema HTTP/1.1" 500 180 "-" "Pulsar-Java-v2.7.2" 69
```
Contributor guide
Research direction
Start with the schema-manage documentation and the referenced ProtobufNativeSchemaTest.java example, then compare it with the shown pulsar-admin schemas upload command. Document how to create the complete protobufnative schema structure, including EXPECTED_SCHEMA_JSON, and clarify the expected upload format and failure behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- cli, documentation
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100