redpanda-data / redpanda-data/console

Support proto definitions from multiple sources

Open
#606 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

backend feature
Dominant language
TypeScript
Stars
4.3k
Forks
432
Avg merge
3d 6h
Merged PRs (30d)
40

Description

Hi, apologies if this exists already (I couldn't find an issue for it) but I'm trying to configure Console with protos from a couple of different sources and have realized that this doesn't really work.

For example, take a config that looks like this:

kafka:
  protobuf:
    enabled: true
    filesystem:
      enabled: true
      paths:
        - /etc/console/protos
    git:
      enabled: true
      repository:
        url: https://github.com/googleapis/googleapis.git
    mappings:
      - topicName: events
        valueProtoType: my.company.Event

The code here seems to prioritize the protos from the filesystem service and, if you also have a Git mapping like the above, it will quietly overwrite the list of files used by the proto registry with the ones from the filesystem. This effectively means that you can only have one Git source that is all-or-nothing, which may not work for a lot of people. In our case, we'd like to tie in the Google proto APIs in addition to our own internal proto definitions.

It would be really nice to be able to define multiple protobuf sources and have Console read them all and merge the results. If there's a good reason not to do this, I'd suggest mentioning in the docs/config/console.yaml documentation that only one of them may be defined.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in backend/pkg/proto/service.go around the source-selection logic described in the issue, then review docs/config/console.yaml for the current configuration contract. Check how filesystem and Git sources are represented and passed to the proto registry. Done means either multiple configured sources are read together, or the documentation clearly states that only one source may be defined.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, go
Domain
backend, documentation
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.