tensorflow / tensorflow/tensorboard
Improve Bazel build handling of protobuf dependency situation
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 7.2k
- Forks
- 1.7k
- Avg merge
- 4d 22h
- Merged PRs (30d)
- 1
Description
This is a tracking issue for trying to improve how we work with protobuf in our Bazel build. Currently, it's complex and challenging to understand. Some particular problems we have:
- Lots of custom code in https://github.com/tensorflow/tensorboard/blob/master/tensorboard/defs/protos.bzl that nobody on the team really understands (probably copied from protobuf's internal Bazel code at some point in the past)
- Poor support for separating the protobuf we use for compiling generated code ("protoc") from the protobuf runtime we run that generated code against. Historically, our setup required these to be identical, but the actual released pip package has a floating dependency on the runtime, and we should be able to simulate that in our Bazel build as needed.
- Slow recompilation of protoc whenever
PATHchanges, a longstanding issue: https://github.com/tensorflow/tensorboard/issues/3213
Also, we recently managed, at last, to update our protoc dep to 3.19.6 (#6147 resolving #5708, which was branched out of the user-facing issue #5703). Eventually, it would be good to drop support for the 3.x protobuf python runtime and migrate completely to 4.21+ (which for protoc is just called 21+, due to a confusing versioning system that is only lightly described in https://protobuf.dev/news/2022-08-03/).
To resolve this, we should talk to the protobuf and Bazel folks to try to understand how Bazel + google-internal projects are expected to manage this. It sounds like there might be some changes in this area that could lead to a clearer set of standard practices we can follow.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with tensorboard/defs/protos.bzl and the discussion of the PATH-related recompilation issue in #3213. Consult the referenced protobuf and Bazel guidance and compare the current protoc and runtime dependency handling. Done means documenting and agreeing on a clearer standard approach for this Bazel build.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100