hashicorp / hashicorp/nomad

`make proto` generates incompatible protobufs with buf v0.41.0

Open
#10,293 11 comments 0 reactions 0 assignees View on GitHub
theme/build-infrastructure type/bug
Dominant language
Go
Stars
17k
Forks
2.1k
Avg merge
1d 9h
Merged PRs (30d)
105

Description

### Nomad version
Nomad v1.1.0-dev

### Issue

Using the wrong version of buf to compile protobufs causes an agent panic.

As of Nomad v1.1.0-dev we need to stick to buf ~v0.30.1~ v0.36.0 which is now installed by `make bootstrap`.

### Reproduction steps

1. Install buf v0.41.0 and run make proto
2. Try to use an external task driver plugin

#### Expected Result

It works. Our wire protocol should always be forward/backward compatible.

#### Actual Result

The Nomad agent panics with:

```
panic: protobuf tag not enough fields in Timestamp.state:

goroutine 350 [running]:
github.com/golang/protobuf/proto.(*unmarshalInfo).computeUnmarshalInfo(0xc000142640)
go/src/github.com/hashicorp/nomad/vendor/github.com/golang/protobuf/proto/table_unmarshal.go:332 +0x1777
github.com/golang/protobuf/proto.(*unmarshalInfo).unmarshal(0xc000142640, 0xc0009c6280, 0xc000850154, 0xc, 0x12, 0x0, 0x2c003b8)
go/src/github.com/hashicorp/nomad/vendor/github.com/golang/protobuf/proto/table_unmarshal.go:136 +0xf45
github.com/golang/protobuf/proto.makeUnmarshalMessagePtr.func1(0xc000850154, 0x12, 0x13, 0xc0001279b0, 0x2, 0xc000507300, 0x2, 0xc000a90320, 0x198, 0x8)
go/src/github.com/hashicorp/nomad/vendor/github.com/golang/protobuf/proto/table_unmarshal.go:1646 +0x12b
github.com/golang/protobuf/proto.(*unmarshalInfo).unmarshal(0xc000142500, 0xc0001279a0, 0xc000850152, 0x14, 0x14, 0x0, 0x2c003b8)
go/src/github.com/hashicorp/nomad/vendor/github.com/golang/protobuf/proto/table_unmarshal.go:173 +0x88e
github.com/golang/protobuf/proto.makeUnmarshalMessagePtr.func1(0xc000850152, 0x14, 0x15, 0xc0006a36b0, 0x2, 0xc00007e9c0, 0x756ea17d84c242, 0x8013eed091f202a3, 0x0, 0xc00007e888)
go/src/github.com/hashicorp/nomad/vendor/github.com/golang/protobuf/proto/table_unmarshal.go:1646 +0x12b
github.com/golang/protobuf/proto.(*unmarshalInfo).unmarshal(0xc0001423c0, 0xc0006a36b0, 0xc000850150, 0x16, 0x16, 0x40f36c, 0x3c05390)
go/src/github.com/hashicorp/nomad/vendor/github.com/golang/protobuf/proto/table_unmarshal.go:173 +0x88e
github.com/golang/protobuf/proto.(*InternalMessageInfo).Unmarshal(0x3bd0240, 0x2bc0dc8, 0xc0006a36b0, 0xc000850150, 0x16, 0x16, 0x1, 0x7ff3f8284bb8)
go/src/github.com/hashicorp/nomad/vendor/github.com/golang/protobuf/proto/table_unmarshal.go:63 +0x66
github.com/hashicorp/nomad/plugins/drivers/proto.(*TaskStatsResponse).XXX_Unmarshal(0xc0006a36b0, 0xc000850150, 0x16, 0x16, 0xc0006a36b0, 0x2705101)
go/src/github.com/hashicorp/nomad/plugins/drivers/proto/driver.pb.go:1139 +0x65
github.com/golang/protobuf/proto.(*Buffer).Unmarshal(0xc000626f38, 0x2bc0dc8, 0xc0006a36b0, 0x0, 0x0)
go/src/github.com/hashicorp/nomad/vendor/github.com/golang/protobuf/proto/decode.go:399 +0xa3
google.golang.org/grpc/encoding/proto.codec.Unmarshal(0xc000850150, 0x16, 0x16, 0x25a6fe0, 0xc0006a36b0, 0x2, 0x2)
go/src/github.com/hashicorp/nomad/vendor/google.golang.org/grpc/encoding/proto/proto.go:93 +0x130
google.golang.org/grpc.recv(0xc000710fc0, 0x7ff3f8378d90, 0x3c05458, 0xc0004b5b00, 0x0, 0x0, 0x25a6fe0, 0xc0006a36b0, 0x7fffffff, 0x0, ...)
go/src/github.com/hashicorp/nomad/vendor/google.golang.org/grpc/rpc_util.go:711 +0x118
google.golang.org/grpc.(*csAttempt).recvMsg(0xc0004b3880, 0x25a6fe0, 0xc0006a36b0, 0x0, 0xc011f76016e2d671, 0x35d3bec09)
go/src/github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:885 +0xee
google.golang.org/grpc.(*clientStream).RecvMsg.func1(0xc0004b3880, 0x8, 0x2823a2a)
go/src/github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:736 +0x46
google.golang.org/grpc.(*clientStream).withRetry(0xc0000d0fc0, 0xc0004d9e80, 0xc0004d9e50, 0x30, 0x28)
go/src/github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:594 +0x9f
google.golang.org/grpc.(*clientStream).RecvMsg(0xc0000d0fc0, 0x25a6fe0, 0xc0006a36b0, 0xc00035ab08, 0xc0008cff7a)
go/src/github.com/hashicorp/nomad/vendor/google.golang.org/grpc/stream.go:735 +0x105
github.com/hashicorp/nomad/plugins/drivers/proto.(*driverTaskStatsClient).Recv(0xc000632e10, 0xc0008cff20, 0x4df569, 0xc0005eaa08)
go/src/github.com/hashicorp/nomad/plugins/drivers/proto/driver.pb.go:4048 +0x62
github.com/hashicorp/nomad/plugins/drivers.(*driverPluginClient).handleStats(0xc00049e840, 0x2bcdf78, 0xc0008785c0, 0xc000403560, 0x2be6e58, 0xc000632e10)
go/src/github.com/hashicorp/nomad/plugins/drivers/client.go:292 +0x7d
created by github.com/hashicorp/nomad/plugins/drivers.(*driverPluginClient).TaskStats
go/src/github.com/hashicorp/nomad/plugins/drivers/client.go:284 +0x3c5
```

Generating protos with buf v0.41.0

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the make proto and make bootstrap targets, then inspect the generated plugins/drivers/proto/driver.pb.go and the driver client stack shown in the panic. Verify generation with the supported buf version and external task driver use; done means the agent no longer panics and the wire protocol remains compatible.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend-api-design, build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.