temporal-history panic

Expected Behavior

temporal-history pod should not panic and exit.

Actual Behavior

"2024-09-23T09:07:36.195Z, "Fail to process task""2024-09-23T09:07:36.195Z, "Critical error processing task, retrying."
"2024-09-23T09:08:19.217Z, "panic: runtime error: slice bounds out of range [-30:]"
"2024-09-23T09:08:19.217Z, "goroutine 150888820 [running]:"
"2024-09-23T09:08:19.217Z, "go.temporal.io/api/common/v1.(*WorkflowExecution).MarshalToSizedBuffer(0x403a217260, {0x403f2be000, 0x47, 0x2fa?})"
"2024-09-23T09:08:19.217Z, "/go/pkg/mod/go.temporal.io/api@v1.24.0/common/v1/message.pb.go:1712 +0x20c"
"2024-09-23T09:08:19.217Z, "go.temporal.io/api/workflow/v1.(*WorkflowExecutionInfo).MarshalToSizedBuffer(0x40163aea20, {0x403f2be000, 0x2ca, 0x2fa})"
"2024-09-23T09:08:19.217Z, "/go/pkg/mod/go.temporal.io/api@v1.24.0/workflow/v1/message.pb.go:1761 +0xaf0"
"2024-09-23T09:08:19.217Z, "go.temporal.io/server/api/historyservice/v1.(*DescribeWorkflowExecutionResponse).MarshalToSizedBuffer(0x403acb79f0, {0x403f2be000, 0x2fa, 0x2fa})"
"2024-09-23T09:08:19.218Z, "/home/builder/temporal/api/historyservice/v1/request_response.pb.go:13966 +0x134"
"2024-09-23T09:08:19.218Z, "go.temporal.io/server/api/historyservice/v1.(*DescribeWorkflowExecutionResponse).Marshal(0x401d86f630?)"
"2024-09-23T09:08:19.218Z, "/home/builder/temporal/api/historyservice/v1/request_response.pb.go:13907 +0x50"
"2024-09-23T09:08:19.218Z, "google.golang.org/protobuf/internal/impl.legacyMarshal({{}, {0x26d7548, 0x401d86f630}, {0x0, 0x0, 0x0}, 0x0})"
"2024-09-23T09:08:19.218Z, "/go/pkg/mod/google.golang.org/protobuf@v1.31.0/internal/impl/legacy_message.go:402 +0x84"
"2024-09-23T09:08:19.218Z, "google.golang.org/protobuf/proto.MarshalOptions.marshal({{}, 0xc8?, 0x0, 0x0}, {0x0, 0x0, 0x0}, {0x26d7548, 0x401d86f630})"
"2024-09-23T09:08:19.218Z, "/go/pkg/mod/google.golang.org/protobuf@v1.31.0/proto/encode.go:166 +0x1b8"
"2024-09-23T09:08:19.218Z, "google.golang.org/protobuf/proto.MarshalOptions.MarshalAppend({{}, 0xa0?, 0x17?, 0x1?}, {0x0, 0x0, 0x0}, {0x26a7ac0?, 0x401d86f630?})"
"2024-09-23T09:08:19.218Z, "/go/pkg/mod/google.golang.org/protobuf@v1.31.0/proto/encode.go:125 +0x70"
"2024-09-23T09:08:19.218Z, "github.com/golang/protobuf/proto.marshalAppend({0x0, 0x0, 0x0}, {0xffff781c1fc8?, 0x403acb79f0?}, 0x0?)"
"2024-09-23T09:08:19.218Z, "/go/pkg/mod/github.com/golang/protobuf@v1.5.3/proto/wire.go:40 +0x78"
"2024-09-23T09:08:19.218Z, "github.com/golang/protobuf/proto.Marshal(...)"
"2024-09-23T09:08:19.218Z, "/go/pkg/mod/github.com/golang/protobuf@v1.5.3/proto/wire.go:23"
"2024-09-23T09:08:19.218Z, "google.golang.org/grpc/encoding/proto.codec.Marshal({}, {0x20117a0, 0x403acb79f0})"
"2024-09-23T09:08:19.218Z, "/go/pkg/mod/google.golang.org/grpc@v1.57.0/encoding/proto/proto.go:45 +0x54"
"2024-09-23T09:08:19.218Z, "google.golang.org/grpc.encode({0xffff78ca0ba8?, 0x3b1ec18?}, {0x20117a0?, 0x403acb79f0?})"
"2024-09-23T09:08:19.218Z, "/go/pkg/mod/google.golang.org/grpc@v1.57.0/rpc_util.go:633 +0x40"
"2024-09-23T09:08:19.218Z, "google.golang.org/grpc.(*Server).sendResponse(0x4000a741e0, {0x26cf060, 0x40092c1ba0}, 0x402ce2bb00, {0x20117a0?, 0x403acb79f0}, {0x0, 0x0}, 0xffffffffffffffff?, {0x0, ...})"
"2024-09-23T09:08:19.218Z, "/go/pkg/mod/google.golang.org/grpc@v1.57.0/server.go:1123 +0x160"
"2024-09-23T09:08:19.218Z, "google.golang.org/grpc.(*Server).processUnaryRPC(0x4000a741e0, {0x26cf060, 0x40092c1ba0}, 0x402ce2bb00, 0x4000be1890, 0x3abd138, 0x0)"
"2024-09-23T09:08:19.218Z, "/go/pkg/mod/google.golang.org/grpc@v1.57.0/server.go:1407 +0xd44"
"2024-09-23T09:08:19.218Z, "google.golang.org/grpc.(*Server).handleStream(0x4000a741e0, {0x26cf060, 0x40092c1ba0}, 0x402ce2bb00, 0x0)"
"2024-09-23T09:08:19.218Z, "/go/pkg/mod/google.golang.org/grpc@v1.57.0/server.go:1737 +0x80c"
"2024-09-23T09:08:19.218Z, "google.golang.org/grpc.(*Server).serveStreams.func1.1()"
"2024-09-23T09:08:19.218Z, "/go/pkg/mod/google.golang.org/grpc@v1.57.0/server.go:982 +0x84"
"2024-09-23T09:08:19.218Z, "created by google.golang.org/grpc.(*Server).serveStreams.func1"
"2024-09-23T09:08:19.218Z, "/go/pkg/mod/google.golang.org/grpc@v1.57.0/server.go:980 +0x164"

Steps to Reproduce the Problem

n/a

Specifications

• Version: 1.20
• Platform: eks v1.29.8-eks-a737599, Amazon Linux 2(arm64)

Issue might be related to https://github.com/temporalio/temporal/issues/4159

Start with the panic path in api/historyservice/v1/request_response.pb.go and the generated WorkflowExecution marshaling frame at go.temporal.io/api@v1.24.0/common/v1/message.pb.go:1712. Reproduce the DescribeWorkflowExecution response failure if possible, then trace the malformed response data and verify that temporal-history no longer panics or exits under the reported conditions.