hyperledger / hyperledger/fabric-x

Orderer broadcast can hang indefinitely because client ignores configured connection timeout

Open
#194 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
64
Forks
80
Avg merge
1d 22h
Merged PRs (30d)
15

Description

## Severity
High

## Description
`OrdererClient.send` wraps the caller context using `context.WithCancel`, but does not apply the configured `ConnectionTimeout`. As a result, the broadcast RPC is only bounded if the caller explicitly provides a deadline.

When `SubmitTransaction` or `SubmitTransactionWithWait` is invoked with a long-lived or background context, a healthy connection to an unresponsive orderer can block indefinitely in `Broadcast`, `Send`, or `Recv`.

## Root Cause
Although `ConnectionTimeout` is used during gRPC connection setup, it is not enforced during the broadcast RPC lifecycle.

In:
- `tools/fxconfig/internal/client/orderer.go` (send method)

The context is wrapped with:
```go
context.WithCancel(ctx)
```

to replace applying a timeout.

The call chain from:
- `tools/fxconfig/internal/app/submit.go`

propagates this behavior to both submission paths.

## Steps to Reproduce
1. Configure fxconfig with a valid orderer endpoint and finite ConnectionTimeout.
2. Invoke:
- SubmitTransaction or
- SubmitTransactionWithWait
using a context without a deadline (e.g., `context.Background()`).
3. Use an orderer endpoint that:
- accepts the broadcast stream,
- does not send a final response (e.g., test stub blocking on Recv).
4. Observe:
- the call blocks indefinitely,
- no timeout is triggered.

## Expected Behavior
The broadcast RPC should be bounded by:
- the caller's deadline (if present), OR
- the configured ConnectionTimeout.

## Actual Behavior
The RPC only terminates if the caller context is canceled.
No timeout is enforced from client configuration.

## Impact
to:
d Transaction submission can hang indefinitely,
d CLI commands may never return,
d SubmitTransactionWithWait violates expected bounded wait semantics,
and can cause resource exhaustion in long-running processes.

## Suggested Fix (Minimal)

Apply a timeout only when the caller has not provided one:

```go
if _, ok := ctx.Deadline(); !ok {
ctx, cancel = context.WithTimeout(ctx, oc.cfg.ConnectionTimeout)
} else {
ctx, cancel = context.WithCancel(ctx)
}
defer cancel()
```

This ensures:
- Caller deadlines are respected
- A fallback timeout is enforced otherwise
## Edge Cases

- If caller provides a shorter deadline → it should take precedence.
- Timeout should not override stricter upstream constraints.
- Ensure cancellation is still propagated correctly to gRPC calls.
cc / @mbrandenburger

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.