containerd / containerd/ttrpc

Client context is not honored while waiting for sendLock

Open
#258 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
664
Forks
86
PR merge metrics
No merged PRs in 30d

Description

## Description

This issue tracks a cancellation gap that remains in [#227](https://github.com/containerd/ttrpc/pull/227).

PR #227 makes the active socket write context-aware. That is necessary and fixes the case where the request currently blocked in `Write` or `Flush` has an expiring context.

However, it leaves acquisition of the client-wide send lock unchanged:

```go
c.sendLock.Lock()
defer c.sendLock.Unlock()
```

Consequently, request B cannot observe its own context while request A owns `sendLock`. If A has a non-expiring or long-lived context and blocks while writing, B can remain inside `sync.Mutex.Lock()` indefinitely—even after B's deadline expires.

## Execution path

```text
Request A
-> createStream
-> sendLock.Lock()
-> channel.send
-> bufio.Writer.Flush() // blocked while holding sendLock

Request B
-> createStream
-> sendLock.Lock() // cannot observe B's ctx.Done()
```

`dispatch` does not reach its context-aware response wait until `createStream` returns.

The same problem exists in `Client.send`, affecting streaming `SendMsg` and `CloseSend` calls.


## Expected behavior

A request canceled while waiting to submit its frame should return promptly with `context.Canceled` or `context.DeadlineExceeded`.

Its lifetime should not be determined by the context of the request currently holding the send serializer.

## Actual behavior

If request A blocks while holding `sendLock`:

1. Request B waits in `sendLock.Lock()`.
2. B's context expires.
3. B remains blocked.
4. Further requests accumulate behind the same writer.
5. The goroutines are released only when A returns or the connection is closed externally.

## Impact

This can create connection-wide head-of-line blocking when a single `ttrpc` client is shared by unrelated operations.

In Kata-containers case:

- Metrics and lifecycle requests shared one long-lived ttrpc connection.
- If metrics send stopped making progress it's still holding the send lock.
- Later lifecycle requests can't be submitted.
- A process signal could not reach the guest, preventing normal container shutdown and cleanup.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating Client.createStream, Client.send, dispatch, and sendLock, then trace how cancellation is handled around channel.send and Flush. Done means a request waiting for the serializer returns promptly with context.Canceled or context.DeadlineExceeded; the issue names no file or test to run, so the surrounding client cancellation tests will need to be identified.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend-api-design
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.