Assertion failed: (!REPLY_RING_EMPTY(sc)), function send_response_common, file src/lib/pci_virtio_sock.c, line 934.
Open
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 3.7k
- Forks
- 336
- PR merge metrics
- No merged PRs in 30d
Description
In https://github.com/docker/for-mac/issues/2872#issuecomment-386560236 a Docker container is shutdown and a set of AF_VSOCK connections are simultaneously closed:
2018-05-02 15:21:58.519471-0700 localhost com.docker.driver.amd64-linux[19791]: proxy << POST /v1.37/containers/ce3d053af106a4f13790f82e721f88ece
8e85aa907f465afbc59d7f78cbc3676/wait?condition=next-exit
2018-05-02 15:21:58.520536-0700 localhost com.docker.driver.amd64-linux[19791]: proxy << POST /v1.37/containers/ce3d053af106a4f13790f82e721f88ece8e85aa907f465afbc59d7f78cbc3676/attach?stderr=1&stdin=1&stdout=1&stream=1
2018-05-02 15:21:58.559605-0700 localhost com.docker.driver.amd64-linux[19791]: proxy << POST /v1.25/containers/777ba957b4f05c6b359a64c4dfe54383e8d3e3e0374c23d82dfd69284fd1333b/attach?logs=0&stdout=1&stderr=1&stream=1
2018-05-02 15:21:58.560708-0700 localhost com.docker.driver.amd64-linux[19791]: Error forwarding raw stream from container: write unix s60->: write: broken pipe
2018-05-02 15:21:58.560819-0700 localhost com.docker.driver.amd64-linux[19791]: Error closing raw stream from container: close unix s60->: shutdown: socket is not connected
2018-05-02 15:21:58.560880-0700 localhost com.docker.driver.amd64-linux[19791]: proxy << POST /v1.25/containers/7374b83bcea635423f15aaaebe0e68c52de4f783393f3f03c5d4c04bfe792f58/attach?logs=0&stdout=1&stderr=1&stream=1
2018-05-02 15:21:58.560971-0700 localhost com.docker.driver.amd64-linux[19791]: Error forwarding raw stream from container: write unix s60->: write: broken pipe
2018-05-02 15:21:58.561036-0700 localhost com.docker.driver.amd64-linux[19791]: Error closing raw stream from container: close unix s60->: shutdown: socket is not connected
2018-05-02 15:21:58.561133-0700 localhost com.docker.driver.amd64-linux[19791]: proxy << POST /v1.25/containers/6b12ec8ec282ed688811f66dd5dfc06fb1baa8ab0de37ead0bb4aa401dc285c6/attach?logs=0&stdout=1&stderr=1&stream=1
2018-05-02 15:21:58.561180-0700 localhost com.docker.driver.amd64-linux[19791]: Error forwarding raw stream from container: write unix s60->: write: broken pipe
2018-05-02 15:21:58.561217-0700 localhost com.docker.driver.amd64-linux[19791]: Error closing raw stream from container: close unix s60->: shutdown: socket is not connected
2018-05-02 15:21:58.561276-0700 localhost com.docker.driver.amd64-linux[19791]: proxy << POST /v1.25/containers/1d3c0a5d337ac23ff97ae6b561cf40bde538181e348800e89f7dc4fa3edc6877/attach?logs=0&stdout=1&stderr=1&stream=1
2018-05-02 15:21:58.561370-0700 localhost com.docker.driver.amd64-linux[19791]: Error forwarding raw stream from container: write unix s60->: write: broken pipe
2018-05-02 15:21:58.561438-0700 localhost com.docker.driver.amd64-linux[19791]: Error closing raw stream from container: close unix s60->: shutdown: socket is not connected
2018-05-02 15:21:58.561588-0700 localhost com.docker.driver.amd64-linux[19791]: proxy << POST /v1.25/containers/764caeb2552f7eeda49f6292fc104e0173fa8952376f17c99395d86d59aa1ef5/attach?logs=0&stdout=1&stderr=1&stream=1
2018-05-02 15:21:58.561650-0700 localhost com.docker.driver.amd64-linux[19791]: proxy << POST /v1.25/containers/989e12d766b90e09fb66cf34c2761737eb8dee757530dcd8eccb78d890f8443e/attach?logs=0&stdout=1&stderr=1&stream=1
2018-05-02 15:21:58.561720-0700 localhost com.docker.driver.amd64-linux[19791]: Error forwarding raw stream from container: write unix s60->: write: broken pipe
2018-05-02 15:21:58.561750-0700 localhost com.docker.driver.amd64-linux[19791]: Error closing raw stream from container: close unix s60->: shutdown: socket is not connected
2018-05-02 15:21:58.561910-0700 localhost com.docker.driver.amd64-linux[19791]: proxy << POST /v1.25/containers/7367d897b4239a174ef2d7bc5df9e3a58cfddbb7780de092ae779286515a35d6/attach?logs=0&stdout=1&stderr=1&stream=1
2018-05-02 15:21:58.561919-0700 localhost com.docker.driver.amd64-linux[19791]: Error forwarding raw stream from container: write unix s60->: write: broken pipe
2018-05-02 15:21:58.561974-0700 localhost com.docker.driver.amd64-linux[19791]: Error closing raw stream from container: close unix s60->: shutdown: socket is not connected
2018-05-02 15:21:58.562102-0700 localhost com.docker.driver.amd64-linux[19791]: proxy << POST /v1.25/containers/b7c958118e20cd6aebbe20081dd0bdbec34f6b1844bb10e607635d64caf8c935/attach?logs=0&stdout=1&stderr=1&stream=1
2018-05-02 15:21:58.562167-0700 localhost com.docker.driver.amd64-linux[19791]: Error forwarding raw stream from container: write unix s60->: write: broken pipe
2018-05-02 15:21:58.562189-0700 localhost com.docker.driver.amd64-linux[19791]: Error closing raw stream from container: close unix s60->: shutdown: socket is not connected
2018-05-02 15:21:58.562222-0700 localhost com.docker.driver.amd64-linux[19791]: proxy << POST /v1.25/containers/b43381a52e98a6f057e3eb3ddb22b11d7ba7df43ad0a3eaac30b2358da8e7fb5/attach?logs=0&stdout=1&stderr=1&stream=1
2018-05-02 15:21:58.562407-0700 localhost com.docker.driver.amd64-linux[19791]: Error forwarding raw stream from container: write unix s60->: write: broken pipe
2018-05-02 15:21:58.562431-0700 localhost com.docker.driver.amd64-linux[19791]: Error closing raw stream from container: close unix s60->: shutdown: socket is not connected
2018-05-02 15:21:58.562466-0700 localhost com.docker.driver.amd64-linux[19791]: proxy << POST /v1.25/containers/2734b5133d06521b91cbf12ec933d4e6710388dc58ca8224f91256c2b26b3066/attach?logs=0&stdout=1&stderr=1&stream=1
2018-05-02 15:21:58.565070-0700 localhost com.docker.driver.amd64-linux[19791]: proxy << POST /v1.25/containers/0ebb0d164dcb574a1c9293f4b89f6c5980b28ea6b7afbbfa02f61c48aafc5d62/attach?logs=0&stdout=1&stderr=1&stream=1
2018-05-02 15:21:58.565306-0700 localhost com.docker.driver.amd64-linux[19791]: proxy << POST /v1.25/containers/1a10c59d4fd2ff59249fd950bac9c6e98cda4bd2e63e8418cbd9121c95e1e649/attach?logs=0&stdout=1&stderr=1&stream=1
2018-05-02 15:21:58.565350-0700 localhost com.docker.driver.amd64-linux[19791]: proxy << POST /v1.25/containers/23febad602e8110ae73f53887e8c5c8547cbcc41c788705216426403a4cf57f8/attach?logs=0&stdout=1&stderr=1&stream=1
2018-05-02 15:21:58.565941-0700 localhost com.docker.driver.amd64-linux[19791]: proxy << POST /v1.25/containers/25549937a3743c5771ef8c7354be68090ad1e057302f723ed7a1decd3fe25391/attach?logs=0&stdout=1&stderr=1&stream=1
2018-05-02 15:21:58.566170-0700 localhost com.docker.driver.amd64-linux[19791]: Error forwarding raw stream from container: write unix s60->: write: broken pipe
2018-05-02 15:21:58.566198-0700 localhost com.docker.driver.amd64-linux[19791]: Error closing raw stream from container: close unix s60->: shutdown: socket is not connected
2018-05-02 15:21:58.566319-0700 localhost com.docker.driver.amd64-linux[19791]: proxy << POST /v1.25/containers/f4288843126045dc510da935b41319d9c261ed9b4c2636885d0b240c5224204e/attach?logs=0&stdout=1&stderr=1&stream=1
2018-05-02 15:21:58.566712-0700 localhost com.docker.driver.amd64-linux[19791]: proxy << POST /v1.25/containers/0d618ab342e4225c4ddd8aeebc45dd1497d9775aa2a30b35ed9588e7af30ddfd/attach?logs=0&stdout=1&stderr=1&stream=1
This triggers
2018-05-02 15:22:03.400455-0700 localhost com.docker.driver.amd64-linux[19791]: Assertion failed: (!REPLY_RING_EMPTY(sc)), function send_response_common, file src/lib/pci_virtio_sock.c, line 934.
the code has:
/*
* We have just incremented reply_prod above but we hold the
* lock so the consumer cannot have caught us up. Hence for
* the ring to appear empty it must actually have just overflowed.
*/
assert(!REPLY_RING_EMPTY(sc));
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at src/lib/pci_virtio_sock.c around send_response_common line 934 and trace reply-ring handling during simultaneous AF_VSOCK connection closes. Reproduce the container shutdown sequence described in the report, then verify that the assertion no longer occurs during concurrent connection teardown.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100