Failed to start container, context deadline exceeded
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 694
- Forks
- 304
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 28
Description
I am using docker compose to start about 15 containers and occasionally one will fail to start and if I wait long enough it will fail. The only resolution I have found is to restart the docker service and try again and keep doing this until all containers start.
Here are my version:
Docker Engine: 19.03.8
Docker Compose: 1.25.4, build 8d51620a
Windows 10: 1809 (17763.1098)
Docker Base Image: mcr.microsoft.com/dotnet/framework/aspnet:4.8-20200114-windowsservercore-ltsc2019
Here are the entries from the Docker log file with the container id in question
[15:11:19.098][WindowsDaemon ][Info ] debug: hcsshim::CreateScratchLayer [path=C:\ProgramData\Docker\windowsfilter\4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb]
[15:11:19.198][WindowsDaemon ][Info ] debug: hcsshim::CreateScratchLayer - succeeded [path=C:\ProgramData\Docker\windowsfilter\4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb]
[15:11:19.207][WindowsDaemon ][Info ] debug: Calling GET /v1.38/containers/4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb/json
[15:11:19.208][WindowsDaemon ][Info ] debug: form data: {"Container":"4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb"}
[15:11:19.221][WindowsDaemon ][Info ] debug: form data: {"Container":"4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb","EndpointConfig":{"Aliases":["4c6e788f78e1","mytestapp"]}}
[15:11:19.226][WindowsDaemon ][Info ] debug: Calling GET /v1.40/containers/4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb/json
[15:11:19.228][WindowsDaemon ][Info ] debug: Calling POST /v1.38/containers/4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb/start
[15:11:19.289][WindowsDaemon ][Info ] debug: 7dadaf6eba6c06f8bab8402da9481355901ef23a7bd99ad8cba13beebabb2acc (10cb5d7).addSvcRecords(4c6e788f78e1, 172.20.74.178, <nil>, false) updateSvcRecord
sid:7dadaf6eba6c06f8bab8402da9481355901ef23a7bd99ad8cba13beebabb2acc
[15:11:19.292][WindowsDaemon ][Info ] debug: 7dadaf6eba6c06f8bab8402da9481355901ef23a7bd99ad8cba13beebabb2acc (10cb5d7).addSvcRecords(4c6e788f78e1, 172.20.74.178, <nil>, false) updateSvcRecord
sid:7dadaf6eba6c06f8bab8402da9481355901ef23a7bd99ad8cba13beebabb2acc
[15:11:19.297][WindowsDaemon ][Info ] debug: EnableService 4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb START
[15:11:19.297][WindowsDaemon ][Info ] debug: EnableService 4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb DONE
[15:11:19.302][WindowsDaemon ][Info ] debug: Generated spec: {"ociVersion":"1.0.1-dev","process":{"user":{"uid":0,"gid":0},"args":["powershell.exe",".\\Run-Scripts\\Start.ps1"],"cwd":"C:\\"},"root":{"path":""},"hostname":"4c6e788f78e1","windows":{"layerFolders":["C:\\ProgramDa
ta\\Docker\\windowsfilter\\f31ed5a6f9b50b61720745df68f6d32d1cfc1881ba7d8bb15ac712d15dbbcb9f","C:\\ProgramData\\Docker\\windowsfilter\\76c3ed71c8ccc6a0b989459fd80c38a8133581749ab108a9f96c9fe2d46218d1","C:\\ProgramData\\Docker\\windowsfilter\\6c30
7fa90e9c983a64618140ab5587d173be4d2b53b08d111d70d5c2a404a2fa","C:\\ProgramData\\Docker\\windowsfilter\\b7eab68db26e22afb1e073c2169ac0edc8a38345a8170b3696c15e6b2d950c65","C:\\ProgramData\\Docker\\windowsfilter\\65374e7cd0407b38e900f574760e629b767
2f795b55f0115bf06a20431fa15eb","C:\\ProgramData\\Docker\\windowsfilter\\8dc3e10f3d8c5e4f5c5876152d2a388c40a4f6dfb9e9f03a72fd5114f2c874ed","C:\\ProgramData\\Docker\\windowsfilter\\813b37fd300bb8e72925f1764dba8c2f92a226827263e8bdea4773008a1c9b01",
"C:\\ProgramData\\Docker\\windowsfilter\\b9c0f1f1bb328b8f97f06da47a8266a235da6f6046c5f9f1199694e8186312e9","C:\\ProgramData\\Docker\\windowsfilter\\7f2a55e015671a473d46549b7c13b6803563e6fd82c0cbdaad98cac218480bda","C:\\ProgramData\\Docker\\windo
wsfilter\\77d29cb7574377eac26f6ccba10ab888ce7849364eab918999bf2452d702a11e","C:\\ProgramData\\Docker\\windowsfilter\\7549620862c1834754bbf41e196b8b57c7f12130df4ca6854f3f1e11a1a60cb5","C:\\ProgramData\\Docker\\windowsfilter\\be78f515328c91a932ae5
aaee0de1929ee0997e7c1609ddda45f216ec07f26d2","C:\\ProgramData\\Docker\\windowsfilter\\2d98c30fef9cb307b99545f2f08d3e2e3d9e32775eeb0fef2b657a966ad07050","C:\\ProgramData\\Docker\\windowsfilter\\c0039ec7557688567e27c64d2ed2606c2fb847bfd0e706f8387a
77bb9184d17b","C:\\ProgramData\\Docker\\windowsfilter\\d4b3cf6117537636a90c83abd5c5a5dd26c46ac76a0a45912b71f0109c8a8227","C:\\ProgramData\\Docker\\windowsfilter\\ec840a04a0e1f58198d10d5e981e2aca940693a9b4f2df26e997c6f3c65e9827","C:\\ProgramData\
\Docker\\windowsfilter\\81d92ae8ad6e22a6f38afb0f3c9f801102b3a8d8b6f7867df9f9739b6713ae67","C:\\ProgramData\\Docker\\windowsfilter\\79bc5444dcacd6ccd54da387e685b6090bfeeb0fcc2d6504ffc104c7358302b5","C:\\ProgramData\\Docker\\windowsfilter\\7649e2f
7b6153819913ca208d45ecbb5e2b86dae7e2634bf268236e4a8fab5c4","C:\\ProgramData\\Docker\\windowsfilter\\888f9d5513c68ada00574617455810382a2fbfdafdfe2b004ba259e01a537fb9","C:\\ProgramData\\Docker\\windowsfilter\\16bf8835d64e68cc580ebc0de69df5759aa578
7b44fbafdd059e345344a8237f","C:\\ProgramData\\Docker\\windowsfilter\\292bdc7e0bde0ba5666de26eed91c0e9470112c628a7f69b5e249c7f9144c85e","C:\\ProgramData\\Docker\\windowsfilter\\d77f299447c12926df0d84b5eee70e1323869a9db6b80e2005840471c5ceaca4","C:
\\ProgramData\\Docker\\windowsfilter\\2bcf1679e3fbede5b6a2eb90701197e8c99622cde4917292007581da2911f2cd","C:\\ProgramData\\Docker\\windowsfilter\\0173ef3fe228a75c1d0f2de3b0c3c29ab17bfc36c948dc229ab7bff4f61fb9e2","C:\\ProgramData\\Docker\\windowsf
ilter\\9327fe84aedb5d9966a6ea2c7e0fd4afa5eb6d2f1907a770708a6360be737b83","C:\\ProgramData\\Docker\\windowsfilter\\eb2d78875b98375fda58621301fb29bd3ce00dacdb67113e3c456f751c31534e","C:\\ProgramData\\Docker\\windowsfilter\\50bd8fefa1394c5101090be6
b8d4fae8755dde092a8381b2adc1fbb9961026e3","C:\\ProgramData\\Docker\\windowsfilter\\1bedcd5acc13f69c2640fd058c3c03f30a2150c8d31b414cbd2221c6f5fd24e6","C:\\ProgramData\\Docker\\windowsfilter\\4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b
8f1ef66cb"],"ignoreFlushesDuringBoot":true,"hyperv":{},"network":{"endpointList":["D6B4B161-685B-481B-9FA1-FA3C78076AD6"],"allowUnqualifiedDNSQuery":true}}}
[15:11:19.364][WindowsDaemon ][Info ] debug: Calling GET /v1.40/containers/4c6e788f78e1/json
[15:11:19.365][WindowsDaemon ][Info ] debug: HCS notification [notification-type=SystemCreateCompleted system-id=4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb]
[15:11:19.365][WindowsDaemon ][Info ] debug: starting container [container=4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb module=libcontainerd namespace=moby]
[15:11:19.623][WindowsDaemon ][Error ] failed to start container [module=libcontainerd namespace=moby error=container 4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb encountered an error during hcsshim::System::Start:
context deadline exceeded]
[15:11:19.885][WindowsDaemon ][Info ] debug: failed to terminate container [error=container 4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb encountered an error during hcsshim::System::Terminate: context deadline exceeded
container=4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb module=libcontainerd namespace=moby]
[15:11:19.885][WindowsDaemon ][Error ] failed to cleanup after a failed Start [module=libcontainerd namespace=moby error=container 4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb encountered an error during
hcsshim::System::Terminate: context deadline exceeded]
[15:11:19.887][WindowsDaemon ][Info ] debug: 7dadaf6eba6c06f8bab8402da9481355901ef23a7bd99ad8cba13beebabb2acc (10cb5d7).deleteSvcRecords(4c6e788f78e1, 172.20.74.178, <nil>, false) updateSvcRecord
sid:7dadaf6eba6c06f8bab8402da9481355901ef23a7bd99ad8cba13beebabb2acc
[15:11:19.887][WindowsDaemon ][Error ] 4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb cleanup: failed to delete container from containerd: no such container
[15:11:19.887][WindowsDaemon ][Error ] Handler for POST /v1.38/containers/4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb/start returned error: container 4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb
encountered an error during hcsshim::System::Start: context deadline exceeded
[15:11:20.957][WindowsDaemon ][Info ] debug: HCS notification [notification-type=SystemStartCompleted system-id=4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb]
[15:11:20.957][WindowsDaemon ][Info ] debug: HCS notification [notification-type=SystemExited system-id=4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb]
[15:12:27.239][WindowsDaemon ][Info ] debug: hcsshim::CreateScratchLayer [path=C:\ProgramData\Docker\windowsfilter\4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb]
[15:12:27.339][WindowsDaemon ][Info ] debug: hcsshim::CreateScratchLayer - succeeded [path=C:\ProgramData\Docker\windowsfilter\4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb]
[15:12:27.346][WindowsDaemon ][Info ] debug: Calling GET /v1.38/containers/4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb/json
[15:12:27.348][WindowsDaemon ][Info ] debug: form data: {"Container":"4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb"}
[15:12:27.359][WindowsDaemon ][Info ] debug: form data: {"Container":"4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb","EndpointConfig":{"Aliases":["4c6e788f78e1","mytestapp"]}}
[15:12:27.363][WindowsDaemon ][Info ] debug: Calling GET /v1.40/containers/4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb/json
[15:12:27.365][WindowsDaemon ][Info ] debug: Calling POST /v1.38/containers/4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb/start
[15:12:27.410][WindowsDaemon ][Info ] debug: 7dadaf6eba6c06f8bab8402da9481355901ef23a7bd99ad8cba13beebabb2acc (10cb5d7).addSvcRecords(4c6e788f78e1, 172.20.74.178, <nil>, false) updateSvcRecord
sid:7dadaf6eba6c06f8bab8402da9481355901ef23a7bd99ad8cba13beebabb2acc
[15:12:27.415][WindowsDaemon ][Info ] debug: 7dadaf6eba6c06f8bab8402da9481355901ef23a7bd99ad8cba13beebabb2acc (10cb5d7).addSvcRecords(4c6e788f78e1, 172.20.74.178, <nil>, false) updateSvcRecord
sid:7dadaf6eba6c06f8bab8402da9481355901ef23a7bd99ad8cba13beebabb2acc
[15:12:27.419][WindowsDaemon ][Info ] debug: EnableService 4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb START
[15:12:27.419][WindowsDaemon ][Info ] debug: EnableService 4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb DONE
[15:12:27.426][WindowsDaemon ][Info ] debug: Generated spec: {"ociVersion":"1.0.1-dev","process":{"user":{"uid":0,"gid":0},"args":["powershell.exe",".\\Run-Scripts\\Start.ps1"],"cwd":"C:\\"},"root":{"path":""},"hostname":"4c6e788f78e1","windows":{"layerFolders":["C:\\ProgramDa
ta\\Docker\\windowsfilter\\f31ed5a6f9b50b61720745df68f6d32d1cfc1881ba7d8bb15ac712d15dbbcb9f","C:\\ProgramData\\Docker\\windowsfilter\\76c3ed71c8ccc6a0b989459fd80c38a8133581749ab108a9f96c9fe2d46218d1","C:\\ProgramData\\Docker\\windowsfilter\\6c30
7fa90e9c983a64618140ab5587d173be4d2b53b08d111d70d5c2a404a2fa","C:\\ProgramData\\Docker\\windowsfilter\\b7eab68db26e22afb1e073c2169ac0edc8a38345a8170b3696c15e6b2d950c65","C:\\ProgramData\\Docker\\windowsfilter\\65374e7cd0407b38e900f574760e629b767
2f795b55f0115bf06a20431fa15eb","C:\\ProgramData\\Docker\\windowsfilter\\8dc3e10f3d8c5e4f5c5876152d2a388c40a4f6dfb9e9f03a72fd5114f2c874ed","C:\\ProgramData\\Docker\\windowsfilter\\813b37fd300bb8e72925f1764dba8c2f92a226827263e8bdea4773008a1c9b01",
"C:\\ProgramData\\Docker\\windowsfilter\\b9c0f1f1bb328b8f97f06da47a8266a235da6f6046c5f9f1199694e8186312e9","C:\\ProgramData\\Docker\\windowsfilter\\7f2a55e015671a473d46549b7c13b6803563e6fd82c0cbdaad98cac218480bda","C:\\ProgramData\\Docker\\windo
wsfilter\\77d29cb7574377eac26f6ccba10ab888ce7849364eab918999bf2452d702a11e","C:\\ProgramData\\Docker\\windowsfilter\\7549620862c1834754bbf41e196b8b57c7f12130df4ca6854f3f1e11a1a60cb5","C:\\ProgramData\\Docker\\windowsfilter\\be78f515328c91a932ae5
aaee0de1929ee0997e7c1609ddda45f216ec07f26d2","C:\\ProgramData\\Docker\\windowsfilter\\2d98c30fef9cb307b99545f2f08d3e2e3d9e32775eeb0fef2b657a966ad07050","C:\\ProgramData\\Docker\\windowsfilter\\c0039ec7557688567e27c64d2ed2606c2fb847bfd0e706f8387a
77bb9184d17b","C:\\ProgramData\\Docker\\windowsfilter\\d4b3cf6117537636a90c83abd5c5a5dd26c46ac76a0a45912b71f0109c8a8227","C:\\ProgramData\\Docker\\windowsfilter\\ec840a04a0e1f58198d10d5e981e2aca940693a9b4f2df26e997c6f3c65e9827","C:\\ProgramData\
\Docker\\windowsfilter\\81d92ae8ad6e22a6f38afb0f3c9f801102b3a8d8b6f7867df9f9739b6713ae67","C:\\ProgramData\\Docker\\windowsfilter\\79bc5444dcacd6ccd54da387e685b6090bfeeb0fcc2d6504ffc104c7358302b5","C:\\ProgramData\\Docker\\windowsfilter\\7649e2f
7b6153819913ca208d45ecbb5e2b86dae7e2634bf268236e4a8fab5c4","C:\\ProgramData\\Docker\\windowsfilter\\888f9d5513c68ada00574617455810382a2fbfdafdfe2b004ba259e01a537fb9","C:\\ProgramData\\Docker\\windowsfilter\\16bf8835d64e68cc580ebc0de69df5759aa578
7b44fbafdd059e345344a8237f","C:\\ProgramData\\Docker\\windowsfilter\\292bdc7e0bde0ba5666de26eed91c0e9470112c628a7f69b5e249c7f9144c85e","C:\\ProgramData\\Docker\\windowsfilter\\d77f299447c12926df0d84b5eee70e1323869a9db6b80e2005840471c5ceaca4","C:
\\ProgramData\\Docker\\windowsfilter\\2bcf1679e3fbede5b6a2eb90701197e8c99622cde4917292007581da2911f2cd","C:\\ProgramData\\Docker\\windowsfilter\\0173ef3fe228a75c1d0f2de3b0c3c29ab17bfc36c948dc229ab7bff4f61fb9e2","C:\\ProgramData\\Docker\\windowsf
ilter\\9327fe84aedb5d9966a6ea2c7e0fd4afa5eb6d2f1907a770708a6360be737b83","C:\\ProgramData\\Docker\\windowsfilter\\eb2d78875b98375fda58621301fb29bd3ce00dacdb67113e3c456f751c31534e","C:\\ProgramData\\Docker\\windowsfilter\\50bd8fefa1394c5101090be6
b8d4fae8755dde092a8381b2adc1fbb9961026e3","C:\\ProgramData\\Docker\\windowsfilter\\1bedcd5acc13f69c2640fd058c3c03f30a2150c8d31b414cbd2221c6f5fd24e6","C:\\ProgramData\\Docker\\windowsfilter\\4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b
8f1ef66cb"],"ignoreFlushesDuringBoot":true,"hyperv":{},"network":{"endpointList":["D6B4B161-685B-481B-9FA1-FA3C78076AD6"],"allowUnqualifiedDNSQuery":true}}}
[15:12:27.520][WindowsDaemon ][Info ] debug: Calling GET /v1.40/containers/4c6e788f78e1/json
[15:12:27.523][WindowsDaemon ][Info ] debug: HCS notification [notification-type=SystemCreateCompleted system-id=4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb]
[15:12:27.523][WindowsDaemon ][Info ] debug: starting container [container=4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb module=libcontainerd namespace=moby]
[15:12:27.819][WindowsDaemon ][Error ] failed to start container [module=libcontainerd namespace=moby error=container 4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb encountered an error during hcsshim::System::Start:
context deadline exceeded]
[15:12:28.041][WindowsDaemon ][Info ] debug: failed to terminate container [error=container 4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb encountered an error during hcsshim::System::Terminate: context deadline exceeded
container=4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb module=libcontainerd namespace=moby]
[15:12:28.041][WindowsDaemon ][Error ] failed to cleanup after a failed Start [module=libcontainerd namespace=moby error=container 4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb encountered an error during
hcsshim::System::Terminate: context deadline exceeded]
[15:12:28.043][WindowsDaemon ][Info ] debug: 7dadaf6eba6c06f8bab8402da9481355901ef23a7bd99ad8cba13beebabb2acc (10cb5d7).deleteSvcRecords(4c6e788f78e1, 172.20.74.178, <nil>, false) updateSvcRecord
sid:7dadaf6eba6c06f8bab8402da9481355901ef23a7bd99ad8cba13beebabb2acc
[15:12:28.044][WindowsDaemon ][Error ] 4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb cleanup: failed to delete container from containerd: no such container
[15:12:28.044][WindowsDaemon ][Error ] Handler for POST /v1.38/containers/4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb/start returned error: container 4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb
encountered an error during hcsshim::System::Start: context deadline exceeded
[15:12:29.002][WindowsDaemon ][Info ] debug: HCS notification [notification-type=SystemStartCompleted system-id=4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb]
[15:12:29.002][WindowsDaemon ][Info ] debug: HCS notification [notification-type=SystemExited system-id=4c6e788f78e11c8eaea5d325fa79472a9b514ed6f8773e17b267c9b8f1ef66cb]
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The report identifies hcsshim::System::Start and System::Terminate as the failing paths, but names no source file or test. Begin by reproducing the timeout with Docker Compose on Windows using the listed versions, then trace those paths and the accompanying HCS notifications. Done means identifying the cause, adding regression coverage where appropriate, and preventing the failed start and cleanup timeouts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, docker-compose, go
- Domain
- devops, operating-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 22/100