bottlerocket-os / bottlerocket-os/twoliter
slow start-up when not at AWS Seattle offices (I am in Europe)
- Dominant language
- Rust
- Stars
- 34
- Forks
- 43
- Avg merge
- 11h 13m
- Merged PRs (30d)
- 16
Description
Hi! As mentioned during the community meeting, it takes me somewhere around 15-30 seconds before I get to the build stage. This is at a high level what happens:
### Combined Application and Network Timeline
| Time | Type | Event/Message |
|----------|------|--------------|
| 21:32:35 | App | Command started: `strace -t -f -e trace=network twoliter build variant metal-dev` |
| 21:32:35 | Net | Socket: Initial connection to ECR (44.216.203.94:443) |
| 21:32:35 | Log | INFO twoliter::project::lock - Resolving project references to check against lock file |
| 21:32:35 | Log | INFO twoliter::project::lock::image - Resolving dependency image dependency 'bottlerocket-core-kit-5.0.0@public.ecr.aws/bottlerocket/bottlerocket-core-kit:v5.0.0' |
| 21:32:35 | Net | Socket: Secondary ECR connection (44.216.203.94:443) |
| 21:32:36 | Net | Socket: AWS service connection (75.2.101.78:443) |
| 21:32:36 | Net | Socket: AWS service connection (99.83.145.10:443) |
| 21:32:37 | Net | Socket: Additional ECR connection (44.216.203.94:443) |
| 21:32:37 | Net | DNS: Failed NSCD lookup |
| 21:32:41 | Net | IPv6: Multiple connection attempts failed (ENETUNREACH) |
| 21:32:41 | Log | INFO twoliter::project::lock::image - Resolving dependency image dependency 'bottlerocket-kernel-kit-1.0.0@public.ecr.aws/bottlerocket/bottlerocket-kernel-kit:v1.0.0' |
| 21:32:42 | Net | Socket: Additional AWS service connection (99.83.145.10:443) |
| 21:32:43 | Net | Socket: AWS service connection (75.2.101.78:443) |
| 21:32:44 | Net | Socket: Additional AWS service connection (99.83.145.10:443) |
| 21:32:45 | Net | Socket: Additional ECR connection (44.216.203.94:443) |
| 21:32:46 | Net | Socket: AWS service connection (75.2.101.78:443) |
| 21:32:47 | Net | Socket: AWS service connection (99.83.145.10:443) |
| 21:32:48 | Net | Socket: AWS service connection (3.164.226.167:443) |
| 21:32:48 | Log | INFO twoliter::project::lock::image - Resolving dependency image dependency 'bottlerocket-sdk-0.50.0@public.ecr.aws/bottlerocket/bottlerocket-sdk:v0.50.0' |
| 21:32:48 | Net | IPv6: Multiple failed connection attempts |
| 21:32:49 | Net | Socket: Final ECR connection (44.216.203.94:443) |
| 21:32:50 | Net | Socket: Connection termination begins |
| 21:32:52 | Log | INFO cargo-make 0.37.23 |
| 21:32:52 | Log | WARN Unable to parse Cargo.toml via cargo-metadata, fallbacking |
| 21:32:53 | Log | INFO Build File: /home/mikn/devel/molnett/src/platform/base/bottlerocket/build/tools/Makefile.toml |
| 21:32:53 | Log | INFO Task: build |
| 21:32:53 | Log | INFO Profile: development |
| 21:32:53 | Log | INFO Running Task: setup |
| 21:32:53 | Log | INFO Running Task: setup-build |
| 21:32:53 | Log | INFO Running Task: fetch-sdk |
I had this strace output analysed by an AI and produced this log - but this was of a broken run as you can see by the fact that Cargo.toml was not parse:able.
Perhaps we could at least do all of the images in parallel?
Contributor guide
Research direction
Start by reproducing `twoliter build variant metal-dev` and tracing the project lock image-resolution stage described in the logs. Inspect the referenced `src/platform/base/bottlerocket/build/tools/Makefile.toml` and the Cargo.toml parsing path. Done should mean dependency images resolve without the reported startup delay and the build reaches its normal stages successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, rust
- Domain
- build-system, cloud, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100