Add infrastructure to build and verify generated Docker images in tests
- Dominant language
- C#
- Stars
- 6.3k
- Forks
- 991
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 196
Description
## Summary
Today, we generate Dockerfiles in tests such as Aspire.Hosting.Python.Tests, and assert the Dockerfile content as a baseline. However, this is not sufficient to ensure that the resulting container images build and function as intended. We need infrastructure that:
- Builds container images from generated Dockerfiles as part of the test process.
- Inspects image contents (docker inspect, etc.) to validate expected metadata (labels, entrypoint, environment variables, base image, layers).
- Optionally, runs containers for smoke tests and verifies they work for the intended scenarios.
## Proposal
- Extend current test coverage to:
- Build the Dockerfile into an image (e.g., using Docker.Build or equivalent CLI invocations from the test).
- Run `docker inspect ` and assert image metadata is correct.
- Optionally, run the container and verify behavioral aspects (correctly launches expected entrypoint, serves HTTP endpoint if applicable, etc.).
- Integrate this approach for Python app tests, but enable for any tests where Dockerfiles are generated (e.g. Aspire.Hosting.Containers.Tests, Aspire.Hosting.Python.Tests, etc.).
- Should fail CI if the image does not build or fails verification.
## References to Current Test Code
Relevant files for baseline Dockerfile verification and Dockerfile generation:
- [AddPythonAppTests.cs source](https://github.com/dotnet/aspire/blob/main/tests/Aspire.Hosting.Python.Tests/AddPythonAppTests.cs)
- [WithDockerfileTests.cs source](https://github.com/dotnet/aspire/blob/main/tests/Aspire.Hosting.Containers.Tests/WithDockerfileTests.cs)
- [DockerfileUtils.cs source](https://github.com/dotnet/aspire/blob/main/tests/Aspire.Hosting.Tests/Utils/DockerfileUtils.cs)
Samples (from semantic search):
```c#
// AddPythonAppTests.cs verifies Dockerfile contents for several scenarios, e.g.
Assert.Contains("FROM python:3.13-slim-bookworm", dockerfileContent);
// But only does content assertion, not build/run/inspect tests.
// WithDockerfileTests.cs demonstrates launching containers with Dockerfiles, but could be extended for more verification:
await app.StartAsync();
// ... test HTTP endpoint ...
await app.StopAsync();
```
## Desired outcome
- Dockerfiles generated in test runs are actually built and launched as containers.
- CI fails if the generated Docker images are invalid or non-functional.
- Enables future extension (e.g. SBOM, runtime checks, security scans, multi-arch builds).
Contributor guide
Assessment
This issue has not been assessed yet.