microsoft / microsoft/aspire

Add infrastructure to build and verify generated Docker images in tests

Open
#12,898 2 comments 0 reactions 0 assignees View on GitHub
area-app-testing area-engineering-systems docker-compose testing ☑️
Dominant language
C#
Stars
6.3k
Forks
991
Avg merge
2d 15h
Merged PRs (30d)
196

Description

## Summary
Today, we generate Dockerfiles in tests such as Aspire.Hosting.Python.Tests, and assert the Dockerfile content as a baseline. However, this is not sufficient to ensure that the resulting container images build and function as intended. We need infrastructure that:

- Builds container images from generated Dockerfiles as part of the test process.
- Inspects image contents (docker inspect, etc.) to validate expected metadata (labels, entrypoint, environment variables, base image, layers).
- Optionally, runs containers for smoke tests and verifies they work for the intended scenarios.

## Proposal
- Extend current test coverage to:
- Build the Dockerfile into an image (e.g., using Docker.Build or equivalent CLI invocations from the test).
- Run `docker inspect ` and assert image metadata is correct.
- Optionally, run the container and verify behavioral aspects (correctly launches expected entrypoint, serves HTTP endpoint if applicable, etc.).
- Integrate this approach for Python app tests, but enable for any tests where Dockerfiles are generated (e.g. Aspire.Hosting.Containers.Tests, Aspire.Hosting.Python.Tests, etc.).
- Should fail CI if the image does not build or fails verification.

## References to Current Test Code
Relevant files for baseline Dockerfile verification and Dockerfile generation:
- [AddPythonAppTests.cs source](https://github.com/dotnet/aspire/blob/main/tests/Aspire.Hosting.Python.Tests/AddPythonAppTests.cs)
- [WithDockerfileTests.cs source](https://github.com/dotnet/aspire/blob/main/tests/Aspire.Hosting.Containers.Tests/WithDockerfileTests.cs)
- [DockerfileUtils.cs source](https://github.com/dotnet/aspire/blob/main/tests/Aspire.Hosting.Tests/Utils/DockerfileUtils.cs)

Samples (from semantic search):
```c#
// AddPythonAppTests.cs verifies Dockerfile contents for several scenarios, e.g.
Assert.Contains("FROM python:3.13-slim-bookworm", dockerfileContent);

// But only does content assertion, not build/run/inspect tests.

// WithDockerfileTests.cs demonstrates launching containers with Dockerfiles, but could be extended for more verification:
await app.StartAsync();
// ... test HTTP endpoint ...
await app.StopAsync();
```

## Desired outcome
- Dockerfiles generated in test runs are actually built and launched as containers.
- CI fails if the generated Docker images are invalid or non-functional.
- Enables future extension (e.g. SBOM, runtime checks, security scans, multi-arch builds).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.