devcontainer ci image cache is never working

In dc github action I have

```
- name: Dev Container Build and Run Action
uses: devcontainers/ci@v0.3
with:
imageName: ghcr.io/${{ github.repository_owner }}/devcontainers
imageTag: latest
runCmd: |
mvn --version
whoami
ls -lash /home/vscode
echo "Dev Container can start up! Let's push!"
```

this builds and pushes the DC.

In deploy github action I have:

```
- name: Dev Container Build and Run Action
uses: devcontainers/ci@v0.3
with:
imageName: ghcr.io/${{ github.repository_owner }}/devcontainers
cacheFrom: ghcr.io/${{ github.repository_owner }}/devcontainers
# platform: linux/amd64,linux/arm64
push: never
runCmd: |
whoami
ls -lash /home/vscode/
< then a lot more things>
```

This always rebuilds the container instead of loading it from cache.

What am I doing wrong?