Custom Test infinite loop

### Actual behavior
Custom Test will run in an infinite loop of building -> testing.

### Information

- Skaffold version: v1.36.0
- Operating system: macOs
- Installed via: skaffold.dev
- Contents of skaffold.yaml:

```yaml
apiVersion: skaffold/v2beta27
kind: Config
metadata:
name: playlife-sso
build:
artifacts:
- image: playlife/sso
docker:
dockerfile: Dockerfile
test:
- image: playlife/sso
custom:
- command: pytest
deploy:
kubectl:
manifests:
- manifest.yaml
```

### Steps to reproduce the behavior

1. skaffold dev --no-prune=false --cache-artifacts=false --port-forward (or just skaffold dev)

this will result in an infinite loop. Skaffold will build the image, execute the test, than re-build the image and re-execute the test, forever.