Changes in files that should be captured by sync are triggering rebuild

### Expected behavior

Given an application with a nested source directory, a dockerfile that contains `COPY appname /app/appname`, and `build.artifacts[].image.sync.infer` containing `- 'appname'`, When I update a file in, e.g., `appname/apis`, the file is synced to the pod.

### Actual behavior

A full docker rebuild is triggered

### Information

- Skaffold version: 1.35.1
- Operating system: Ubuntu 20.04.3 LTS running on WSL2 on Windows 11
- Installed via: `curl -Lo skaffold https://storage.googleapis.com/skaffold/releases/v1.35.1/skaffold-linux-amd64 && chmod +x skaffold && sudo mv skaffold /usr/local/bin`
- Contents of skaffold.yaml:

```yaml
apiVersion: skaffold/v2beta26
kind: Config
build:
local:
push: false
artifacts:
- image: myorg/appname
context: .
sync:
infer:
- 'pyproject.toml'
- 'poetry.lock'
- 'db/*'
- 'appname/**/*.py'
- 'utils/*'
deploy:
helm:
releases:
- name: dev-app-name
namespace: some-namespace
artifactOverrides:
image: myorg/appname
imageStrategy:
helm: {}
chartPath: k8s/appname
valuesFiles:
- './k8s/appname/values.dev.yaml'
skipBuildDependencies: true

```

### Steps to reproduce the behavior