ko builder fails on empty ldflags

Playing around with github.com/gardener/gardener to try to reproduce reported problems. This repository uses a helper script to set an environment variable LD_FLAGS which is used by the Ko builds:
```yaml
build:
artifacts:
- image: eu.gcr.io/gardener-project/gardener/apiserver
ko:
main: ./cmd/gardener-apiserver
ldflags:
- "{{.LD_FLAGS}}"
dependencies:
paths:
```

### Expected behavior

When `LD_FLAGS` is unset, the build should skip.

### Actual behavior

```
Building [eu.gcr.io/gardener-project/gardener/seed-admission-controller]...
Target platforms: [linux/arm64]
Using base gcr.io/distroless/static:nonroot@sha256:ed05c7a5d67d6beebeba19c6b9082a5513d5f9c3e22a883b9dc73ec39ba41c04 for github.com/gardener/gardener/cmd/gardener-seed-admission-controller
Using build config eu.gcr.io/gardener-project/gardener/seed-admission-controller for github.com/gardener/gardener/cmd/gardener-seed-admission-controller
Building github.com/gardener/gardener/cmd/gardener-seed-admission-controller for linux/arm64
Unexpected error running "go build": exit status 2
invalid value "" for flag -ldflags: missing = in =
usage: go build [-o output] [build flags] [packages]
Run 'go help build' for details.
Building [eu.gcr.io/gardener-project/gardener/scheduler]...
```

### Information

- Skaffold version: HEAD
- Operating system: macOS arm64

```yaml

```

### Steps to reproduce the behavior

1. github.com/gardener/gardener
2. `skaffold debug`