Scan `kubernetes/kubernetes` with `govulncheck`

**Background**: Today we have scanning implemented using [`snyk`](https://github.com/kubernetes/sig-security/blob/main/sig-security-tooling/vulnerability-mgmt/build-time-dependencies.md). It has worked quite well with addition of some smart optimization to reduce false positives.

Go team recently released https://go.dev/blog/govulncheck v1.0.0. It promises to provide prioritized vulnerability scanning for CVEs that affect the functions that the code is calling. This is promising in terms of having a really really low false positive since most vulnerability scan reports are in general [notoriously hard to wrangle](https://youtube.com/watch?v=2cvWmY4xvLU).

**Usecases**: We have three real workflows for injecting this type of scanning:

1. On `k/k` PRs: Create a diff between vulnerability scan report run on `master` branch and the one run on `HEAD` (current) branch. If the diff is non-zero, fail the pre-merge test. This can be run on symbol and module level depending on context of the PR
2. On `k/k` master periodically: Run every few hours to get a sense of vulnerability impact for tip of the contributions
3. On `k/k` release branches: Run every few hours to get a sense of vulnerability impact for release branches so `cherry-picks` can be created as needed

## Tasklist

- [x] https://github.com/kubernetes/sig-security/issues/99
- [x] https://github.com/kubernetes/sig-security/issues/100
- [ ] https://github.com/kubernetes/sig-security/issues/116

**How it works**

Example output on August 4 2023

```
demo$ govulncheck ./...
Using go1.20.6 and govulncheck@v1.0.0 with vulnerability data from https://vuln.go.dev (last modified 2023-08-02 20:33:39 +0000 UTC).

Scanning your code and 1968 packages across 204 dependent modules for known vulnerabilities...

Vulnerability #1: GO-2023-1987
Large RSA keys can cause high CPU usage in crypto/tls
More info: https://pkg.go.dev/vuln/GO-2023-1987
Standard library
Found in: crypto/tls@go1.20.6
Fixed in: crypto/tls@go1.21rc4
Example traces found:
#1: pkg/kubelet/server/server.go:234:24: server.ListenAndServePodResources calls grpc.Server.Serve, which eventually calls tls.Conn.Handshake
#2: pkg/proxy/healthcheck/proxier_health.go:179:24: healthcheck.proxierHealthServer.Run calls http.Server.Serve, which eventually calls tls.Conn.HandshakeContext
#3: test/e2e/framework/network/utils.go:1026:25: network.PokeHTTP calls io.ReadAll, which calls tls.Conn.Read
#4: cmd/kubeadm/app/preflight/checks.go:544:13: preflight.SystemVerificationCheck.Check calls bufio.Writer.Flush, which calls tls.Conn.Write
#5: test/utils/apiserver/testapiserver.go:73:59: apiserver.writeKubeConfigForWardleServerToKASConnection calls cert.GetServingCertificatesForURL, which eventually calls tls.Dial
#6: test/e2e/framework/websocket/websocket_util.go:61:29: websocket.OpenWebSocketForURL calls websocket.DialConfig, which eventually calls tls.DialWithDialer
#7: test/images/agnhost/inclusterclient/main.go:100:24: inclusterclient.debugRt.RoundTrip calls spdy.SpdyRoundTripper.RoundTrip, which eventually calls tls.Dialer.DialContext

=== Informational ===

Found 1 vulnerability in packages that you import, but there are no call
stacks leading to the use of this vulnerability. You may not need to
take any action. See https://pkg.go.dev/golang.org/x/vuln/cmd/govulncheck
for details.

Vulnerability #1: GO-2023-1988
Improper rendering of text nodes in golang.org/x/net/html
More info: https://pkg.go.dev/vuln/GO-2023-1988
Module: golang.org/x/net
Found in: golang.org/x/net@v0.12.0
Fixed in: golang.org/x/net@v0.13.0

Your code is affected by 1 vulnerability from the Go standard library.
```
Some more examples from @liggitt https://gist.github.com/liggitt/4674c7eb194738989183abf08feb333f

**Open Questions**:

These questions need to be discussed and reached a consensus on amongst K8s SRC, SIG Architecture, Release and Security

1. Do we make the scan results available in the CI output of the tests assuming the publicly available tool can not provide any new info than what is available in the CI script ?
1. Do we create two separates tests one for triaged but _not yet_ merged fixes & one for triaged and merged fixes (Triaged issues refer to CVE IDs which have been mentioned in GitHub Issues in k/k)
1. How often does the vulnerability database gets updated?
1. Does it have any differences in refresh cycles for vulnerabilities in go standard libraries (e.g. crypto/tls) v/s go packages in https://pkg.go.dev/ (e.g. `golang.org/x/net v0.12.0`)
1. Is there a definitive GOOS and GOARCH support we need to adhere to for scanning as the scanning can give different results based on the values for these env vars?

*Post-script*
In case there is anyone worried about the above output:

GO-2023-1987 fixed in:
* https://github.com/kubernetes/release/pull/3189
* https://github.com/kubernetes/k8s.io/pull/5679
* https://github.com/kubernetes/k8s.io/pull/5680
* https://github.com/kubernetes/k8s.io/pull/5681

GO-2023-1988 fixed in:
* https://github.com/kubernetes/kubernetes/pull/119725

Previous discussions:
* [v1.0.0 blog post](https://go.dev/blog/govulncheck),
* [slack](https://kubernetes.slack.com/archives/C019LFTGNQ3/p1690218753020059),
* [sig-security discussion on 2023-07-27](https://docs.google.com/document/d/1GgmmNYN88IZ2v2NBiO3gdU8Riomm0upge_XNVxEYXp0/edit#bookmark=id.zbo0brizgl9k)

/sig security architecture release
/committee security-response

Start by running the documented `govulncheck ./...` example against `kubernetes/kubernetes`, using the stated Go and govulncheck versions, and review the linked task issues 99, 100, and 116. Resolve the listed questions with the K8s SRC, SIG Architecture, Release, and Security groups, then define the agreed PR, periodic master, and release-branch scanning workflows and their expected CI results.

Written by the indexing model from the issue text.