Vault deployment with consul sidecar injection doesn't work due to service count

I've installed consul on a kubernetes cluster with helm and got the service mesh working.

Now I'm installing Vault with this helm chart.

These are the configuration parameters

```
server:
annotations: |
"consul.hashicorp.com/connect-inject": "true"
enabled: true
dev:
enabled: true
```

Pod doesn't start up and stops on Init and this is in the logs:

```
% kubectl logs vault-0 consul-connect-inject-init
2021-07-22T16:46:29.839Z [INFO] Unable to find registered services; retrying
2021-07-22T16:46:30.841Z [INFO] Unable to find registered services; retrying
2021-07-22T16:46:31.845Z [INFO] Unable to find registered services; retrying
2021-07-22T16:46:32.847Z [INFO] Unable to find registered services; retrying
2021-07-22T16:46:33.850Z [INFO] Unable to find registered services; retrying
2021-07-22T16:46:34.852Z [INFO] Unable to find registered services; retrying
2021-07-22T16:46:35.854Z [INFO] Unable to find registered services; retrying
2021-07-22T16:46:36.856Z [INFO] Unable to find registered services; retrying
2021-07-22T16:46:37.857Z [INFO] Unable to find registered services; retrying
2021-07-22T16:46:38.860Z [INFO] Unable to find registered services; retrying
2021-07-22T16:46:38.860Z [INFO] Check to ensure a Kubernetes service has been created for this application.
```

After 10 failures:

```
2021-07-22T15:51:59.299Z [ERROR] Timed out waiting for service registration: error="did not find correct number of services: 4"
```

Looking at consul-k8s source code at https://github.com/hashicorp/consul-k8s/blob/7d7ce5dc3f9ec74bf21f06b172fe3d97bddef62b/subcommand/connect-init/command.go

I can see:

```go
if len(serviceList) != 2 {
c.logger.Info("Unable to find registered services; retrying")
// Once every 10 times we're going to print this informational message to the pod logs so that
// it is not "lost" to the user at the end of the retries when the pod enters a CrashLoop.
if registrationRetryCount%10 == 0 {
c.logger.Info("Check to ensure a Kubernetes service has been created for this application.")
}
return fmt.Errorf("did not find correct number of services: %d", len(serviceList))
}
```

Going by the count of services it actually finds (4) vs the expected value of 2, there's an indication that things are double.

I therefor rendered out the helm templates and deployed each component individually one by one to identify the problem. It seems that this happens because the helm chart deploys two services to front Vault.. and I can't really understand why that's done at all. What's the point of the headless service?

These are functionally identical?

```yaml
---
# Source: vault/templates/server-service.yaml
# Service for Vault cluster
apiVersion: v1
kind: Service
metadata:
name: RELEASE-NAME-vault
namespace: default
labels:
helm.sh/chart: vault-0.13.0
app.kubernetes.io/name: vault
app.kubernetes.io/instance: RELEASE-NAME
app.kubernetes.io/managed-by: Helm
annotations:

spec:
publishNotReadyAddresses: true
ports:
- name: http
port: 8200
targetPort: 8200
- name: https-internal
port: 8201
targetPort: 8201
selector:
app.kubernetes.io/name: vault
app.kubernetes.io/instance: RELEASE-NAME
component: server

---
# Source: vault/templates/server-headless-service.yaml
# Service for Vault cluster
apiVersion: v1
kind: Service
metadata:
name: RELEASE-NAME-vault-internal
namespace: default
labels:
helm.sh/chart: vault-0.13.0
app.kubernetes.io/name: vault
app.kubernetes.io/instance: RELEASE-NAME
app.kubernetes.io/managed-by: Helm
annotations:

spec:
clusterIP: None
publishNotReadyAddresses: true
ports:
- name: "http"
port: 8200
targetPort: 8200
- name: https-internal
port: 8201
targetPort: 8201
selector:
app.kubernetes.io/name: vault
app.kubernetes.io/instance: RELEASE-NAME
component: server
```

I see no clear way to configure the helm chart to only render the server-service and not the server-headless-service.

Why is this headless service, how can we ignore it?

Start with templates/server-service.yaml and templates/server-headless-service.yaml to understand why both Services are rendered, then compare their behavior with subcommand/connect-init/command.go, which expects a specific service count. Reproduce the Vault deployment with Consul sidecar injection and verify how the chart can avoid the registration failure. Done means the documented configuration or chart change lets the deployment start successfully without breaking the required Vault service behavior.

Written by the indexing model from the issue text.