Clarify k8s cluster mode deployment / Helm chart use

Related Discord thread: https://discordapp.com/channels/908281611840282624/915644818988863558/1263200596039827617
Note: The following assumes I've understood the necessary Quickwit configuration, which might not be the case!

Clarifying cluster mode service endpoints (with examples)

Previously I'd just been deploying Quickwit locally using docker compose which doesn't require any knowledge about the microservices created in cluster mode, and the endpoint to use for exporting traces/logs and searching is pretty obvious (i.e, http://quickwit:7821 for the exporter endpoint, http://127.0.0.1:7281 for exporting its own telemetry, http://quickwit:7280/api/v1 for the Grafana datasource URL). I believe this is already mentioned somewhere in the existing documentation.

When moving to using Helm, however, the configuration is a bit less intuitive and relies some understanding/guessing which services are responsible for what. While I initially assumed that using the headless service endpoint would be appropriate (quickwit-headless.<namespace>.svc.cluster.local:<port>), this turned out not to be the case and instead required exporting traces to quickwit-indexer and using quickwit-searcher as the datasource. I don't believe this is explicitly mentioned anywhere in the documentation, and would be helpful to call out in the guide for using the Helm chart with a standard opentelemetry-collector setup, for example:
otel-collector-config.yml

...

    exporters:
      otlp/quickwit:
        endpoint: quickwit-indexer.<quickwit-namespace>.svc.cluster.local:7281
        tls:
          insecure: true

...

examples of typical Helm chart values (example of lowering retention here would also be nice):
values.yaml

environment:
  QW_ENABLE_OPENTELEMETRY_OTLP_EXPORTER: "true" # not sure if this is needed anymore or enabled by default now?
  OTEL_EXPORTER_OTLP_ENDPOINT: "http://quickwit-indexer:7281"

and referencing the appropriate endpoint in any Grafana datasources:

apiVersion: 1

datasources:
- name: Quickwit Traces
  uid: quickwit-traces
  type: quickwit-quickwit-datasource
  url: http://quickwit-searcher.<quickwit-namespace>.svc.cluster.local:7280/api/v1
  editable: false
  jsonData:
    index: 'otel-traces-v0_7'
  ...

What do you all think?
👍 I would love to see it!
🚀 I would love to help!

Thank you for your request!

Start with the existing Helm chart guide and the standard opentelemetry-collector setup described in the issue, checking how quickwit-indexer and quickwit-searcher are used. Done means the documentation clearly maps exporter and Grafana datasource endpoints to the relevant services and includes the suggested configuration examples, including the optional retention setting.