[stress] Get an OpenTelemetry collector working so we can start reporting trace/metrics to Azure Monitor in all languages

# Background

Getting OpenTelemetry metrics to destination like Azure Monitor requires an exporter.

Java and C# can send OpenTelemetry traces/metrics directly to AppInsights via their OpenTelemetry packages. Other languages require an external process, called a collector, which understands OTLP (OpenTelemetry Protocol).
> NOTE: There is a note on the Monitor page that they're adding an OTLP endpoint but there's no release date information on there. So this could become less complicated for us at some indeterminate point in the future. [link](https://learn.microsoft.com/en-us/azure/azure-monitor/app/opentelemetry-overview#telemetry-routing:~:text=Azure%20Monitor%20is%20developing%20an%20agent%20and%20ingestion%20endpoint%20that%20supports%20Open%20Telemetry%20Protocol%20(OTLP)%2C%20providing%20a%20path%20for%20any%20OpenTelemetry%2Dsupported%20programming%20language%20beyond%20our%20supported%20languages%20to%20use%20to%20Azure%20Monitor.)

# Work

- [ ] Build the otel collector, publish an image. I'm unclear on how well the collector scales, so it might be that every pod needs to have this running as a sidecar or perhaps we can just run a few instances as a K8S Service and just let everyone send to a known endpoint.

Here's the Dockerfile:

```Dockerfile
FROM golang:1.21
# this is the automated builder app - you can use it to create a "distribution"
# of the opentelemetry collector that includes `contrib` components (ie, not part of the
# base opentelemetry libraries but still part of the project)
RUN go install go.opentelemetry.io/collector/cmd/builder@latest

WORKDIR /root
# the .yaml tells us what to include in the build. It's defined just below this section
ADD ./otelcol-builder.yaml otelcol-builder.yaml
RUN builder --config=otelcol-builder.yaml
```

Here's the `otelcol-builder.yaml`, which includes the Azure Monitor exporter and enables the OTLP receiver:

```yaml
# https://github.com/open-telemetry/opentelemetry-collector/tree/main/cmd/builder
dist:
name: otelcol-custom
description: Local OpenTelemetry Collector binary
output_path: /tmp/dist
exporters:
# More exporters at: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main
- gomod: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/azuremonitorexporter v0.83.0
- gomod: go.opentelemetry.io/collector/exporter/loggingexporter v0.83.0

receivers:
- gomod: go.opentelemetry.io/collector/receiver/otlpreceiver v0.83.0

processors:
- gomod: go.opentelemetry.io/collector/processor/batchprocessor v0.83.0
```

- [ ] Deploy the collector with the proper config, either as a sidecar or within the same pod. We need some code to inject our APPINSIGHTS_INSTRUMENTATIONKEY into the YAML file so it can load it from there (or maybe it already implicitly understands that variable). I haven't gotten to try this part out:

```yaml
receivers:
otlp:
protocols:
grpc: # no need to set address if the default one is used

exporters:
azuremonitor:
instrumentation_key:
logging:
verbosity: detailed #not sure, but maybe we need to set it

processors:
batch:
# as a cherry on the top, it might be worth looking into
# https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/processor/k8sattributesprocessor/README.md
# or resourcedetection processor - they can stamp k8s ids on metrics/logs/traces

service:
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [azuremonitor]
metrics:
receivers: [otlp]
processors: [batch]
exporters: [azuremonitor]
logs:
receivers: [otlp]
processors: [batch]
exporters: [azuremonitor, logging]
```

- [ ] Use the correct exporter for your language that understands it has to forward to a collector and hook it up to your traceprovider:
- Go: https://opentelemetry.io/docs/instrumentation/go/exporters/#otlp-endpoint
- JS: https://opentelemetry.io/docs/instrumentation/js/exporters/#otlp-endpoint
- Python: https://opentelemetry.io/docs/instrumentation/python/exporters/#otlp-endpoint-or-collector
- C++: https://opentelemetry.io/docs/instrumentation/cpp/exporters/#otlp-http-exporter
- NOTE: Java and C# already have built-in exporters and don't require a collector.