Switch from apm-agent-go to OpenTelemetry

We currently use the internal apm-agent-go library for self-instrumentation of this service.
We should migrate to OpenTelemetry, so we can support sending this self instrumentation data in OTLP format.

We don't want to make a big bang migration either. So we should set up apmotel (See #11192) to be able to switch bits one after the other and only remove the apm agent once everything has been migrated.