Filtering doesn't seem to work

Summary:

I can't get metrics filtering to work. My goal is to filter on multiple different user metadata labels, but right now I can't get a single one to work.

Version: v0.12.0

Config (abridged, in a k8s deployment manifest):

 spec:
      containers:
      - args:
        - --monitoring.metrics-type-prefixes=cloudsql.googleapis.com/database
        - --monitoring.filters='cloudsql.googleapis.com/database:metadata.user_labels."my-label"="my-value"'

Given that config I'd expect the filter to be included in the debug log here but that doesn't seem to be the case. The log statements generally only include the metric.type filter:

ts=2023-01-24T06:45:40.496Z caller=monitoring_collector.go:290 level=debug msg="retrieving Google Stackdriver Monitoring metrics with filter" filter="metric.type=\"cloudsql.googleapis.com/database/postgresql/insights/aggregate/execution_time\""

and the metrics scraped include metrics from databases not tagged with my-label=my-value.

I'm new to GCP, and assume this is just something silly stemming from my lack of familiarity with Stackdriver. Would love any pointers.

Start in collectors/monitoring_collector.go around the debug logging referenced in the issue, then reproduce the v0.12.0 configuration from the Kubernetes manifest against a metric with the stated user label. Check how the metric type and metadata filter are assembled and logged. Done means the filter is sent to Google Cloud Monitoring and scraped metrics exclude databases without my-label=my-value.