Deduplication failes for groups with different values for non-grouped labels

Description

If I have an alert grouping that contains labels with different values, messages from prometheus will not be deduplicated, i.e. old messages that have been confirmed by the receiver will be send again.

As example szenario, I will use a simple metric that tracks a team's rank on a scoreboard. An alert will constantly fire the current rank as a label.

Expected result

Every time the rank changes, one resolved alert with the old rank and one firing alert with the new rank is send to the webhook.

Actual result

Every time prometheus sends some resolved alerts, all resolved alerts send by prometheus and the current firing alert will be forwarded to the webhook despite no new firing/resolved alert was send by prometheus.

Minimum example

Here is a minimum docker-compose example for reproducing the bug. The issue is best observed at the end of the sample when the rank stays constant and prometheus resends resolved messages for previous values. If I group by rank, alertmanager will successfully deduplicate the alerts.

services:
  node:
    image: node:alpine
    configs:
      - source: node-app-js
        target: /app/app.js
    working_dir: /app
    entrypoint: /bin/sh
    command:
      - "-c"
      - "npm install express && node app.js"
    network_mode: host

  alertmanager:
    image: quay.io/prometheus/alertmanager:main
    configs:
      - source: alertmanager-config
        target: /etc/alertmanager/alertmanager.yml
    volumes:
      - alertmanager-storage:/alertmanager
    command:
      - "--config.file=/etc/alertmanager/alertmanager.yml"
      - "--storage.path=/alertmanager"
      - "--cluster.listen-address="
      - "--log.level=debug"
    depends_on:
      - node
    network_mode: host

  prometheus:
    image: quay.io/prometheus/prometheus:v3.6.0
    configs:
      - source: prometheus-config
        target: /etc/prometheus/prometheus.yml
      - source: prometheus-rules
        target: /etc/prometheus/rules.yml
    volumes:
      - prometheus-storage:/prometheus
    command:
      - "--config.file=/etc/prometheus/prometheus.yml"
      - "--storage.tsdb.path=/prometheus"
      - "--log.level=debug"
    depends_on:
      - alertmanager
      - node
    network_mode: host

volumes:
  prometheus-storage: {}
  alertmanager-storage: {}

configs:
  node-app-js:
    content: |
      const express = require("express")
      const app = express()
      app.use(express.json())

      let index = 0
      const rank = [28, 28, 28, 28, 28, 28, 26, 59, 73, 83, 87, 84, 83, 81, 79, 78, 76, 74, 70, 64, 55, 57, 58, 53, 45, 5, 4, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4]

      app.post("/alertmanager", (req, res) => {
          console.log(req.body.alerts)
          res.sendStatus(200)
      })

      app.get("/metrics", (req, res) => {
          res.contentType("text/plain")
          res.send(`rank $${rank[index++]}`)
      })

      app.listen(9099)
      console.log("listening on port 9099")

  alertmanager-config:
    content: |
      route:
        group_by: ['alertname']
        group_wait: 2s          # time to wait for first alert in group
        group_interval: 10s     # time to wait for additional alerts in group
        repeat_interval: 30m    # time to send the alert again
        receiver: 'web.hook'
      receivers:
        - name: 'web.hook'
          webhook_configs:
            - url: 'http://localhost:9099/alertmanager'

  prometheus-config:
    content: |
      global:
        scrape_interval: 10s
        evaluation_interval: 10s
      rule_files:
        - /etc/prometheus/rules.yml
      scrape_configs:
        - job_name: scoreboard
          static_configs:
           - targets: ['localhost:9099']
      alerting:
        alertmanagers:
          - static_configs:
              - targets: ['localhost:9093']

  prometheus-rules:
    content: |
      groups:
        - name: scoreboard
          rules:
            - alert: RankChange
              expr: rank
              labels:
                type: info
                rank: '{{ $$value }}'

Start by running the minimum docker-compose reproduction and tracing Alertmanager's grouping and deduplication behavior for changing rank labels. Done means rank changes produce one resolved alert for the old rank and one firing alert for the new rank, while repeated resolved messages are not forwarded again.