apache / apache/cloudstack

VM snapshot usage records are attributed to the wrong snapshot; older snapshots stop accruing usage entirely

Aperta Adatta ai principianti
#13,921 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
bug
Lingua principale
Java
Stelle
3.1k
Fork
1.4k
Merge medio
6g 19h
PR unite (30g)
32

Descrizione

### problem

When a VM has more than one VM snapshot, VMSnapshotUsageParser mis-attributes usage and silently drops usage for all but the newest snapshot. Two distinct defects combine:

1. The usage interval is labelled with the wrong snapshot ID. Duration, size and disk offering are taken from the previous event, but the snapshot ID is taken from the current one (line 88):

> createUsageRecord(UsageTypes.VM_SNAPSHOT, duration, previousCreated, createDate, account, volId, zoneId,
> previousEvent.getDiskOfferingId(), vmId,
> previousEvent.getSize(), // previous snapshot
> usageRec.getVmSnapshotId()); // current snapshot ← wrong label

2. Concurrent snapshots of the same volume overwrite each other. unprocessedUsage is keyed on VM + volume only (line 68), so a newer event evicts the older one and the older snapshot never reaches the closing loop:

> String key = vmId + ":" + volId;
> ...
> unprocessedUsage.put(key, usageRec); // line 96 — evicts the previous snapshot

**Net effect: the first snapshot's usage is credited to the second snapshot, and the first snapshot stops accruing usage even though it still exists and is Ready.**

Observed in a lab with two snapshots on one VM, neither deleted (both Ready, removed IS NULL):

DB entries from repro environment confirming the above

> mysql> select * from cloud.vm_snapshots where removed is null;
> +----+--------------------------------------+------------------------------+--------------+-------------+-------+------------+-----------+---------------------+------------------+-------+--------+---------+--------------+---------------------+---------------------+---------+
> | id | uuid | name | display_name | description | vm_id | account_id | domain_id | service_offering_id | vm_snapshot_type | state | parent | current | update_count | updated | created | removed |
> +----+--------------------------------------+------------------------------+--------------+-------------+-------+------------+-----------+---------------------+------------------+-------+--------+---------+--------------+---------------------+---------------------+---------+
> | 4 | 09bd5ac1-6a90-487f-afed-0798d5440107 | i-2-193-VM_VS_20260818150857 | Testsnap1 | Testsnap1 | 193 | 2 | 1 | 1 | Disk | Ready | NULL | 0 | 2 | 2026-08-18 15:08:59 | 2026-08-18 15:08:57 | NULL |
> | 5 | 71bcbec9-0760-4b34-8551-9fcc5a891027 | i-2-193-VM_VS_20260818154342 | Testsnap2 | Testsnap2 | 193 | 2 | 1 | 1 | Disk | Ready | 4 | 1 | 2 | 2026-08-18 15:43:45 | 2026-08-18 15:43:42 | NULL |
> +----+--------------------------------------+------------------------------+--------------+-------------+-------+------------+-----------+---------------------+------------------+-------+--------+---------+--------------+---------------------+---------------------+---------+
> 2 rows in set (0.00 sec)
>
> mysql> select * from cloud_usage where usage_type=25 and usage_id=5;
> +---------+---------+------------+-----------+-------------------------------------------------------------------------------+---------------+------------+--------------------+----------------+---------+-------------+-------------+----------+------+------------+------------+---------------------+---------------------+--------------+-----------+-----------+--------+------------------+-----------+-------+
> | id | zone_id | account_id | domain_id | description | usage_display | usage_type | raw_usage | vm_instance_id | vm_name | offering_id | template_id | usage_id | type | size | network_id | start_date | end_date | virtual_size | cpu_speed | cpu_cores | memory | quota_calculated | is_hidden | state |
> +---------+---------+------------+-----------+-------------------------------------------------------------------------------+---------------+------------+--------------------+----------------+---------+-------------+-------------+----------+------+------------+------------+---------------------+---------------------+--------------+-----------+-----------+--------+------------------+-----------+-------+
> | 1803640 | 1 | 2 | 1 | VMSnapshot Id: 5 Usage: VM Id: 193 Volume Id: 223 Size: (8.00 GB) 8589934592 | 0.579445 Hrs | 25 | 0.5794447064399719 | 193 | NULL | NULL | NULL | 5 | NULL | 8589934592 | NULL | 2026-08-18 15:08:59 | 2026-08-18 15:43:45 | NULL | NULL | NULL | NULL | 0 | 0 | NULL |
> | 1803641 | 1 | 2 | 1 | VMSnapshot Id: 5 Usage: VM Id: 193 Volume Id: 223 Size: (8.00 GB) 8589934592 | 8.270833 Hrs | 25 | 8.270833015441895 | 193 | NULL | NULL | NULL | 5 | NULL | 8589934592 | NULL | 2026-08-18 15:43:45 | 2026-08-18 23:59:59 | NULL | NULL | NULL | NULL | 0 | 0 | NULL |
> +---------+---------+------------+-----------+-------------------------------------------------------------------------------+---------------+------------+--------------------+----------------+---------+-------------+-------------+----------+------+------------+------------+---------------------+---------------------+--------------+-----------+-----------+--------+------------------+-----------+-------+
> 2 rows in set (2.45 sec)

Record 1803640 spans Testsnap1's creation to Testsnap2's creation, Testsnap1's usage interval , but carries usage_id = 5. **Testsnap1 (id 4) has no usage records at all.**

Below is the list of Usage expected vs Actual

Testsnap1 (15:08:59 (created time) --> 23:59:59) - Expected Usage: 8.85 hrs, Actual Usage: 0
Testsnap2 (15:43:45 (created time) --> 23:49:59) - Expected Usage: 8.27 Hrs, Actual Usage: 8.85 Hrs

**Note: The visible symptom depends on aggregation timing. If both snapshots are created within one aggregation window, the first snapshot gets no records at all. If the second is created in a later window, the first gets records only up to the last completed bucket, and the interval between that boundary and the second snapshot's creation is credited to the second snapshot.**

Snapshot chaining makes this the normal case rather than an edge case, note parent = 4 on Testsnap2.

### versions

Cloudstack Version: 4.22/4.22.1
Hypervisor: KVM
Primary Storage: NFS
Database: MySQL 8.4

Usage Settings:

> +-----------------------------------+-------+
> | name | value |
> +-----------------------------------+-------+
> | usage.aggregation.timezone | GMT |
> | usage.execution.timezone | GMT |
> | usage.sanity.check.interval | NULL |
> | usage.snapshot.virtualsize.select | false |
> | usage.stats.job.aggregation.range | 1440 |
> | usage.stats.job.exec.time | 00:15 |
> +-----------------------------------+-------+

### The steps to reproduce the bug

1. Deploy an instance on any hypervisor supporting VM snapshots.
2. Take a VM snapshot (Testsnap1).
3. Take a second VM snapshot on the same instance (Testsnap2). Do not delete either.
4. Wait for the usage aggregation job to run.
5. Query the usage records:

> SELECT usage_id, description, usage_display, start_date, end_date
> FROM cloud_usage.cloud_usage
> WHERE usage_type = 25 ORDER BY start_date;
>
> SELECT id, vm_id, volume_id, vm_snapshot_id, size, created, processed
> FROM cloud_usage.usage_vmsnapshot WHERE vm_id = ;

### What to do about it?

Two changes in VMSnapshotUsageParser:

1. Line 88 — label the record with the snapshot the interval belongs to:
> previousEvent.getSize(), previousEvent.getVmSnapshotId());

2. Line 68 — include the snapshot ID in the key so concurrent snapshots of one volume don't evict each other:
> String key = vmId + ":" + volId + ":" + usageRec.getVmSnapshotId();

Note the snapshot ID is already populated correctly on insert. createUsageVMSnapshot() reads it from usage_event_details and calls vsVO.setVmSnapshotId(...), so the data needed for both fixes is present; it's only ignored by the parser.

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Inizia individuando VMSnapshotUsageParser e ispeziona la costruzione della chiave intorno alla riga 68 e createUsageRecord intorno alla riga 88. Riproduci il caso con due snapshot e interroga i record di utilizzo descritti nell’issue. Il lavoro è completato quando gli intervalli di ciascuno snapshot riportano il proprio ID e gli snapshot più vecchi continuano ad accumulare utilizzo senza essere espulsi.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
java
Ambito
backend
Tipo di issue
Bug
Difficoltà
2/5
Tempo stimato
1-3 ore
Stato di attività
Attiva
Chiarezza
Specificata chiaramente
Idoneità per principianti
84/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.