microsoft / microsoft/finops-toolkit
[Hubs] Price ingestion creates extra rows
Open
Nobody has claimed this yet.
Skill: KQL
Tool: FinOps hubs
- Dominant language
- PowerShell
- Stars
- 603
- Forks
- 248
- Avg merge
- 7d 11h
- Merged PRs (30d)
- 11
Description
🐛 Problem
Price ingestion is adding more rows than are coming from Cost Management exports.
👣 Repro steps
- Set the raw ingestion to 1 or more days
- Drop all extents for the Prices_raw table to clear it, if there is data
- Ingest one month of data
- Compare the row count in Prices_raw and Prices_final_v1_0
Prices_raw | count // = 1,021,818
Prices_transform_v1_0 | count // = 1,264,006
🤔 Expected
Row counts should be the same
🙋♀️ Ask for the community
We could use your help:
- Please vote this issue up (👍) to prioritize it.
- Leave comments to help us solidify the vision.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the price ingestion path that populates Prices_raw and Prices_transform_v1_0. Reproduce the one-month ingestion with raw ingestion set to at least one day, then compare both Kusto row counts. Done means the transformed count matches the source count without extra rows.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure
- Domain
- cloud, data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100