microsoft / microsoft/finops-toolkit

Power BI storage reports: incremental refresh isn't detectable out of the box

Open
#2,314 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Tool: Power BI
Dominant language
PowerShell
Stars
603
Forks
248
Avg merge
7d 11h
Merged PRs (30d)
11

Description

🤖 [AI] Filed from findings raised in discussion #1799.

⚠️ Problem

Configuring Power BI Incremental refresh against the toolkit's storage-based reports (e.g. Cost summary) doesn't work — the Incremental Refresh dialog can't detect RangeStart/RangeEnd as assignable to any column. Reported independently by two users.

🛠️ Solution

Root cause, verified against src/power-bi/storage/Shared.Dataset/definition/expressions.tmdl, tables/Costs.tmdl, and src/power-bi/queries/ftk_NormalizeSchema.pq:

  1. RangeStart/RangeEnd are only used inside the shared ftk_Storage function to pre-filter which parquet files are read — they're never applied as a Table.SelectRows step directly on the top-level Costs query, which is what Power BI's Incremental Refresh dialog needs to detect the feature.
  2. ChargePeriodStart is cast to type date (not datetime) in ftk_NormalizeSchema.pq, which doesn't match the Date/Time-typed RangeStart/RangeEnd parameters.

Fix: cast ChargePeriodStart to type datetime, and add a Table.SelectRows step filtering on RangeStart/RangeEnd as the last step of the Costs query in the Shared.Dataset definition, so Power BI can auto-detect and apply incremental refresh.

ℹ️ Additional context

A manual workaround was posted in the discussion in the meantime. Also worth a docs update to docs-mslearn/toolkit/power-bi/setup.md once fixed.

Related: #1799 (discussion)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with src/power-bi/storage/Shared.Dataset/definition/expressions.tmdl, tables/Costs.tmdl, and src/power-bi/queries/ftk_NormalizeSchema.pq to trace the existing storage query and parameter types. Verify the change through Power BI's Incremental Refresh dialog, and review docs-mslearn/toolkit/power-bi/setup.md for the documented workaround. Done means Power BI detects RangeStart and RangeEnd for the Costs query and the setup documentation is current.

Written by the indexing model from the issue text.

Assessment

Domain
data-visualization
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.