microsoft / microsoft/finops-toolkit
Harden Cost Management export retries for HTTP 429
Nobody has claimed this yet.
- Dominant language
- PowerShell
- Stars
- 603
- Forks
- 248
- Avg merge
- 7d 11h
- Merged PRs (30d)
- 11
Description
🐛 Problem
Start-FinOpsCostExport can fail instead of retrying when the Cost Management Run API returns HTTP 429. Invoke-Rest emits Write-Error for the response before the caller can reliably inspect Throttled, and the loop condition does not repeat a single-export run even when throttling is returned non-terminatingly.
The current unit test mocks Invoke-Rest returning a throttled response directly, so it does not cover the terminating error behavior observed in CI.
👣 Repro steps
- Run
New-FinOpsCostExport -ExecuteorStart-FinOpsCostExportin a context where PowerShell treatsWrite-Erroras terminating. - Have the Cost Management export Run API return HTTP 429 with a
Retry-Aftervalue. - Observe that the command exits with an error instead of waiting and retrying.
🤔 Expected
- HTTP 429 responses are returned to
Start-FinOpsCostExportin a form it can handle. - The command honors the server-provided
Retry-Aftervalue, with a safe fallback when absent or invalid. - Both single-export and multi-month runs retry the same request after throttling.
- Retries are bounded to prevent an infinite loop.
- Unit tests reproduce the real
Invoke-Rest/Write-Errorbehavior and cover single-export, multi-month, retry-delay, retry-limit, and eventual-success cases.
📷 Screenshots
N/A
🔧 Environment
- FinOps hub version: Development branch / PR #2187
- Billing account type: N/A
- Power BI report type: Storage-based deployment
- Cost Management export: FOCUS cost
ℹ️ Additional context
Observed in the Hubs (manual) check for PR #2187: deployment completed, then the initial export run failed with Too many requests. Please retry after 60 seconds. (Code: 429).
Failed run: https://github.com/microsoft/finops-toolkit/actions/runs/31678558903
The two immediately preceding PR deployment runs succeeded, indicating the service-side 429 was transient while the retry-handling failure is deterministic.
🙋♀️ Ask for the community
We could use your help:
- Please vote this issue up (👍) to prioritize it.
- Share other Cost Management throttling responses, especially variations in status codes or retry headers.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with Start-FinOpsCostExport and its Invoke-Rest call, then inspect the existing unit test that mocks a throttled response directly. Reproduce the terminating Write-Error behavior and cover single-export, multi-month, delay, retry-limit, and eventual-success cases. Done means HTTP 429 responses are retried with Retry-After handling and bounded retries.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, powershell
- Domain
- api, cloud, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100