microsoft / microsoft/finops-toolkit

FOCUS 1.4 phase 1: add FOCUS 1.3 ingestion (IngestionSetup_v1_3.kql)

Open
#2,124 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Tool: FinOps hubs Tool: FOCUS
Dominant language
PowerShell
Stars
603
Forks
248
Avg merge
7d 11h
Merged PRs (30d)
11

Description

Parent: #2120

Goal

Add FOCUS 1.3 ingestion support: a new IngestionSetup_v1_3.kql that defines _v1_3 raw transforms and final tables, plus the new columns added by FOCUS 1.3 in the IngestionSetup_RawTables.kql raw schema.

This phase covers Cost and Usage only. The new Contract Commitment dataset is its own phase (3).

FOCUS 1.3 Cost and Usage column additions

These are new in 1.3 and need handling in the 1_3 transform / final table:

Column Type Mandatory Notes
AllocatedMethodId String Conditional Data generator-calculated split cost allocation method ID
AllocatedMethodDetails JSON Recommended Allocation factors
AllocatedResourceId String Conditional Resource ID of allocated charge
AllocatedResourceName String Conditional Resource name of allocated charge
AllocatedTags JSON Conditional Tags of allocated charge
ContractApplied JSON Conditional Per-row contract commitment application
ServiceProviderName String Mandatory Replacement for deprecated ProviderName
HostProviderName String Mandatory (nullable) Underlying infrastructure provider

Deprecations to handle

  • ProviderName — deprecated in 1.3, removed in 1.4. Keep in v1_3 schema with same value as ServiceProviderName for backward compatibility.
  • PublisherName — deprecated in 1.3, removed in 1.4. Keep in v1_3 schema (legacy consumers).

Scope

File: src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/IngestionSetup_v1_3.kql (new)
  1. Copy IngestionSetup_v1_2.kql as the starting point.
  2. Rename functions/tables/policies: _v1_2_v1_3.
  3. Rename comments referencing FOCUS 1.2 → FOCUS 1.3.
  4. Update Costs_final_v1_3 table schema to include the 8 new columns.
  5. Update Costs_transform_v1_3() function to populate the 8 new columns from the raw table (mostly direct passthrough since Cost Management hasn't shipped 1.3 exports yet — null defaults are OK).
  6. Update raw-table update policy: the transform function should target _v1_3 final tables, and the v1_2 update policy should be deleted/redirected per the existing pattern.
File: IngestionSetup_RawTables.kql

Add new columns under each affected raw table (Costs_raw at minimum) with version annotation comments matching the existing convention (e.g., // FOCUS 1.3+).

Out of scope for this phase
  • New ContractCommitment_raw table and transform → phase 3
  • HubSetup_v1_3.kql with versioned hub functions → phase 2
  • Cost Management focuscost_1.3.json schema → out of scope (CM-owned, only updated when CM ships an official 1.3 export)
  • Open data FocusCost_1.3.json → phase 6

Branch

flanakin/focus14-phase1-ingestion-1.3 off flanakin/focus14. PR target: flanakin/focus14.

Done when

  • New file compiles cleanly (no KQL parse errors via the existing test pattern).
  • Raw tables include 1.3 column annotations.
  • Existing v1_2 ingestion still parses (no regression).
  • PR opened against flanakin/focus14.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing IngestionSetup_v1_2.kql with the new IngestionSetup_v1_3.kql requested under src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/. Then update IngestionSetup_RawTables.kql and follow the existing KQL test pattern to check parsing and v1_2 regression. Done means the new transforms, final tables, raw annotations, and update policies support the listed FOCUS 1.3 columns without parse errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure
Domain
data
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
66/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.