Skip to content

[Optimization Engine] Keep reservations visible when their SKU has no ISF ratio - #2310

Closed
Roland Krummenacher (RolandKrummenacher) wants to merge 1 commit into
devfrom
RolandKrummenacher/aoe-isf-leftouter
Closed

[Optimization Engine] Keep reservations visible when their SKU has no ISF ratio#2310
Roland Krummenacher (RolandKrummenacher) wants to merge 1 commit into
devfrom
RolandKrummenacher/aoe-isf-leftouter

Conversation

@RolandKrummenacher

Copy link
Copy Markdown
Collaborator

Summary

Fixes #2300. The reservation workbooks join the instance size flexibility open data with kind=inner. That data comes from the Azure Catalogs API, which only returns SKUs you can still purchase — so an active reservation on a size Azure no longer sells has no row to match and is dropped from the report entirely. Not shown at zero utilization: absent. The customer keeps paying for a reservation the tool stops mentioning.

Reservations run one or three years and routinely outlive the sellability of their size, so this is reachable in ordinary use. It is what the reporter of #2300 hit.

Change

All seven ISF joins in reservations-usage.json become leftouter, with the group and ratio falling back to the SKU itself:

| join kind=leftouter ( ISFGroups ) on $left.SKUName_s == $right.ArmSKUName
| extend ISFGroup = coalesce(ISFGroup, SKUName_s), Ratio = coalesce(Ratio, 1.0)

The coalesce is not optional. A plain leftouter is worse than today: Ratio and ISFGroup come back null, TotalReservedQuantity_s * Ratio becomes null, and summarize ... by ISFGroup collapses every unmatched reservation into a single null bucket with an empty utilization column.

Ratio = 1 is correct for these rows. Util7Days_s comes from Azure and already accounts for instance size flexibility, so the percentage is right as-is. The ratio only converts between group units, which is a no-op for a group of one.

reservations-potential.json and benefits-simulation.json keep kind=inner. They model what to buy, and there restricting to purchasable SKUs is correct.

Known limitation

Drilldowns fall back to coalesce(ISFGroup, VMSize) — the consumed size — while the overview falls back to SKUName_s, the reserved size. For a reservation on Standard_DS4_v2 running DS3_v2 underneath, those don't meet: which sizes belong to a retired group is exactly the information the ISF data no longer carries. So a drilldown on an unmatched reservation shows usage of the identical SKU only, not the group's flexibility.

Better than showing nothing, but not a full restoration. Recovering it would need the retired ratios, which is what #2300 originally proposed and #2308 explicitly stopped short of.

Why not fix the data instead

An earlier revision of #2308 backfilled 1,220 retired SKUs from the ratio file the API replaced. It was dropped: those rows would be frozen forever, since nothing can refresh or correct them once the source is gone, inside a dataset whose contract is a weekly refresh from the authoritative API. Fixing the consumer avoids carrying dead reference data.

Test plan

  • Open Reservations usage against a subscription holding a reservation on a retired size (Av2, Dv2, Dv3, Ev3, F, G, H, NC, NV, …) and confirm it now appears with a correct utilization percentage
  • Confirm reservations on current sizes are unchanged — same groups, same quantities, same percentages
  • Confirm the unused-cost view lists the same reservations as the utilization view

This is reasoned, not verified. No test in this repo executes the workbook KQL, and I have no Optimization Engine workspace with reservation data to run it against. The join semantics and the null behaviour that motivates the coalesce should be confirmed by someone who does before merging.

🤖 Generated with Claude Code

The reservation workbooks join the instance size flexibility open data with
kind=inner. The Catalogs API that feeds it only returns SKUs that can still
be purchased, so an active reservation on a retired size has no row to match
and is dropped from the report entirely -- not shown at zero, absent. The
customer keeps paying for a reservation the tool stops mentioning.

Reservations run one or three years and outlive the sellability of their
size, so this is reachable in normal use (#2300).

All seven ISF joins in reservations-usage.json become leftouter, with the
group and ratio falling back to the SKU itself:

    | extend ISFGroup = coalesce(ISFGroup, SKUName_s), Ratio = coalesce(Ratio, 1.0)

Without that fallback a plain leftouter is worse than the current behaviour:
Ratio and ISFGroup come back null, every unmatched reservation collapses into
one null bucket, and the utilization column reads empty.

Ratio 1 is correct for these rows. Util7Days_s comes from Azure and already
accounts for flexibility, so the percentage is right; the ratio only converts
between group units, which is a no-op for a single-SKU group.

reservations-potential.json and benefits-simulation.json keep kind=inner.
They model what to buy, where restricting to purchasable SKUs is correct.

Note the workbook KQL is not executed by any test in this repo, so this
change is reasoned rather than verified.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Some ISFGroups leftouter joins elsewhere in the same workbook still lack the required coalesce fallback, which can leave null group names or null/incorrect calculations for SKUs missing from the ISF data.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Updates the Optimization Engine Reservations usage workbook KQL so reservations remain visible even when their SKU no longer appears in the instance size flexibility (ISF) open-data feed (e.g., retired / no-longer-purchasable SKUs), preventing active reservations from disappearing from utilization reporting.

Changes:

  • Switch key ISF enrichment joins from kind=inner to kind=leftouter in reservations-usage.json.
  • Add coalesce(...) fallbacks for ISFGroup and Ratio (defaulting to the reserved SKU / 1.0) in the updated queries to avoid null-group bucketing and null math.
File summaries
File Description
src/optimization-engine/views/workbooks/reservations-usage.json Adjusts workbook KQL join semantics and introduces coalesce fallbacks so reservations with missing ISF rows are still included and computed correctly.
Review details
  • Files reviewed: 1/1 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

"content": {
"version": "KqlItem/1.0",
"query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://raw.githubusercontent.com/microsoft/finops-toolkit/dev/src/open-data/InstanceSizeFlexibility.csv\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nlet LinuxOnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption' and MeterSubCategory_s !endswith \"Windows\"\r\n| extend MeterSubCategory_s = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Linux'))\r\n| summarize LinuxUnitPrice=max(todouble(UnitPrice_s)) by LinuxMeterId=MeterID_g, MeterName_s, MeterSubCategory_s, MeterRegion_s, LinuxUnitOfMeasure=UnitOfMeasure_s;\r\nlet VMOnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption'\r\n| extend NonWindowsMeterSubcategory = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Windows'))\r\n| extend WindowsMeterSubCategory = MeterSubCategory_s\r\n| extend NonWindowsMeterSubcategory = substring(NonWindowsMeterSubcategory, 0, indexof(NonWindowsMeterSubcategory, ' Linux'))\r\n| summarize UnitPrice_s=max(todouble(UnitPrice_s)) by MeterID_g, MeterName_s, NonWindowsMeterSubcategory, WindowsMeterSubCategory, MeterRegion_s, UnitOfMeasure_s\r\n| join kind=leftouter ( LinuxOnDemandPriceSheet ) on MeterName_s, MeterRegion_s, $left.NonWindowsMeterSubcategory == $right.MeterSubCategory_s\r\n| extend PricesheetPrice = iif(isnotempty(LinuxUnitPrice), LinuxUnitPrice, UnitPrice_s)\r\n| extend PricesheetUnitOfMeasure = iif(isnotempty(LinuxUnitOfMeasure), LinuxUnitOfMeasure, UnitOfMeasure_s)\r\n| extend UnitHrs = toint(substring(PricesheetUnitOfMeasure, 0, indexof(PricesheetUnitOfMeasure, 'Hour')-1))\r\n| extend OnDemandUnitPrice = PricesheetPrice/UnitHrs\r\n| distinct MeterID_g, OnDemandUnitPrice;\r\nlet OnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s != 'Virtual Machines' and PriceType_s == 'Consumption'\r\n| summarize OnDemandUnitPrice = max(todouble(UnitPrice_s)) by MeterID_g\r\n| distinct MeterID_g, OnDemandUnitPrice\r\n| union (VMOnDemandPriceSheet)\r\n| summarize OnDemandUnitPrice=min(OnDemandUnitPrice) by MeterID_g;\r\nlet ReservationPricesheet = AzureOptimizationReservationsPriceV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| extend Term_s = iif(reservationTerm_s == '3 Years', 'P3Y', 'P1Y')\r\n| extend TermDivider = iif(reservationTerm_s == '3 Years', 3, 1)\r\n| extend ReservationPrice = todouble(replace_string(unitPrice_s,',','.'))/TermDivider/12/730\r\n| summarize arg_max(TimeGenerated, ReservationPrice) by Location_s=tolower(armRegionName_s), SkuName=tolower(armSkuName_s), Term_s;\r\nlet ReservationOnDemandMeters = AzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and TimeGenerated < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and isnotempty(ReservationName_s)\r\n| extend SkuName = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| distinct ReservationId_g, MeterId_g, SkuName;\r\nAzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and TimeGenerated < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and isnotempty(ReservationName_s)\r\n| where ReservationId_g in ({Reservation:value})\r\n| extend RINormalizationRatio = tostring(parse_json(AdditionalInfo_s).RINormalizationRatio)\r\n| extend UsedRIs = todouble(Quantity_s) * todouble(RINormalizationRatio) / 24\r\n| summarize UsedRIsDaily=round(sum(UsedRIs),2) by Date_s, ReservationId_g\r\n| summarize AvgRIsUsedDaily=round(avg(UsedRIsDaily),2) by ReservationId_g\r\n| join kind=rightouter (\r\n AzureOptimizationReservationsUsageV1_CL\r\n | where TimeGenerated > ago(1d)\r\n | where ReservationId_g in ({Reservation:value})\r\n | summarize arg_max(TimeGenerated, *) by ReservationId_g\r\n | where ProvisioningState_s in ('Succeeded','Expiring')\r\n | extend UsedQuantity = todouble(TotalReservedQuantity_s) * todouble(Util7Days_s) / 100\r\n | extend UsedQuantity30d = todouble(TotalReservedQuantity_s) * todouble(Util30Days_s) / 100\r\n | extend SKUName_s=tolower(SKUName_s)\r\n | project ReservationId_g, ReservationName_s=DisplayName_s, SKUName_s, Location_s, UsedQuantity, UsedQuantity30d, TotalReservedQuantity_s, Term_s, AppliedScopeType_s\r\n) on ReservationId_g\r\n| project ReservationId_g=ReservationId_g1, ReservationName_s, TotalReservedQuantity_s, SKUName_s, Location_s, AvgRIsUsedDaily=iif(isempty(AvgRIsUsedDaily), 0.0, AvgRIsUsedDaily), UsedQuantity, UsedQuantity30d, Term_s, AppliedScopeType_s\r\n| join kind=inner ( ISFGroups ) on $left.SKUName_s == $right.ArmSKUName\r\n| join kind=leftouter ( ReservationOnDemandMeters ) on ReservationId_g\r\n| summarize arg_max(MeterId_g, *) by ReservationId_g\r\n| join kind=leftouter ( ReservationPricesheet ) on SkuName and Location_s and Term_s\r\n| join kind=leftouter ( OnDemandPriceSheet ) on $left.MeterId_g == $right.MeterID_g\r\n| extend DiscountPercent = (1 - ReservationPrice/OnDemandUnitPrice) * 100\r\n| extend AvgRIsUsedInSmallestRatio = Ratio * AvgRIsUsedDaily\r\n| summarize TotalReservedQuantity_s=sum(todouble(TotalReservedQuantity_s)*Ratio), AvgRIsUsedDaily=sum(AvgRIsUsedInSmallestRatio), UsedQuantity=sum(UsedQuantity*Ratio), UsedQuantity30d=sum(UsedQuantity30d*Ratio), AvgDiscountPercent=avg(DiscountPercent) by ISFGroup, Location_s, Term_s, AppliedScopeType_s\r\n| extend Util7Days_s = UsedQuantity/TotalReservedQuantity_s*100, Util30Days_s = UsedQuantity30d/TotalReservedQuantity_s*100\r\n| extend AvgRIUsagePercentInSmallestRatio = round(AvgRIsUsedDaily / TotalReservedQuantity_s * 100, 1)\r\n| extend AvgDiscountPercent=iif(AvgDiscountPercent > 0.0, AvgDiscountPercent, 0.0)\r\n| extend SavingsMargin=round(todouble(Util7Days_s))-100.0+AvgDiscountPercent \r\n| project-away AvgRIUsagePercentInSmallestRatio, AvgRIsUsedDaily\r\n| project-reorder ISFGroup, Location_s, Term_s, AppliedScopeType_s, TotalReservedQuantity_s, Util7Days_s, UsedQuantity, Util30Days_s, UsedQuantity30d\r\n| order by Util7Days_s asc",
"query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://raw.githubusercontent.com/microsoft/finops-toolkit/dev/src/open-data/InstanceSizeFlexibility.csv\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nlet LinuxOnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption' and MeterSubCategory_s !endswith \"Windows\"\r\n| extend MeterSubCategory_s = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Linux'))\r\n| summarize LinuxUnitPrice=max(todouble(UnitPrice_s)) by LinuxMeterId=MeterID_g, MeterName_s, MeterSubCategory_s, MeterRegion_s, LinuxUnitOfMeasure=UnitOfMeasure_s;\r\nlet VMOnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption'\r\n| extend NonWindowsMeterSubcategory = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Windows'))\r\n| extend WindowsMeterSubCategory = MeterSubCategory_s\r\n| extend NonWindowsMeterSubcategory = substring(NonWindowsMeterSubcategory, 0, indexof(NonWindowsMeterSubcategory, ' Linux'))\r\n| summarize UnitPrice_s=max(todouble(UnitPrice_s)) by MeterID_g, MeterName_s, NonWindowsMeterSubcategory, WindowsMeterSubCategory, MeterRegion_s, UnitOfMeasure_s\r\n| join kind=leftouter ( LinuxOnDemandPriceSheet ) on MeterName_s, MeterRegion_s, $left.NonWindowsMeterSubcategory == $right.MeterSubCategory_s\r\n| extend PricesheetPrice = iif(isnotempty(LinuxUnitPrice), LinuxUnitPrice, UnitPrice_s)\r\n| extend PricesheetUnitOfMeasure = iif(isnotempty(LinuxUnitOfMeasure), LinuxUnitOfMeasure, UnitOfMeasure_s)\r\n| extend UnitHrs = toint(substring(PricesheetUnitOfMeasure, 0, indexof(PricesheetUnitOfMeasure, 'Hour')-1))\r\n| extend OnDemandUnitPrice = PricesheetPrice/UnitHrs\r\n| distinct MeterID_g, OnDemandUnitPrice;\r\nlet OnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s != 'Virtual Machines' and PriceType_s == 'Consumption'\r\n| summarize OnDemandUnitPrice = max(todouble(UnitPrice_s)) by MeterID_g\r\n| distinct MeterID_g, OnDemandUnitPrice\r\n| union (VMOnDemandPriceSheet)\r\n| summarize OnDemandUnitPrice=min(OnDemandUnitPrice) by MeterID_g;\r\nlet ReservationPricesheet = AzureOptimizationReservationsPriceV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| extend Term_s = iif(reservationTerm_s == '3 Years', 'P3Y', 'P1Y')\r\n| extend TermDivider = iif(reservationTerm_s == '3 Years', 3, 1)\r\n| extend ReservationPrice = todouble(replace_string(unitPrice_s,',','.'))/TermDivider/12/730\r\n| summarize arg_max(TimeGenerated, ReservationPrice) by Location_s=tolower(armRegionName_s), SkuName=tolower(armSkuName_s), Term_s;\r\nlet ReservationOnDemandMeters = AzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and TimeGenerated < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and isnotempty(ReservationName_s)\r\n| extend SkuName = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| distinct ReservationId_g, MeterId_g, SkuName;\r\nAzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and TimeGenerated < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and isnotempty(ReservationName_s)\r\n| where ReservationId_g in ({Reservation:value})\r\n| extend RINormalizationRatio = tostring(parse_json(AdditionalInfo_s).RINormalizationRatio)\r\n| extend UsedRIs = todouble(Quantity_s) * todouble(RINormalizationRatio) / 24\r\n| summarize UsedRIsDaily=round(sum(UsedRIs),2) by Date_s, ReservationId_g\r\n| summarize AvgRIsUsedDaily=round(avg(UsedRIsDaily),2) by ReservationId_g\r\n| join kind=rightouter (\r\n AzureOptimizationReservationsUsageV1_CL\r\n | where TimeGenerated > ago(1d)\r\n | where ReservationId_g in ({Reservation:value})\r\n | summarize arg_max(TimeGenerated, *) by ReservationId_g\r\n | where ProvisioningState_s in ('Succeeded','Expiring')\r\n | extend UsedQuantity = todouble(TotalReservedQuantity_s) * todouble(Util7Days_s) / 100\r\n | extend UsedQuantity30d = todouble(TotalReservedQuantity_s) * todouble(Util30Days_s) / 100\r\n | extend SKUName_s=tolower(SKUName_s)\r\n | project ReservationId_g, ReservationName_s=DisplayName_s, SKUName_s, Location_s, UsedQuantity, UsedQuantity30d, TotalReservedQuantity_s, Term_s, AppliedScopeType_s\r\n) on ReservationId_g\r\n| project ReservationId_g=ReservationId_g1, ReservationName_s, TotalReservedQuantity_s, SKUName_s, Location_s, AvgRIsUsedDaily=iif(isempty(AvgRIsUsedDaily), 0.0, AvgRIsUsedDaily), UsedQuantity, UsedQuantity30d, Term_s, AppliedScopeType_s\r\n| join kind=leftouter ( ISFGroups ) on $left.SKUName_s == $right.ArmSKUName\r\n| extend ISFGroup = coalesce(ISFGroup, SKUName_s), Ratio = coalesce(Ratio, 1.0)\r\n| join kind=leftouter ( ReservationOnDemandMeters ) on ReservationId_g\r\n| summarize arg_max(MeterId_g, *) by ReservationId_g\r\n| join kind=leftouter ( ReservationPricesheet ) on SkuName and Location_s and Term_s\r\n| join kind=leftouter ( OnDemandPriceSheet ) on $left.MeterId_g == $right.MeterID_g\r\n| extend DiscountPercent = (1 - ReservationPrice/OnDemandUnitPrice) * 100\r\n| extend AvgRIsUsedInSmallestRatio = Ratio * AvgRIsUsedDaily\r\n| summarize TotalReservedQuantity_s=sum(todouble(TotalReservedQuantity_s)*Ratio), AvgRIsUsedDaily=sum(AvgRIsUsedInSmallestRatio), UsedQuantity=sum(UsedQuantity*Ratio), UsedQuantity30d=sum(UsedQuantity30d*Ratio), AvgDiscountPercent=avg(DiscountPercent) by ISFGroup, Location_s, Term_s, AppliedScopeType_s\r\n| extend Util7Days_s = UsedQuantity/TotalReservedQuantity_s*100, Util30Days_s = UsedQuantity30d/TotalReservedQuantity_s*100\r\n| extend AvgRIUsagePercentInSmallestRatio = round(AvgRIsUsedDaily / TotalReservedQuantity_s * 100, 1)\r\n| extend AvgDiscountPercent=iif(AvgDiscountPercent > 0.0, AvgDiscountPercent, 0.0)\r\n| extend SavingsMargin=round(todouble(Util7Days_s))-100.0+AvgDiscountPercent \r\n| project-away AvgRIUsagePercentInSmallestRatio, AvgRIsUsedDaily\r\n| project-reorder ISFGroup, Location_s, Term_s, AppliedScopeType_s, TotalReservedQuantity_s, Util7Days_s, UsedQuantity, Util30Days_s, UsedQuantity30d\r\n| order by Util7Days_s asc",
@RolandKrummenacher

Copy link
Copy Markdown
Collaborator Author

Closing — folded into #2308 instead. Splitting this out was my call and the wrong one; the change belongs in the same PR as the rest of the ISF work. Same commit, same content, now on `RolandKrummenacher/isf-legacy-floor`.

@microsoft-github-policy-service

Copy link
Copy Markdown

@allcontributors

Copy link
Copy Markdown
Contributor

@microsoft-github-policy-service[bot]

Roland Krummenacher (@RolandKrummenacher) already contributed before to code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Needs: Review 👀 PR that is ready to be reviewed Tool: Optimization Engine Azure Optimization Engine

Projects

None yet

Development

Successfully merging this pull request may close these issues.

InstanceSizeFlexibility incomplete

5 participants