InstanceSizeFlexibility incomplete

[JacobStruiksma]

🐛 Problem

src/open-data/InstanceSizeFlexibility.csv is incomplete. Multiple instance size flexibility groups that exist in Microsoft's published ISF ratio blob (isfratioblob.csv) are missing entirely from the open data file, even though those SKUs are still deployed and covered by active reservations.

Examples of groups that are completely absent:

InstanceSizeFlexibilityGroup	ArmSkuName	Ratio
Av2 Series	Standard_A1_v2	1
Av2 Series	Standard_A2_v2	2.1
Av2 Series	Standard_A4_v2	4.4
Av2 Series	Standard_A8_v2	9.2
D Series High Memory	Standard_D11	1
D Series High Memory	Standard_D12	2
D Series High Memory	Standard_D13	4
D Series High Memory	Standard_D14	8
DS Series High Memory	Standard_DS11	1
DS Series High Memory	Standard_DS12	2
DS Series High Memory	Standard_DS13	4
DS Series High Memory	Standard_DS14	8

👣 Repro steps

compare src/open-data/InstanceSizeFlexibility.csv with https://aka.ms/isf and filter out the ArmSkuName starts with "Standard_" and maybe filter out the promo's

🤔 Expected

A complete list for src/open-data/InstanceSizeFlexibility.csv with all the vm's that are currently still being used

📷 Screenshots

🔧 Environment

• FinOps hub version: 15
• Billing account type: MCA (Microsoft Customer Agreement)
• Power BI report type: ADX (Azure Data Explorer)
• Cost Management export: (e.g., FOCUS 1.2-preview, FOCUS 1.0, actual, amortized)

ℹ️ Additional context

TODO: Add any other context about the problem here. Remove if not applicable.

isf-missing.csv

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) to prioritize it.
2. Leave comments to help us solidify the vision.

[microsoft-github-policy-service]

Thank you for submitting this issue! We try to respond within 3 business days. In the meantime:

• 📖 Check the troubleshooting guide for common issues
• 💬 Ask questions in Discussions
• 📅 Join our biweekly office hours for live help from the team

[RolandKrummenacher]

Investigated this, and the fix moved from the data to the consumer. Summarizing so the reasoning is on the record.

The premise was challenged and it holds only halfway. The argument that ISF matters only for new reservations, so only purchasable SKUs need ratios, is right for the purchase-oriented views — reservations-potential.json, benefits-simulation.json, and the Power BI reservation recommendations all restrict to what you can buy, correctly.

It does not hold for reservations-usage.json, which analyses reservations you already own:

AzureOptimizationReservationsUsageV1_CL
| where ProvisioningState_s in ("Succeeded","Expiring") and todatetime(ExpiryDate_s) > now()
...
| join kind=inner ( ISFGroups ) on $left.SKUName_s == $right.ArmSKUName

An active, unexpired reservation whose SKU the Catalogs API no longer returns is dropped by that inner join — not shown at zero, absent from the report. Reservations run one or three years and outlive the sellability of their size, which is what you hit.

Fixed in #2310 by making the ISF joins in that workbook leftouter with the group and ratio falling back to the SKU itself. Your reservations appear again, with utilization percentages that are correct because Azure computes them and they already account for flexibility.

What we decided against, and why it matters for how you use the file: backfilling the retired ratios into src/open-data/InstanceSizeFlexibility.csv. Those rows can never be refreshed or corrected now that the source blob is gone, and the dataset is otherwise regenerated weekly from the API. So the published CSV stays scoped to purchasable SKUs.

If you consume the CSV directly rather than through the Optimization Engine workbooks, say so here — that is a different problem from the one #2310 fixes, and worth tracking separately.

Separately, #2308 fixes three real defects the investigation turned up in the generator: a hardcoded 26-region sweep that was missing SKUs, culture-dependent ratio parsing that could publish 2,1, and a SKU published with a ratio of 0 that the benefits simulation divides by.