Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions docs-mslearn/toolkit/hubs/configure-scopes.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,17 @@ If you can't grant permissions for your scope, you can create Cost Management ex
- We recommend exporting from either an **EA billing account** or **MCA billing profile** scope to access additional datasets, including price sheets and reservation recommendations.
- Price sheet exports are required to populate missing prices and costs.
- Reservation recommendation exports are used on the Rate optimization Reservation recommendations page.

> [!IMPORTANT]
> **Microsoft Customer Agreement (MCA) scope requirements**
>
> For MCA contracts, certain datasets are **only available at the billing profile level**, not at the billing account level:
> - Price sheet data
> - Reservation recommendations
> - Reservation details
>
> You must use the billing profile scope (`/providers/Microsoft.Billing/billingAccounts/###/billingProfiles/###`) for these exports. This is a Cost Management limitation.

- We recommend creating daily exports for each export type supported at your chosen billing scope:
- Enterprise Agreement billing account: FocusCosts, Pricesheet, ReservationTransactions, ReservationDetails, ReservationRecommendations
- Microsoft Customer Agreement billing profile: FocusCosts, Pricesheet, ReservationTransactions, ReservationDetails, ReservationRecommendations
Expand Down
2 changes: 1 addition & 1 deletion docs-mslearn/toolkit/hubs/data-processing.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ Transforms:
- Update `ProviderName` and `PublisherName` when not specified.
- Add `x_SourceName`, `x_SourceProvider`, `x_SourceType`, and `x_SourceVersion` to identify the original ingested dataset.
- Populate missing `ListCost`, `ListUnitPrice`, `ContractedCost`, and `ContractedUnitPrice` values based on the price sheet.
- This process requires prices to be exported before the cost. Prices being missing for the first day of the month if costs are ingested before pries are available for the month.
- This process requires prices to be exported before the cost. Prices being missing for the first day of the month if costs are ingested before prices are available for the month.
- Fix `ContractedCost` when set incorrectly due to a bug in Cost Management.
- Lowercase `ResourceName` and `x_ResourceGroupName` to address casing consistency issues that break grouping and filtering.
- Add `x_BillingAccountAgreement` based on the account type.
Expand Down
83 changes: 76 additions & 7 deletions docs-mslearn/toolkit/hubs/deploy.md
Original file line number Diff line number Diff line change
Expand Up @@ -253,6 +253,9 @@ You can ingest data from Microsoft Cost Management by creating exports manually

The following command is part of the FinOps toolkit PowerShell module. To install the module, see [Install the FinOps toolkit PowerShell module](../powershell/powershell-commands.md#install-the-module).

> [!NOTE]
> The examples below are shown in the recommended order for setting up exports. Always create price exports before cost exports to ensure accurate savings calculations.

```powershell
# Prices (required to populate missing prices)
# Only supported for EA billing accounts and MCA billing profiles
Expand Down Expand Up @@ -341,42 +344,108 @@ To ingest data from other data providers that support FOCUS, such as Amazon Web

FinOps hubs don't automatically backfill data. To populate historical data, run historical data exports from the original data provider, including any custom data pipelines used to publish data into the **ingestion** storage container.

> [!IMPORTANT]
> **Backfill order matters for accurate savings calculations**
>
> Always export price data before cost data when backfilling historical information. This ensures that:
> - Reserved Instance (RI) and savings plan benefits are correctly calculated in historical reports
> - Missing price information doesn't cause incomplete savings analysis
> - Data Explorer reports show accurate cost optimization opportunities across all time periods
>
> If you've already exported cost data before price data, rerun the **ingestion_ExecuteETL** pipeline for each affected month after price data is available.

For Microsoft Cost Management:

<!-- markdownlint-disable-next-line -->
### [Azure portal](#tab/azure-portal)

1. From the Azure portal, open [Cost Management](https://aka.ms/costmgmt).
2. Select the desired scope from the scope picker towards the top of the page.
> [!NOTE]
> **For Microsoft Customer Agreement (MCA) contracts**: Price sheet, reservation recommendations, and reservation details exports must be configured at the **billing profile level**, not the billing account level. This is a Cost Management limitation, not a FinOps hubs requirement.
3. In the menu on the left, select **Reporting + analytics** > **Exports**.
4. Select the desired export in the list of exports.
4. **Backfill in the correct order**: Start with price sheet exports, then proceed to other data types.
1. **First**: Export price sheet data (if available at your scope)
2. **Second**: Export cost and usage data
3. **Third**: Export reservation and recommendation data (if needed)
5. Select the desired export in the list of exports.
- Always export prices before costs to ensure they're available to populate missing prices in the cost and usage dataset.
- If costs are exported first, rerun the **ingestion_ExecuteETL** pipeline for the month's cost data to populate the missing prices.
5. Select **Export selected dates** and specify the desired month. Always export the full month.
6. Repeat step 5 for all desired months.
6. Select **Export selected dates** and specify the desired month. Always export the full month.
7. Repeat step 6 for all desired months.
- Cost Management only supports exporting up to the last 12 months from the Azure portal.
- Consider using PowerShell to export beyond the last 12 months.
7. Repeat steps 4-6 for each export.
8. Repeat steps 2-7 for each scope.
8. Repeat steps 5-7 for each export, maintaining the order specified in step 4.
9. Repeat steps 2-8 for each scope.

<!-- markdownlint-disable-next-line -->
### [PowerShell](#tab/powershell)

The following command is part of the FinOps toolkit PowerShell module. To install the module, see [Install the FinOps toolkit PowerShell module](../powershell/powershell-commands.md#install-the-module).

Run the following command for price exports first, then other datasets.
> [!IMPORTANT]
> **For Microsoft Customer Agreement (MCA) contracts**: Use the billing profile scope (`/providers/Microsoft.Billing/billingAccounts/###/billingProfiles/###`) rather than the billing account scope for price sheet, reservation, and recommendation exports.

**Export data in the following order to ensure accurate savings calculations:**

1. **First - Price sheet data** (if available at your scope):
```powershell
Start-FinOpsCostExport `
-Scope '/providers/Microsoft.Billing/billingAccounts/###/billingProfiles/###' `
-Name '{export-name}' `
-Name 'finops-hub-prices' `
-Dataset PriceSheet `
-Backfill 13 # or desired number of months
```

2. **Second - Cost and usage data**:
```powershell
Start-FinOpsCostExport `
-Scope '/providers/Microsoft.Billing/billingAccounts/###/billingProfiles/###' `
-Name 'finops-hub-costs' `
-Dataset FocusCost `
-Backfill 13 # or desired number of months
```

3. **Third - Reservation and recommendation data** (if needed):
```powershell
# Reservation recommendations
Start-FinOpsCostExport `
-Scope '/providers/Microsoft.Billing/billingAccounts/###/billingProfiles/###' `
-Name 'finops-hub-reservations' `
-Dataset ReservationRecommendations `
-Backfill 13 # or desired number of months

# Reservation details
Start-FinOpsCostExport `
-Scope '/providers/Microsoft.Billing/billingAccounts/###/billingProfiles/###' `
-Name 'finops-hub-reservation-details' `
-Dataset ReservationDetails `
-Backfill 13 # or desired number of months
```

For other parameters, see [Start-FinOpsCostExport](../powershell/cost/Start-FinOpsCostExport.md).

---

### Troubleshooting backfill order issues

If you've already exported cost data before price data and notice missing savings calculations (particularly for Reserved Instances and savings plans):

1. **Verify the issue**: Check your Data Explorer reports for missing or incomplete savings data in historical months.

2. **Export missing price data**: Run price sheet exports for the affected months using the methods above.

3. **Reprocess cost data**: After price data is available, manually trigger the **ingestion_ExecuteETL** pipeline for each affected month:
- Navigate to your FinOps hub Data Factory instance in the Azure portal
- Go to **Author & Monitor** > **Pipelines**
- Find and run the **ingestion_ExecuteETL** pipeline
- The pipeline will reprocess the cost data and apply the newly available price information

4. **Verify the fix**: Check your reports again to confirm that savings calculations now appear correctly for historical data.

> [!TIP]
> To avoid this issue in the future, always follow the recommended export order: prices first, then costs, then other datasets.

<br>

## Optional: Connect to Microsoft Fabric as a follower
Expand Down