AWS Savings Plans Inventory Pipeline

[JGrubb]

Summary

Create a DLT pipeline to ingest AWS Savings Plans inventory data to provide commitment tracking alongside billing data.

Background

Savings Plans are AWS's preferred commitment mechanism (replacing Reserved Instances). Understanding active commitments is crucial for cost optimization analysis when combined with actual usage data from CUR.

Scope

Data Sources: Savings Plans inventory (current state)
Granularity: Daily pipeline runs (commitments change infrequently)
Schema: Separate schema from aws_billing (e.g., aws_commitments)
Relationships: Link to CUR data via account IDs and resource identifiers

AWS APIs to Integrate

Primary APIs (boto3.client('savingsplans'))

• describe_savings_plans() - Active Savings Plans
• describe_savings_plans_offerings() - Available offerings
• describe_savings_plans_offering_rates() - Pricing rates

Implementation Plan

Phase 1: Core Pipeline

• Create aws_savings_plans_source() DLT source
• Implement basic inventory collection using API response payloads
• Create tables in aws_commitments schema
• Add CLI command: ./finops aws import-savings-plans

Phase 2: Data Enrichment

• Add offering details and pricing rates
• Implement state tracking (commitments rarely change)
• Create unified view with billing data

Success Criteria

• Daily pipeline ingests current Savings Plans inventory
• Data structure follows AWS API response payloads
• CLI integration matches existing AWS pipeline UX
• Tables ready for joining with CUR data analysis

Related Work

• Reserved Instances Inventory Pipeline (separate issue)
• Utilization Data Discovery (separate issue)
• Issue Does the open-finops-stack intends to show cost optimization recommendations? #10: Cost Optimization Recommendations (Phase 5)

Dependencies

• Existing AWS pipeline infrastructure
• DLT and DuckDB setup

Target Phase: Phase 5 (Production Features)