Scale Considerations

[camrossi]

• This design should be fine with up to 500 K8s nodes.
• We need to add a scaling guideline for ACI as well around
  • BGP Peers
  • BGP Prefixes
  • ExtEPGs
  • ESG Scale

[robvand]

Adding a few scale considerations and limits for ACI:

• Floating L3Out scale with 6 anchors and 32 non-anchor in 6.1(x).

• IPs per MAC = 4096

• BFD neighbors: Up to 256 sessions using these minimum BFD timers: minTx:50, minRx:50, multiplier:3. 257-2,000 sessions using these minimum BFD timers: minTx:300, minRx:300, multiplier:3

• number of BGP neighbors (2000 per leaf with up to 70000 external prefixes with a single path). 20k per fabric scale.

• Shared L3Out (when leaking between VRFs) 2000 IPv4 prefixes.

• External EPGs per L3out (250 per L3out), 600 fabric wide. (number of external EPGs per L3out) * (Number of border leaf switches for the L3out)

• Number of ESGs per Fabric = 10000

• Number of ESGs per VRF = 4000

• Number of ESGs per tenant = 4000

• Number of L3 IP Selectors per leaf = 5000

• Number of IP Longest Prefix Matches (LPM) entries: 20000 IPv4 with default dual stack profile. Worth noting that changing the profile reduces the amount of supported ECMP paths.

The shared L3out limit IPv4 prefix limitation needs additional investigation as its likely to apply in this type of deployment.