Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Preview api lb profile #3020

Merged
merged 18 commits into from
Jul 28, 2023
Merged

Preview api lb profile #3020

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
4c20517
add load balancer profile & generate clients
tony-schndr May 30, 2023
0ff4166
add LoadbalancerProfile to internal/admin API
tony-schndr Jun 21, 2023
8f7e4b9
add lb profile to internal/admin convert
tony-schndr Jun 21, 2023
7649ded
added examples for lb profile
tony-schndr Jun 22, 2023
3627c73
update lb profile to use pointer
tony-schndr Jun 23, 2023
5854f02
lb profile static validation
tony-schndr Jun 23, 2023
d4d6ddc
add default lb profile
tony-schndr Jul 8, 2023
712686c
fix converters
tony-schndr Jul 13, 2023
d3a6c79
validate effectiveOutboundIps
tony-schndr Jul 13, 2023
88c9226
remove mutable tag from admin/internal lb profile fields
tony-schndr Jul 13, 2023
647889c
add tests to cover all permutations
tony-schndr Jul 19, 2023
1352270
refactor static validation
tony-schndr Jul 19, 2023
8834b2c
add test case
tony-schndr Jul 20, 2023
b015e9c
add lb profile to frontend test patch_a_cluster_with_pucm_pending_req…
tony-schndr Jul 20, 2023
a3956bd
fix casing
tony-schndr Jul 21, 2023
5d8b03b
fix typos & regenerate clients
tony-schndr Jul 24, 2023
d2d81dc
set EffectiveOutboundIPs as readonly
tony-schndr Jul 25, 2023
2075648
fix typo in examples
tony-schndr Jul 28, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .sha256sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,5 @@
239c63228da1db172f298cd81d0c3cc0d52ecca907915efe61be98c42b6d8f1d swagger/redhatopenshift/resource-manager/Microsoft.RedHatOpenShift/stable/2022-04-01/redhatopenshift.json
1d167031baf0209fe8c46df9654585c64e8cc9a0c89555d7479c4ed6dc150251 swagger/redhatopenshift/resource-manager/Microsoft.RedHatOpenShift/stable/2022-09-04/redhatopenshift.json
622404e8311c62f27fba778e30e760bb1901e5bd221b23de72f449cafbdf0c45 swagger/redhatopenshift/resource-manager/Microsoft.RedHatOpenShift/stable/2023-04-01/redhatopenshift.json
4df3ebacaf35d77d09f5eab75fb9608241929b6ef8d00fb506455cd38e383640 swagger/redhatopenshift/resource-manager/Microsoft.RedHatOpenShift/preview/2023-07-01-preview/redhatopenshift.json
3aede70b183bad612c23cb776fe5a932c5709334e1fe1ad7ff8772b58be3661f swagger/redhatopenshift/resource-manager/Microsoft.RedHatOpenShift/preview/2023-07-01-preview/redhatopenshift.json
0c2bfc4b4308ff10d3cdd1c66c1356ba4153342ce536d478bcbb515fa1fe5958 swagger/redhatopenshift/resource-manager/Microsoft.RedHatOpenShift/stable/2023-09-04/redhatopenshift.json
44 changes: 40 additions & 4 deletions pkg/api/admin/openshiftcluster.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,41 @@ const (
OutboundTypeLoadbalancer OutboundType = "Loadbalancer"
)

// ResourceReference represents a reference to an Azure resource.
type ResourceReference struct {
bennerv marked this conversation as resolved.
Show resolved Hide resolved
// The fully qualified Azure resource id.
ID string `json:"id,omitempty"`
}

// LoadBalancerProfile represents the profile of the cluster public load balancer.
type LoadBalancerProfile struct {
// The desired managed outbound IPs for the cluster public load balancer.
ManagedOutboundIPs *ManagedOutboundIPs `json:"managedOutboundIps,omitempty"`
cadenmarchese marked this conversation as resolved.
Show resolved Hide resolved
// The list of effective outbound IP addresses of the public load balancer.
EffectiveOutboundIPs []EffectiveOutboundIP `json:"effectiveOutboundIps,omitempty"`
cadenmarchese marked this conversation as resolved.
Show resolved Hide resolved
// The desired outbound IP resources for the cluster load balancer.
OutboundIPs []OutboundIP `json:"outboundIps,omitempty"`
// The desired outbound IP Prefix resources for the cluster load balancer.
OutboundIPPrefixes []OutboundIPPrefix `json:"outboundIpPrefixes,omitempty"`
// The desired number of allocated SNAT ports per VM. Allowed values are in the range of 0 to 64000 (inclusive). The default value is 1024.
AllocatedOutboundPorts *int `json:"allocatedOutboundPorts,omitempty"`
}

// EffectiveOutboundIP represents an effective outbound IP resource of the cluster public load balancer.
type EffectiveOutboundIP ResourceReference

// ManagedOutboundIPs represents the desired managed outbound IPs for the cluster public load balancer.
type ManagedOutboundIPs struct {
// Count represents the desired number of IPv4 outbound IPs created and managed by Azure for the cluster public load balancer. Allowed values are in the range of 1 - 20. The default value is 1.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ideally we'd bake these mins/maxes into our swagger API specification, similar to what AKS is doing.

Might make sense to do it later though:
https://github.com/Azure/azure-rest-api-specs/blob/main/specification/containerservice/resource-manager/Microsoft.ContainerService/aks/stable/2023-06-01/managedClusters.json#L3807-L3819C1

Count int `json:"count,omitempty"`
}

// OutboundIP represents a desired outbound IP resource for the cluster load balancer.
type OutboundIP ResourceReference

// OutboundIPPrefix represents a desired outbound IP Prefix resource for the cluster load balancer.
type OutboundIPPrefix ResourceReference

// NetworkProfile represents a network profile.
type NetworkProfile struct {
// The software defined network (SDN) to use when installing the cluster.
Expand All @@ -151,10 +186,11 @@ type NetworkProfile struct {
MTUSize MTUSize `json:"mtuSize,omitempty"`
OutboundType OutboundType `json:"outboundType,omitempty" mutable:"true"`

APIServerPrivateEndpointIP string `json:"privateEndpointIp,omitempty"`
GatewayPrivateEndpointIP string `json:"gatewayPrivateEndpointIp,omitempty"`
GatewayPrivateLinkID string `json:"gatewayPrivateLinkId,omitempty"`
PreconfiguredNSG PreconfiguredNSG `json:"preconfigureNSG,omitempty"`
APIServerPrivateEndpointIP string `json:"privateEndpointIp,omitempty"`
GatewayPrivateEndpointIP string `json:"gatewayPrivateEndpointIp,omitempty"`
GatewayPrivateLinkID string `json:"gatewayPrivateLinkId,omitempty"`
PreconfiguredNSG PreconfiguredNSG `json:"preconfigureNSG,omitempty"`
LoadBalancerProfile *LoadBalancerProfile `json:"loadBalancerProfile,omitempty"`
}

// PreconfiguredNSG represents whether customers want to use their own NSG attached to the subnets
Expand Down
73 changes: 73 additions & 0 deletions pkg/api/admin/openshiftcluster_convert.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,47 @@ func (c openShiftClusterConverter) ToExternal(oc *api.OpenShiftCluster) interfac
},
}

if oc.Properties.NetworkProfile.LoadBalancerProfile != nil {
out.Properties.NetworkProfile.LoadBalancerProfile = &LoadBalancerProfile{}

if oc.Properties.NetworkProfile.LoadBalancerProfile.AllocatedOutboundPorts != nil {
out.Properties.NetworkProfile.LoadBalancerProfile.AllocatedOutboundPorts = oc.Properties.NetworkProfile.LoadBalancerProfile.AllocatedOutboundPorts
}

if oc.Properties.NetworkProfile.LoadBalancerProfile.ManagedOutboundIPs != nil {
out.Properties.NetworkProfile.LoadBalancerProfile.ManagedOutboundIPs = &ManagedOutboundIPs{
Count: oc.Properties.NetworkProfile.LoadBalancerProfile.ManagedOutboundIPs.Count,
}
}

if oc.Properties.NetworkProfile.LoadBalancerProfile.EffectiveOutboundIPs != nil {
out.Properties.NetworkProfile.LoadBalancerProfile.EffectiveOutboundIPs = make([]EffectiveOutboundIP, 0, len(oc.Properties.NetworkProfile.LoadBalancerProfile.EffectiveOutboundIPs))
for _, effectiveOutboundIP := range oc.Properties.NetworkProfile.LoadBalancerProfile.EffectiveOutboundIPs {
out.Properties.NetworkProfile.LoadBalancerProfile.EffectiveOutboundIPs = append(out.Properties.NetworkProfile.LoadBalancerProfile.EffectiveOutboundIPs, EffectiveOutboundIP{
ID: effectiveOutboundIP.ID,
})
}
}

if oc.Properties.NetworkProfile.LoadBalancerProfile.OutboundIPs != nil {
out.Properties.NetworkProfile.LoadBalancerProfile.OutboundIPs = make([]OutboundIP, 0, len(oc.Properties.NetworkProfile.LoadBalancerProfile.OutboundIPs))
for _, outboundIP := range oc.Properties.NetworkProfile.LoadBalancerProfile.OutboundIPs {
out.Properties.NetworkProfile.LoadBalancerProfile.OutboundIPs = append(out.Properties.NetworkProfile.LoadBalancerProfile.OutboundIPs, OutboundIP{
ID: outboundIP.ID,
})
}
}

if oc.Properties.NetworkProfile.LoadBalancerProfile.OutboundIPPrefixes != nil {
out.Properties.NetworkProfile.LoadBalancerProfile.OutboundIPPrefixes = make([]OutboundIPPrefix, 0, len(oc.Properties.NetworkProfile.LoadBalancerProfile.OutboundIPPrefixes))
for _, outboundIPPrefix := range oc.Properties.NetworkProfile.LoadBalancerProfile.OutboundIPPrefixes {
out.Properties.NetworkProfile.LoadBalancerProfile.OutboundIPPrefixes = append(out.Properties.NetworkProfile.LoadBalancerProfile.OutboundIPPrefixes, OutboundIPPrefix{
ID: outboundIPPrefix.ID,
})
}
}
}

if oc.Properties.WorkerProfiles != nil {
out.Properties.WorkerProfiles = make([]WorkerProfile, 0, len(oc.Properties.WorkerProfiles))
for _, p := range oc.Properties.WorkerProfiles {
Expand Down Expand Up @@ -193,6 +234,38 @@ func (c openShiftClusterConverter) ToInternal(_oc interface{}, out *api.OpenShif
out.Properties.NetworkProfile.APIServerPrivateEndpointIP = oc.Properties.NetworkProfile.APIServerPrivateEndpointIP
out.Properties.NetworkProfile.GatewayPrivateEndpointIP = oc.Properties.NetworkProfile.GatewayPrivateEndpointIP
out.Properties.NetworkProfile.GatewayPrivateLinkID = oc.Properties.NetworkProfile.GatewayPrivateLinkID
if oc.Properties.NetworkProfile.LoadBalancerProfile != nil {
out.Properties.NetworkProfile.LoadBalancerProfile = &api.LoadBalancerProfile{}

if oc.Properties.NetworkProfile.LoadBalancerProfile.AllocatedOutboundPorts != nil {
out.Properties.NetworkProfile.LoadBalancerProfile.AllocatedOutboundPorts = oc.Properties.NetworkProfile.LoadBalancerProfile.AllocatedOutboundPorts
}

if oc.Properties.NetworkProfile.LoadBalancerProfile.ManagedOutboundIPs != nil {
out.Properties.NetworkProfile.LoadBalancerProfile.ManagedOutboundIPs = &api.ManagedOutboundIPs{
Count: oc.Properties.NetworkProfile.LoadBalancerProfile.ManagedOutboundIPs.Count,
}
}
if oc.Properties.NetworkProfile.LoadBalancerProfile.OutboundIPs != nil {
out.Properties.NetworkProfile.LoadBalancerProfile.OutboundIPs = make([]api.OutboundIP, len(oc.Properties.NetworkProfile.LoadBalancerProfile.OutboundIPs))
for i := range oc.Properties.NetworkProfile.LoadBalancerProfile.OutboundIPs {
out.Properties.NetworkProfile.LoadBalancerProfile.OutboundIPs[i].ID = oc.Properties.NetworkProfile.LoadBalancerProfile.OutboundIPs[i].ID
}
}
if oc.Properties.NetworkProfile.LoadBalancerProfile.OutboundIPPrefixes != nil {
out.Properties.NetworkProfile.LoadBalancerProfile.OutboundIPPrefixes = make([]api.OutboundIPPrefix, len(oc.Properties.NetworkProfile.LoadBalancerProfile.OutboundIPPrefixes))
for i := range oc.Properties.NetworkProfile.LoadBalancerProfile.OutboundIPPrefixes {
out.Properties.NetworkProfile.LoadBalancerProfile.OutboundIPPrefixes[i].ID = oc.Properties.NetworkProfile.LoadBalancerProfile.OutboundIPPrefixes[i].ID
}
}
if oc.Properties.NetworkProfile.LoadBalancerProfile.EffectiveOutboundIPs != nil {
out.Properties.NetworkProfile.LoadBalancerProfile.EffectiveOutboundIPs = make([]api.EffectiveOutboundIP, len(oc.Properties.NetworkProfile.LoadBalancerProfile.EffectiveOutboundIPs))
for i := range oc.Properties.NetworkProfile.LoadBalancerProfile.EffectiveOutboundIPs {
out.Properties.NetworkProfile.LoadBalancerProfile.EffectiveOutboundIPs[i].ID = oc.Properties.NetworkProfile.LoadBalancerProfile.EffectiveOutboundIPs[i].ID
}
}
}

out.Properties.MasterProfile.VMSize = api.VMSize(oc.Properties.MasterProfile.VMSize)
out.Properties.MasterProfile.SubnetID = oc.Properties.MasterProfile.SubnetID
out.Properties.MasterProfile.EncryptionAtHost = api.EncryptionAtHost(oc.Properties.MasterProfile.EncryptionAtHost)
Expand Down
9 changes: 9 additions & 0 deletions pkg/api/defaults.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,15 @@ func SetDefaults(doc *OpenShiftClusterDocument) {
if doc.OpenShiftCluster.Properties.NetworkProfile.PreconfiguredNSG == "" {
doc.OpenShiftCluster.Properties.NetworkProfile.PreconfiguredNSG = PreconfiguredNSGDisabled
}

// If OutboundType is Loadbalancer and there is no LoadBalancerProfile, set default one
cadenmarchese marked this conversation as resolved.
Show resolved Hide resolved
if doc.OpenShiftCluster.Properties.NetworkProfile.OutboundType == OutboundTypeLoadbalancer && doc.OpenShiftCluster.Properties.NetworkProfile.LoadBalancerProfile == nil {
doc.OpenShiftCluster.Properties.NetworkProfile.LoadBalancerProfile = &LoadBalancerProfile{
ManagedOutboundIPs: &ManagedOutboundIPs{
Count: 1,
},
}
}
}
}

Expand Down
15 changes: 15 additions & 0 deletions pkg/api/defaults_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,11 @@ func validOpenShiftClusterDocument() *OpenShiftClusterDocument {
SoftwareDefinedNetwork: SoftwareDefinedNetworkOpenShiftSDN,
OutboundType: OutboundTypeLoadbalancer,
PreconfiguredNSG: PreconfiguredNSGDisabled,
LoadBalancerProfile: &LoadBalancerProfile{
ManagedOutboundIPs: &ManagedOutboundIPs{
Count: 1,
},
},
},
MasterProfile: MasterProfile{
EncryptionAtHost: EncryptionAtHostDisabled,
Expand Down Expand Up @@ -108,6 +113,16 @@ func TestSetDefaults(t *testing.T) {
base.OpenShiftCluster.Properties.OperatorFlags = OperatorFlags{}
},
},
{
name: "default lb profile",
want: func() *OpenShiftClusterDocument {
doc := validOpenShiftClusterDocument()
return doc
},
input: func(base *OpenShiftClusterDocument) {
base.OpenShiftCluster.Properties.NetworkProfile.LoadBalancerProfile = nil
},
},
} {
t.Run(tt.name, func(t *testing.T) {
doc := validOpenShiftClusterDocument()
Expand Down
44 changes: 40 additions & 4 deletions pkg/api/openshiftcluster.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,41 @@ const (
OutboundTypeLoadbalancer OutboundType = "Loadbalancer"
)

// ResourceReference represents a reference to an Azure resource.
type ResourceReference struct {
// The fully qualified Azure resource id.
ID string `json:"id,omitempty"`
}

// LoadBalancerProfile represents the profile of the cluster public load balancer.
type LoadBalancerProfile struct {
// The desired managed outbound IPs for the cluster public load balancer.
ManagedOutboundIPs *ManagedOutboundIPs `json:"managedOutboundIps,omitempty"`
// The list of effective outbound IP addresses of the public load balancer.
EffectiveOutboundIPs []EffectiveOutboundIP `json:"effectiveOutboundIps,omitempty"`
// The desired outbound IP resources for the cluster load balancer.
OutboundIPs []OutboundIP `json:"outboundIps,omitempty"`
// The desired outbound IP Prefix resources for the cluster load balancer.
OutboundIPPrefixes []OutboundIPPrefix `json:"outboundIpPrefixes,omitempty"`
// The desired number of allocated SNAT ports per VM. Allowed values are in the range of 0 to 64000 (inclusive). The default value is 1024.
AllocatedOutboundPorts *int `json:"allocatedOutboundPorts,omitempty"`
}

// EffectiveOutboundIP represents an effective outbound IP resource of the cluster public load balancer.
type EffectiveOutboundIP ResourceReference

// ManagedOutboundIPs represents the desired managed outbound IPs for the cluster public load balancer.
type ManagedOutboundIPs struct {
// Count represents the desired number of IPv4 outbound IPs created and managed by Azure for the cluster public load balancer. Allowed values are in the range of 1 - 20. The default value is 1.
Count int `json:"count,omitempty"`
}

// OutboundIP represents a desired outbound IP resource for the cluster load balancer.
type OutboundIP ResourceReference

// OutboundIPPrefix represents a desired outbound IP Prefix resource for the cluster load balancer.
type OutboundIPPrefix ResourceReference

// NetworkProfile represents a network profile
type NetworkProfile struct {
MissingFields
Expand All @@ -274,10 +309,11 @@ type NetworkProfile struct {
MTUSize MTUSize `json:"mtuSize,omitempty"`
OutboundType OutboundType `json:"outboundType,omitempty"`

APIServerPrivateEndpointIP string `json:"privateEndpointIp,omitempty"`
GatewayPrivateEndpointIP string `json:"gatewayPrivateEndpointIp,omitempty"`
GatewayPrivateLinkID string `json:"gatewayPrivateLinkId,omitempty"`
PreconfiguredNSG PreconfiguredNSG `json:"preconfiguredNSG,omitempty"`
APIServerPrivateEndpointIP string `json:"privateEndpointIp,omitempty"`
GatewayPrivateEndpointIP string `json:"gatewayPrivateEndpointIp,omitempty"`
GatewayPrivateLinkID string `json:"gatewayPrivateLinkId,omitempty"`
PreconfiguredNSG PreconfiguredNSG `json:"preconfiguredNSG,omitempty"`
LoadBalancerProfile *LoadBalancerProfile `json:"loadBalancerProfile,omitempty"`
}

// PreconfiguredNSG represents whether customers want to use their own NSG attached to the subnets
Expand Down
38 changes: 38 additions & 0 deletions pkg/api/v20230701preview/openshiftcluster.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,41 @@ const (
OutboundTypeLoadbalancer OutboundType = "Loadbalancer"
)

// ResourceReference represents a reference to an Azure resource.
type ResourceReference struct {
// The fully qualified Azure resource id.
ID string `json:"id,omitempty"`
}

// LoadBalancerProfile represents the profile of the cluster public load balancer.
type LoadBalancerProfile struct {
// The desired managed outbound IPs for the cluster public load balancer.
ManagedOutboundIPs *ManagedOutboundIPs `json:"managedOutboundIps,omitempty" mutable:"true"`
tony-schndr marked this conversation as resolved.
Show resolved Hide resolved
// The list of effective outbound IP addresses of the public load balancer.
EffectiveOutboundIPs []EffectiveOutboundIP `json:"effectiveOutboundIps,omitempty"`
// The desired outbound IP resources for the cluster load balancer.
OutboundIPs []OutboundIP `json:"outboundIps,omitempty" mutable:"true"`
// The desired outbound IP Prefix resources for the cluster load balancer.
OutboundIPPrefixes []OutboundIPPrefix `json:"outboundIpPrefixes,omitempty" mutable:"true"`
// The desired number of allocated SNAT ports per VM. Allowed values are in the range of 0 to 64000 (inclusive). The default value is 1024.
AllocatedOutboundPorts *int `json:"allocatedOutboundPorts,omitempty" mutable:"true"`
tony-schndr marked this conversation as resolved.
Show resolved Hide resolved
}

// EffectiveOutboundIP represents an effective outbound IP resource of the cluster public load balancer.
type EffectiveOutboundIP ResourceReference

// ManagedOutboundIPs represents the desired managed outbound IPs for the cluster public load balancer.
type ManagedOutboundIPs struct {
// Count represents the desired number of IPv4 outbound IPs created and managed by Azure for the cluster public load balancer. Allowed values are in the range of 1 - 20. The default value is 1.
Count int `json:"count,omitempty"`
}

// OutboundIP represents a desired outbound IP resource for the cluster load balancer.
type OutboundIP ResourceReference

// OutboundIPPrefix represents a desired outbound IP Prefix resource for the cluster load balancer.
type OutboundIPPrefix ResourceReference

// NetworkProfile represents a network profile.
type NetworkProfile struct {
// The CIDR used for OpenShift/Kubernetes Pods.
Expand All @@ -143,6 +178,9 @@ type NetworkProfile struct {

// The OutboundType used for egress traffic.
OutboundType OutboundType `json:"outboundType,omitempty"`

// The cluster load balancer profile.
LoadBalancerProfile *LoadBalancerProfile `json:"loadBalancerProfile,omitempty"`
}

// EncryptionAtHost represents encryption at host state
Expand Down
Loading