feat(poolautoscaler): add PoolAutoscaler CRD with capacity-based and cron scaling by chrisliu1995 · Pull Request #625 · openkruise/agents

chrisliu1995 · 2026-07-09T03:21:06Z

Ⅰ. Describe what this PR does

Add PoolAutoscaler CRD and controller that automatically scales SandboxSet replicas based on claim demand. Key components:

PoolAutoscaler CRD type (api/v1alpha1/poolautoscaler_types.go): Defines spec with scaleTargetRef, minReplicas/maxReplicas, capacityPolicy (targetAvailable, tolerance, scaleUp/scaleDown stabilization), and cronPolicies for time-based scaling.
Capacity controller (pkg/controller/poolautoscaler/capacity.go): Implements observation window with rolling samples (averages available/statusReplicas over a configurable period) and cooldown-based stabilization window. The observation window prevents thrashing from transient fluctuations; the cooldown model uses lastScaleUpAt/lastScaleDownAt timestamps with first-action exemption for responsive initial scaling.
Configurable observation parameters via CLI flags: --observation-window-seconds (default 15) and --sampling-interval-seconds (default 5).
Cron scaling (pkg/controller/poolautoscaler/cron.go): Supports time-based scaling rules with timezone, start/end time, and recurrence patterns.
Validating webhook (pkg/webhook/poolautoscaler/): Validates PoolAutoscaler spec for targetRef, replica bounds, and capacity policy configuration.
Feature gate (pkg/features/features.go): PoolAutoscalerController gate for controller registration.
Generated code: clientset, informers, listers, CRD manifest, RBAC, and webhook configuration.

Ⅱ. Does this pull request fix one issue?

NONE

Ⅲ. Describe how to verify it

Run unit tests:

go test ./pkg/controller/poolautoscaler/...
go test ./pkg/webhook/poolautoscaler/...

Deploy the controller with the CRD applied, create a SandboxSet and a PoolAutoscaler targeting it:

apiVersion: agents.kruise.io/v1alpha1
kind: PoolAutoscaler
metadata:
  name: demo-pa
  namespace: default
spec:
  scaleTargetRef:
    kind: SandboxSet
    name: demo-pool
  minReplicas: 20
  maxReplicas: 2000
  capacityPolicy:
    targetAvailable: 50%
    tolerance: 5%
    scaleUp:
      stabilizationWindowSeconds: 0
    scaleDown:
      stabilizationWindowSeconds: 0

Create SandboxClaims to trigger scale-up; verify the SandboxSet replicas increase to meet demand.
Delete SandboxClaims; verify the SandboxSet scales back down to minReplicas.
Verify observation window behavior: with --observation-window-seconds=15 --sampling-interval-seconds=5, the controller averages 3 samples before computing desired replicas.

Ⅳ. Special notes for reviews

The observation window uses a capacityMonitor struct with a mutex-protected samples slice. Samples are pruned to the window size on each reconcile. The aggregatedValues() method returns sampleCount to avoid lock-free reads of the samples slice.
The stabilization window uses a cooldown model (not a continuous-condition model): lastScaleUpAt/lastScaleDownAt timestamps are set after each scale action. First scale action is exempt from cooldown (IsZero() check) for responsive initial scaling. Subsequent actions must wait for the configured stabilization window to elapse.
recordScaleAction does NOT clear samples after scaling — the observation window continues to accumulate, providing smooth transitions.
The controller's RequeueAfter follows samplingIntervalSeconds (default 5s) rather than a fixed interval, tying reconcile frequency to the observation granularity.
Memory profiled with pprof under load (8000+ sandboxes): heap scales linearly at ~25KB/sandbox, dominated by informer cache (ObjectMeta, PodStatus) and Prometheus metric series. No memory leaks detected — heap drops from 192MB to 24MB after sandbox cleanup.

kruise-bot · 2026-07-09T03:21:11Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign furykerry for approval by writing /assign @furykerry in a comment. For more information see:The Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

…cron scaling Add PoolAutoscaler controller that automatically scales SandboxSet replicas based on claim demand. Key components: - PoolAutoscaler CRD type with capacityPolicy and cronPolicies - Capacity controller with observation window (rolling samples) and cooldown-based stabilization window for scale up/down - Configurable observation window and sampling interval via CLI flags (--observation-window-seconds, --sampling-interval-seconds) - Cron scaling policies for time-based scaling rules - Validating webhook for PoolAutoscaler spec validation - Feature gate for controller registration - Generated clientset, informers, listers, CRD manifest, RBAC, and webhook configuration The observation window averages available/status replicas over a configurable time period (default 15s) before computing desired replicas, preventing thrashing from transient fluctuations. The cooldown model uses lastScaleUpAt/lastScaleDownAt timestamps with first-action exemption for responsive initial scaling. Signed-off-by: ChrisLiu <chrisliu1995@163.com>

codecov · 2026-07-09T03:37:20Z

Codecov Report

❌ Patch coverage is 87.83270% with 64 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.23%. Comparing base (5c9549f) to head (8a22353).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
...roller/poolautoscaler/poolautoscaler_controller.go	82.38%	26 Missing and 5 partials ⚠️
...olautoscaler/validating/poolautoscaler_validate.go	85.00%	9 Missing and 6 partials ⚠️
pkg/controller/poolautoscaler/capacity.go	93.01%	7 Missing and 6 partials ⚠️
pkg/controller/controllers.go	0.00%	1 Missing and 1 partial ⚠️
pkg/controller/poolautoscaler/cron.go	96.29%	1 Missing and 1 partial ⚠️
pkg/webhook/server.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #625      +/-   ##
==========================================
+ Coverage   80.00%   80.23%   +0.23%     
==========================================
  Files         225      230       +5     
  Lines       17645    18171     +526     
==========================================
+ Hits        14117    14580     +463     
- Misses       2953     2998      +45     
- Partials      575      593      +18

Flag	Coverage Δ
unittests	`80.23% <87.83%> (+0.23%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…vert Signed-off-by: ChrisLiu <chrisliu1995@163.com>

… and coverage gaps Signed-off-by: ChrisLiu <chrisliu1995@163.com>

zmberg · 2026-07-10T03:07:30Z

+	// MinReplicas is the lower limit for the number of replicas to which the autoscaler
+	// can scale down. It defaults to 0 pods.
+	// +optional
+	MinReplicas *int32 `json:"minReplicas,omitempty"`


Is the pointer necessary here? The default value of 0 seems sufficient.

zmberg · 2026-07-10T03:07:30Z

+
+	// CronPolicies is a list of potential cron scaling policies which can be used during scaling.
+	// +optional
+	CronPolicies []CronScalingPolicy `json:"cronPolicies,omitempty"`


Are CronPolicies and CapacityPolicy mutually exclusive? If both are set simultaneously, would the implementation be complex and the logic unclear?

If not well thought through, I suggest not supporting both at the same time for now — this can be enforced via webhook validation. Additionally, if the controller detects both are set, it should log a warning. CronPolicies should take higher priority.

zmberg · 2026-07-10T03:07:30Z

+// aggregatedValues returns the average available and statusReplicas
+// from samples within the observation window, along with the sample count.
+// Returns ok=false if no samples exist.
+func (m *capacityMonitor) aggregatedValues() (avgAvailable, avgStatusReplicas int32, sampleCount int, ok bool) {


avgStatusReplicas -> avgReplicas

zmberg · 2026-07-10T03:07:30Z

+	var sumAvailable, sumStatus int64
+	for _, s := range m.samples {
+		sumAvailable += int64(s.available)
+		sumStatus += int64(s.statusReplicas)


sumStatus looks odd, how about sumReplicas?

zmberg · 2026-07-10T03:07:31Z

+	monitor.addSampleIfDue(rawAvailable, rawStatusReplicas, now)
+	monitor.pruneSamples(now)
+
+	avgAvail, avgStatus, sampleCount, ok := monitor.aggregatedValues()


Consider adding debug logs here to print all samples for easier troubleshooting.

zmberg · 2026-07-10T03:07:31Z

+// +kubebuilder:rbac:groups=agents.kruise.io,resources=sandboxsets/scale,verbs=get;update;patch
+
+func (r *Reconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
+	logger := logf.FromContext(ctx).WithValues("poolautoscaler", req.NamespacedName)


zmberg · 2026-07-10T03:07:31Z

+	rawStatusReplicas := sbs.Status.Replicas
+	rawAvailable := sbs.Status.AvailableReplicas
+
+	avgAvailable, avgStatusReplicas := r.observeAndAggregate(ctx, pa, rawAvailable, rawStatusReplicas)


I think rawAvailable is not an appropriate name here — it should be specReplicas.

zmberg · 2026-07-10T03:07:31Z

+
+// updateStatus updates the PoolAutoscaler status fields.
+func (r *Reconciler) updateStatus(ctx context.Context, pa *agentsv1alpha1.PoolAutoscaler, currentReplicas, desiredReplicas, available int32, appliedCronPolicies []agentsv1alpha1.CronScalingPolicyStatus, suspended bool) error {
+	patch := client.MergeFrom(pa.DeepCopy())


Check whether the status actually changed before patching, to avoid unnecessary Patch requests.

zmberg · 2026-07-10T03:07:31Z

+	if pa.Spec.MinReplicas != nil {
+		minReplicas = *pa.Spec.MinReplicas
+	}
+	if desiredReplicas == pa.Spec.MaxReplicas && desiredReplicas == minReplicas {


Does this mean minReplicas == maxReplicas? This seems overly complex — the webhook validation should enforce that minReplicas != maxReplicas.

zmberg · 2026-07-10T03:07:31Z

+}
+
+// setCondition updates or appends a condition in the PoolAutoscaler status.
+func setCondition(pa *agentsv1alpha1.PoolAutoscaler, condition metav1.Condition) {


Implementation reference: https://github.com/openkruise/agents/blob/master/pkg/utils/utils.go#L50

kruise-bot requested review from AiRanthem, PersistentJZH, furykerry and zmberg July 9, 2026 03:21

kruise-bot added needs-rebase size/XXL labels Jul 9, 2026

chrisliu1995 force-pushed the feature/pool-autoscaler branch from e1c9b82 to 07ab971 Compare July 9, 2026 03:31

kruise-bot removed the needs-rebase label Jul 9, 2026

chrisliu1995 added 2 commits July 9, 2026 11:41

fix: remove unnecessary int conversion flagged by golangci-lint uncon…

b7960a6

…vert Signed-off-by: ChrisLiu <chrisliu1995@163.com>

test(poolautoscaler): add unit tests for controller, webhook handler,…

8a22353

… and coverage gaps Signed-off-by: ChrisLiu <chrisliu1995@163.com>

zmberg reviewed Jul 10, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(poolautoscaler): add PoolAutoscaler CRD with capacity-based and cron scaling#625

feat(poolautoscaler): add PoolAutoscaler CRD with capacity-based and cron scaling#625
chrisliu1995 wants to merge 3 commits into
openkruise:masterfrom
chrisliu1995:feature/pool-autoscaler

chrisliu1995 commented Jul 9, 2026

Uh oh!

kruise-bot commented Jul 9, 2026

Uh oh!

codecov Bot commented Jul 9, 2026 •

edited

Loading

Uh oh!

zmberg Jul 10, 2026

Uh oh!

zmberg Jul 10, 2026

Uh oh!

zmberg Jul 10, 2026

Uh oh!

zmberg Jul 10, 2026

Uh oh!

zmberg Jul 10, 2026

Uh oh!

zmberg Jul 10, 2026

Uh oh!

zmberg Jul 10, 2026

Uh oh!

zmberg Jul 10, 2026

Uh oh!

zmberg Jul 10, 2026

Uh oh!

zmberg Jul 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

chrisliu1995 commented Jul 9, 2026

Ⅰ. Describe what this PR does

Ⅱ. Does this pull request fix one issue?

Ⅲ. Describe how to verify it

Ⅳ. Special notes for reviews

Uh oh!

kruise-bot commented Jul 9, 2026

Uh oh!

codecov Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov Bot commented Jul 9, 2026 •

edited

Loading