diff --git a/README.md b/README.md index 53771902..c69edd0f 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # EC2 Fleet Plugin for Jenkins -The EC2 Fleet Plugin scales your [Auto Scaling Group](https://docs.aws.amazon.com/autoscaling/ec2/userguide/AutoScalingGroup.html) or [EC2 Spot Fleet](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-fleet.html) +The EC2 Fleet Plugin scales your [Auto Scaling Group](https://docs.aws.amazon.com/autoscaling/ec2/userguide/AutoScalingGroup.html), [EC2 Fleet](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-fleet.html), or [Spot Fleet](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-fleet.html) automatically for your Jenkins workload. [![Build Status](https://ci.jenkins.io/buildStatus/icon?job=Plugins/ec2-fleet-plugin/master)](https://ci.jenkins.io/blue/organizations/jenkins/Plugins%2Fec2-fleet-plugin/activity) [![](https://img.shields.io/jenkins/plugin/v/ec2-fleet.svg)](https://github.com/jenkinsci/ec2-fleet-plugin/releases) @@ -24,8 +24,8 @@ automatically for your Jenkins workload. # Overview -The EC2 Fleet Plugin scales your [Auto Scaling Group](https://docs.aws.amazon.com/autoscaling/ec2/userguide/AutoScalingGroup.html) or [EC2 Spot Fleet](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-fleet.html) -automatically for your Jenkins workload. It handles launching new instances that match the criteria set in your ASG or Spot Fleet e.g. [allocation strategy](https://docs.aws.amazon.com/autoscaling/ec2/userguide/ec2-auto-scaling-mixed-instances-groups.html#allocation-strategies), +The EC2 Fleet Plugin scales your [Auto Scaling Group](https://docs.aws.amazon.com/autoscaling/ec2/userguide/AutoScalingGroup.html), [EC2 Fleet](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-fleet.html), or [Spot Fleet](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-fleet.html) +automatically for your Jenkins workload. It handles launching new instances that match the criteria set in your ASG, EC2 Fleet, or Spot Fleet e.g. [allocation strategy](https://docs.aws.amazon.com/autoscaling/ec2/userguide/ec2-auto-scaling-mixed-instances-groups.html#allocation-strategies), and terminating idle instances that breach that criteria or those in your Jenkins Cloud configuration. > [!WARNING] @@ -39,8 +39,8 @@ Minimum Jenkins version: 2.277.2 > [Jenkins version 2.403](https://github.com/jenkinsci/jenkins/releases/tag/jenkins-2.403) includes [significant changes to cloud management](https://issues.jenkins.io/browse/JENKINS-70729). > If you are using that version, and see unexpected behavior, create an issue and let us know. -- Supports EC2 Spot Fleet or Auto Scaling Group as Jenkins Workers -- Supports all features provided by EC2 Spot Fleet or Auto Scaling Groups e.g. [multiple instance types across Spot and On-Demand](https://docs.aws.amazon.com/autoscaling/ec2/userguide/ec2-auto-scaling-mixed-instances-groups.html#allocation-strategies) +- Supports EC2 Fleet, Spot Fleet, or Auto Scaling Group as Jenkins Workers +- Supports all features provided by EC2 Fleet, Spot Fleet, or Auto Scaling Groups e.g. [multiple instance types across Spot and On-Demand](https://docs.aws.amazon.com/autoscaling/ec2/userguide/ec2-auto-scaling-mixed-instances-groups.html#allocation-strategies) - Auto resubmit failed jobs caused by Spot interruptions - No delay scale up strategy: enable ```No Delay Provision Strategy``` in configuration - Add tags to EC2 instances used by plugin, for easy search, tag format ```ec2-fleet-plugin:cloud-name=``` @@ -53,8 +53,8 @@ Minimum Jenkins version: 2.277.2 ## Comparison to EC2-Plugin [EC2-Plugin](https://plugins.jenkins.io/ec2/) is a similar Jenkins plugin that will request EC2 instances when excess workload is -detected. The main difference between the two plugins is that EC2-Fleet-Plugin uses ASG and EC2 Spot Fleet to request and manage -instances instead of doing it manually with EC2 RunInstances. This gives EC2-Fleet-Plugin all the benefits of ASG and EC2 Spot Fleet: +detected. The main difference between the two plugins is that EC2-Fleet-Plugin uses ASG, EC2 Fleet, and Spot Fleet to request and manage +instances instead of doing it manually with EC2 RunInstances. This gives EC2-Fleet-Plugin all the benefits of ASG, EC2 Fleet, and Spot Fleet: allocation strategies, automatic availability zone re-balancing (ASG only), access to launch templates and launch configurations , instance weighting, etc. See [which-spot-request-method-to-use](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-best-practices.html#which-spot-request-method-to-use). @@ -62,8 +62,8 @@ See [which-spot-request-method-to-use](https://docs.aws.amazon.com/AWSEC2/latest | EC2-Fleet-Plugin | EC2-Plugin | |--------------------------------------------------------------------|-----------------------------------------------| | Supports On-Demand & Spot Instances | Supports On-Demand & Spot Instances | -| Scales with ASG or EC2 Spot Fleet | Scales with RunInstances | -| ASG and EC2 Spot Fleet Allocation Strategies | No Allocation Strategies | +| Scales with ASG, EC2 Fleet, or Spot Fleet | Scales with RunInstances | +| ASG, EC2 Fleet, and Spot Fleet Allocation Strategies | No Allocation Strategies | | Use launch template/config to set instance settings | Manually set instances settings within plugin | | Custom instance weighting | No custom instance weighting | | Supports mixed configuration like instance types, purchase options | Supports single instance type only | @@ -94,13 +94,13 @@ Go to [AWS account](http://aws.amazon.com/ec2/) and follow instructions. #### 2. Create IAM User Specify ```programmatic access``` during creation and record the credentials. These will -be used by Jenkins EC2 Fleet Plugin to connect to your Spot Fleet. +be used by Jenkins EC2 Fleet Plugin to connect to your EC2 Fleet or Spot Fleet. *Alternatively, you may use [AWS EC2 instance roles](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html)* #### 3. Configure User permissions -Add an inline policy to the IAM user or EC2 instance role to allow it to use EC2 Spot Fleet and Auto Scaling Group. +Add an inline policy to the IAM user or EC2 instance role to allow it to use EC2 Fleet, Spot Fleet, and Auto Scaling Group. [AWS documentation about this](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-fleet-requests.html#spot-fleet-prerequisites) ```json @@ -117,7 +117,11 @@ Add an inline policy to the IAM user or EC2 instance role to allow it to use EC2 "ec2:DescribeInstances", "ec2:TerminateInstances", "ec2:DescribeInstanceStatus", - "ec2:DescribeSpotFleetRequests" + "ec2:DescribeSpotFleetRequests", + "ec2:DescribeFleets", + "ec2:DescribeFleetInstances", + "ec2:ModifyFleet", + "ec2:DescribeInstanceTypes" ], "Resource": "*" }, @@ -160,7 +164,7 @@ Add an inline policy to the IAM user or EC2 instance role to allow it to use EC2 } ``` -#### 4. Create an Auto Scaling Group or Spot Fleet +#### 4. Create an Auto Scaling Group, EC2 Fleet, or Spot Fleet https://docs.aws.amazon.com/cli/latest/reference/autoscaling/create-auto-scaling-group.html Here is a [getting started tutorial for ASG](https://docs.aws.amazon.com/autoscaling/ec2/userguide/GettingStartedTutorial.html). @@ -176,14 +180,14 @@ Use Auto Scaling Groups instead. #### 5. Configure Jenkins -Once your ASG or Spot Fleet is ready, you can use it by adding a new **EC2 Fleet** cloud in the Jenkins configuration. +Once your ASG, EC2 Fleet, or Spot Fleet is ready, you can use it by adding a new **EC2 Fleet** cloud in the Jenkins configuration. 1. Goto ```Manage Jenkins > Plugin Manager``` 1. Install ```EC2 Fleet Jenkins Plugin``` 1. Goto ```Manage Jenkins > Configure Clouds``` 1. Click ```Add a new cloud``` and select ```Amazon EC2 Fleet``` 1. Configure AWS credentials, or leave empty to use the EC2 instance role -1. Specify Auto Scaling Group or EC2 Spot Fleet to use +1. Specify Auto Scaling Group, EC2 Fleet, or Spot Fleet to use More information on the configuration options can be found [here](https://github.com/jenkinsci/ec2-fleet-plugin/blob/master/docs/CONFIGURATION-OPTIONS.md). @@ -195,7 +199,7 @@ You can use the History tab in the AWS console to view the scaling history. ## Groovy -Below is a Groovy script to setup EC2 Spot Fleet Plugin for Jenkins and configure it. You can +Below is a Groovy script to setup Spot Fleet Plugin for Jenkins and configure it. You can run the script with [Jenkins Script Console](https://wiki.jenkins.io/display/JENKINS/Jenkins+Script+Console). ```groovy @@ -214,8 +218,7 @@ import jenkins.model.Jenkins // just modify this config other code just logic config = [ region: "us-east-1", - // EC2 Spot Fleet ID - // or Auto Scaling Group Name + // Spot Fleet ID, EC2 Fleet ID, or Auto Scaling Group Name fleetId: "...", idleMinutes: 10, minSize: 0, @@ -246,7 +249,7 @@ BasicSSHUserPrivateKey instanceCredentials = new BasicSSHUserPrivateKey( "my private key to ssh ec2 for jenkins" ) // find detailed information about parameters on plugin config page or -// https://github.com/jenkinsci/ec2-fleet-plugin/blob/master/src/main/java/com/amazon/jenkins/ec2fleet/EC2FleetCloud.java +// https://github.com/jenkinsci/ec2-fleet-plugin/blob/master/src/main/java/com/amazon/jenkins/ec2fleet/FleetCloud.java FleetCloud fleetCloud = new FleetCloud( "", // fleetCloudName null, @@ -309,7 +312,7 @@ Linux plus Java, Maven etc. Then, when EC2 Fleet launches new EC2 instances with this AMI they will automatically get all the required software. Nice =) 1. Create a custom AMI as described [here](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html#creating-an-ami) -1. Create EC2 Spot Fleet with this AMI +2. Create EC2 Fleet or Spot Fleet with this AMI ### EC2 Instance User Data @@ -329,11 +332,11 @@ not be able to connect to the instance until the User Data script is done. More launcher can be found [here](https://github.com/jenkinsci/ssh-slaves-plugin/blob/master/doc/CONFIGURE.md). 1. Open Jenkins -1. Go to ```Manage Jenkins > Configure System``` -1. Find proper fleet configuration and click ```Advanced...``` for SSH Launcher -1. Add checking command into field ```Prefix Start Agent Command``` +2. Go to ```Manage Jenkins > Configure System``` +3. Find proper fleet configuration and click ```Advanced...``` for SSH Launcher +4. Add checking command into field ```Prefix Start Agent Command``` - example ```java -version && ``` -1. To apply for existing instances, restart Jenkins or Delete Nodes from Jenkins so they will be reconnected +5. To apply for existing instances, restart Jenkins or Delete Nodes from Jenkins so they will be reconnected # FAQ diff --git a/docs/CONFIGURATION-AS-CODE.md b/docs/CONFIGURATION-AS-CODE.md index abe69d34..e59755e6 100644 --- a/docs/CONFIGURATION-AS-CODE.md +++ b/docs/CONFIGURATION-AS-CODE.md @@ -8,32 +8,32 @@ [Definition](https://github.com/jenkinsci/ec2-fleet-plugin/blob/master/src/main/java/com/amazon/jenkins/ec2fleet/FleetCloud.java#L156-L179) -| Property | Type | Required | Description | -|----------------------------|---------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------| -| name | string | yes, default ```""``` | A unique name for Jenkins cloud. "" signals the plugin to generate a unique default name for the Cloud. e.g. FleetCloud-jBGChqOP | -| awsCredentialsId | string | no, default ```null``` | [Leave blank to use AWS EC2 instance role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html) | -| computerConnector | object | yes | for example ```sshConnector``` | -| region | string | yes | ```us-east-2```, full [list](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html) | -| fleet | string | yes | my-fleet | -| endpoint | string | no | Set only if you need to use custome endpoint ```http://a.com``` | -| fsRoot | string | no | my-root | -| privateIpUsed | boolean | no, default ```false``` | connect to EC2 instance by private id instead of public | -| alwaysReconnect | boolean | no, default ```false``` || -| labelString | string | yes || -| idleMinutes | int | no, default ```0``` || -| minSize | int | no, default ```0``` || -| maxSize | int | no, default ```0``` || -| minSpareSize | int | no, default ```0``` || minimum number of instances allowed to be idle, ready to pickup work. maxSize overrides minSpareSize. Such instances are exempted from 'Max Idle Minutes Before Scaledown' config. -| maxTotalUses | int | no, default ```-1``` i.e. unlimited uses || maximum number of times a node can be used. Overrides minSize and minSpareSize, if set. -| numExecutors | int | no, default ```1``` || -| addNodeOnlyIfRunning | boolean | no, default ```false``` || -| restrictUsage | boolean | no, default ```false``` | if ```true``` fleet nodes will executed only jobs with same label | -| scaleExecutorsByWeight | boolean | no, default ```false``` || -| disableTaskResubmit | boolean | no, default ```false``` || -| initOnlineTimeoutSec | int | no, default ```180``` || -| initOnlineCheckIntervalSec | int | no, default ```15``` || -| cloudStatusIntervalSec | int | no, default ```10``` || -| noDelayProvision | boolean | no, default ```false``` || +| Property | Type | Required | Description | +|----------------------------|---------|------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------| +| name | string | yes, default ```""``` | A unique name for Jenkins cloud. "" signals the plugin to generate a unique default name for the Cloud. e.g. FleetCloud-jBGChqOP | +| awsCredentialsId | string | no, default ```null``` | [Leave blank to use AWS EC2 instance role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html) | +| computerConnector | object | yes | for example ```sshConnector``` | +| region | string | yes | ```us-east-2```, full [list](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html) | +| fleet | string | yes | my-fleet | +| endpoint | string | no | Set only if you need to use custome endpoint ```http://a.com``` | +| fsRoot | string | no | my-root | +| privateIpUsed | boolean | no, default ```false``` | connect to EC2 instance by private id instead of public | +| alwaysReconnect | boolean | no, default ```false``` | | +| labelString | string | yes | | +| idleMinutes | int | no, default ```0``` | | +| minSize | int | no, default ```0``` | | +| maxSize | int | no, default ```0``` | | +| minSpareSize | int | no, default ```0``` | | minimum number of instances allowed to be idle, ready to pickup work. maxSize overrides minSpareSize. Such instances are exempted from 'Max Idle Minutes Before Scaledown' config. +| maxTotalUses | int | no, default ```-1``` i.e. unlimited uses | | maximum number of times a node can be used. Overrides minSize and minSpareSize, if set. +| numExecutors | int | no, default ```1``` | | +| addNodeOnlyIfRunning | boolean | no, default ```false``` | | +| restrictUsage | boolean | no, default ```false``` | if ```true``` fleet nodes will executed only jobs with same label | +| executorScaler | object | no, default ```NoScaler``` | Can be set to noScaler, weightedScaler, or nodeHardwareScaler. nodeHardwareScaler has subfields of vCpuPerExecutor and memoryGiBPerExecutor | +| disableTaskResubmit | boolean | no, default ```false``` | | +| initOnlineTimeoutSec | int | no, default ```180``` | | +| initOnlineCheckIntervalSec | int | no, default ```15``` | | +| cloudStatusIntervalSec | int | no, default ```10``` | | +| noDelayProvision | boolean | no, default ```false``` | | ## FleetLabelCloud @@ -85,7 +85,10 @@ jenkins: numExecutors: 12 addNodeOnlyIfRunning: true restrictUsage: true - scaleExecutorsByWeight: true + executorScaler: + nodeHardwareScaler: + memoryGiBPerExecutor: 2 + vCpuPerExecutor: 3 initOnlineTimeoutSec: 181 initOnlineCheckIntervalSec: 13 cloudStatusIntervalSec: 11 diff --git a/docs/CONFIGURATION-OPTIONS.md b/docs/CONFIGURATION-OPTIONS.md index 7bdaa518..9f67ee45 100644 --- a/docs/CONFIGURATION-OPTIONS.md +++ b/docs/CONFIGURATION-OPTIONS.md @@ -1,29 +1,29 @@ # EC2 Fleet -Parameter | Description | Default ---- | --- | --- -Name | The name of the cloud for Jenkins use. | FleetCloud -AWS Credentials | AWS Credentials used for connecting to instances, provisioning new ones, etc. | -Region | AWS region the Fleet or ASG will reside in. | ap-east-1 (first item in alphabetical list) -Endpoint | AWS regional endpoint. Optional field only necessary if the desired region is not present in the region dropdown menu. | -EC2 Fleet | Spot Fleet or ASG used for this Jenkins cloud. | -Show all fleets | Show all Spot fleets in the region, not just the ones that are active and use maintain target capacity. | disabled -Launcher | Options for connecting to new instances and creating Jenkins nodes from them. | -Private IP | Connect to an instance using its private ip address instead of its public ip address. | disabled (use public IP) -Always Reconnect | Reconnect to offline nodes unless they've been terminated. | disabled -Restrict usage | Only allow jobs that have a matching label to use this fleet. If unchecked, the fleet will be considered for any pending job. | disabled -Label | The string value of the label that will restrict job placement. | -Jenkins Filesystem Root | Location of the Jenkins Filesystem Root. If left blank, a new directory will be made under the default directory. | `/tmp/jenkins-` -Number of Executors | The number of executors per instance. Changing the number of executors will only affect future instances and existing instances (if any) will remain unaffected. | 1 -Scale Executors by Weight | Multiply the number of executors on an instance by the weight of the instance in the Spot Fleet. Good for Spot Fleets with instances of various sizes. | disabled -Max Idle Minutes Before Scaledown | After this amount of time an idle instance will be scheduled for termination. **If set to 0, instances will never be terminated**. | 0 -Minimum Cluster Size | The lower limit on the number of instances the fleet can have. Uses instance weight, when applicable. Should be set equal to the ASG or Spot Fleet max instances. | 1 -Maximum Cluster Size | The upper limit on the number of instances the fleet can have. Uses instance weight, when applicable. Should be set equal to the ASG or Spot Fleet max instances. | 1 -Maximum Total Uses | Set a maximum total uses allowed for instances. After running 'maximum total uses' amount of jobs, the instance would be terminated. Set -1 for unlimited | -1 -Disable Build Resubmit | Do not automatically resubmit jobs that were interrupted due to an instance termination (manual termination, Spot interruption, etc.) | disabled -Maximum Init Connection Timeout in sec | EC2 instances aren't ready immediately after they're provisioned. They must become active and complete any userdata script. If that process takes longer than the time set here, consider that EC2 instance lost. | 180 -Cloud Status Interval in sec | How long to wait between update cycles. Shorter times enable the fleet to scale faster, but cause more API calls. | 10 -No Delay Provision Strategy | The default Jenkins strategy scales exponentially, meaning it might take a few cycles before all the pending jobs are provisioned. The "No Delay Provisioning Strategy" tries to get enough executors for all pending jobs in a single cycle. | disabled +Parameter | Description | Default +--- |---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------- +Name | The name of the cloud for Jenkins use. | FleetCloud +AWS Credentials | AWS Credentials used for connecting to instances, provisioning new ones, etc. | +Region | AWS region the Fleet or ASG will reside in. | ap-east-1 (first item in alphabetical list) +Endpoint | AWS regional endpoint. Optional field only necessary if the desired region is not present in the region dropdown menu. | +EC2 Fleet | Spot Fleet or ASG used for this Jenkins cloud. | +Show all fleets | Show all Spot fleets in the region, not just the ones that are active and use maintain target capacity. | disabled +Launcher | Options for connecting to new instances and creating Jenkins nodes from them. | +Private IP | Connect to an instance using its private ip address instead of its public ip address. | disabled (use public IP) +Always Reconnect | Reconnect to offline nodes unless they've been terminated. | disabled +Restrict usage | Only allow jobs that have a matching label to use this fleet. If unchecked, the fleet will be considered for any pending job. | disabled +Label | The string value of the label that will restrict job placement. | +Jenkins Filesystem Root | Location of the Jenkins Filesystem Root. If left blank, a new directory will be made under the default directory. | `/tmp/jenkins-` +Number of Executors | The number of executors per instance. Changing the number of executors will only affect future instances and existing instances (if any) will remain unaffected. | 1 +Executor Scaler | Scales the number of executors. No Scaler does not scale. Weighted Scaler scales by multiplying the number of executors on an instance by the weight of the instance. Node Hardware Scaler scales by the vCPU count or the memory of the instance| No Scaler +Max Idle Minutes Before Scaledown | After this amount of time an idle instance will be scheduled for termination. **If set to 0, instances will never be terminated**. | 0 +Minimum Cluster Size | The lower limit on the number of instances the fleet can have. Uses instance weight, when applicable. Should be set equal to the ASG or Spot Fleet max instances. | 1 +Maximum Cluster Size | The upper limit on the number of instances the fleet can have. Uses instance weight, when applicable. Should be set equal to the ASG or Spot Fleet max instances. | 1 +Maximum Total Uses | Set a maximum total uses allowed for instances. After running 'maximum total uses' amount of jobs, the instance would be terminated. Set -1 for unlimited | -1 +Disable Build Resubmit | Do not automatically resubmit jobs that were interrupted due to an instance termination (manual termination, Spot interruption, etc.) | disabled +Maximum Init Connection Timeout in sec | EC2 instances aren't ready immediately after they're provisioned. They must become active and complete any userdata script. If that process takes longer than the time set here, consider that EC2 instance lost. | 180 +Cloud Status Interval in sec | How long to wait between update cycles. Shorter times enable the fleet to scale faster, but cause more API calls. | 10 +No Delay Provision Strategy | The default Jenkins strategy scales exponentially, meaning it might take a few cycles before all the pending jobs are provisioned. The "No Delay Provisioning Strategy" tries to get enough executors for all pending jobs in a single cycle. | disabled # EC2 Fleet Label Based diff --git a/src/main/java/com/amazon/jenkins/ec2fleet/FleetCloud.java b/src/main/java/com/amazon/jenkins/ec2fleet/FleetCloud.java index 3fe70638..7f147a53 100644 --- a/src/main/java/com/amazon/jenkins/ec2fleet/FleetCloud.java +++ b/src/main/java/com/amazon/jenkins/ec2fleet/FleetCloud.java @@ -5,22 +5,13 @@ import com.amazon.jenkins.ec2fleet.fleet.Fleet; import com.amazon.jenkins.ec2fleet.fleet.Fleets; import com.amazonaws.services.ec2.AmazonEC2; -import com.amazonaws.services.ec2.model.Instance; -import com.amazonaws.services.ec2.model.InstanceStateName; +import com.amazonaws.services.ec2.model.*; import com.cloudbees.jenkins.plugins.awscredentials.AWSCredentialsHelper; import com.google.common.collect.Sets; import hudson.Extension; -import hudson.model.Computer; -import hudson.model.Descriptor; -import hudson.model.Failure; -import hudson.model.Label; -import hudson.model.Node; -import hudson.model.Queue; -import hudson.model.TaskListener; -import hudson.slaves.Cloud; -import hudson.slaves.ComputerConnector; -import hudson.slaves.NodeProperty; -import hudson.slaves.NodeProvisioner; +import hudson.ExtensionPoint; +import hudson.model.*; +import hudson.slaves.*; import hudson.util.FormValidation; import hudson.util.ListBoxModel; import hudson.util.ListBoxModel.Option; @@ -29,6 +20,7 @@ import org.apache.commons.lang.StringUtils; import org.jenkinsci.Symbol; import org.kohsuke.stapler.DataBoundConstructor; +import org.kohsuke.stapler.DataBoundSetter; import org.kohsuke.stapler.QueryParameter; import org.kohsuke.stapler.StaplerRequest; @@ -124,7 +116,7 @@ public class FleetCloud extends AbstractFleetCloud { private final int numExecutors; private final boolean addNodeOnlyIfRunning; private final boolean restrictUsage; - private final boolean scaleExecutorsByWeight; + private final ExecutorScaler executorScaler; private final Integer initOnlineTimeoutSec; private final Integer initOnlineCheckIntervalSec; private final Integer cloudStatusIntervalSec; @@ -186,9 +178,9 @@ public FleetCloud(@Nonnull final String name, final boolean disableTaskResubmit, final Integer initOnlineTimeoutSec, final Integer initOnlineCheckIntervalSec, - final boolean scaleExecutorsByWeight, final Integer cloudStatusIntervalSec, - final boolean noDelayProvision) { + final boolean noDelayProvision, + final ExecutorScaler executorScaler) { super(StringUtils.isNotBlank(name) ? name : CloudNames.generateUnique(BASE_DEFAULT_FLEET_CLOUD_ID)); init(); this.credentialsId = credentialsId; @@ -212,13 +204,13 @@ public FleetCloud(@Nonnull final String name, this.numExecutors = Math.max(numExecutors, 1); this.addNodeOnlyIfRunning = addNodeOnlyIfRunning; this.restrictUsage = restrictUsage; - this.scaleExecutorsByWeight = scaleExecutorsByWeight; this.disableTaskResubmit = disableTaskResubmit; this.initOnlineTimeoutSec = initOnlineTimeoutSec; this.initOnlineCheckIntervalSec = initOnlineCheckIntervalSec; this.cloudStatusIntervalSec = cloudStatusIntervalSec; this.noDelayProvision = noDelayProvision; - + this.executorScaler = executorScaler == null ? new NoScaler().withNumExecutors(this.numExecutors) : + executorScaler.withNumExecutors(this.numExecutors); if (fleet != null) { this.stats = Fleets.get(fleet).getState( getAwsCredentialsId(), region, endpoint, getFleet()); @@ -322,8 +314,8 @@ public int getNumExecutors() { return numExecutors; } - public boolean isScaleExecutorsByWeight() { - return scaleExecutorsByWeight; + public ExecutorScaler getExecutorScaler() { + return this.executorScaler; } public String getJvmSettings() { @@ -833,13 +825,7 @@ private void addNewAgent(final AmazonEC2 ec2, final Instance instance, FleetStat effectiveFsRoot = fsRoot; } - final Double instanceTypeWeight = stats.getInstanceTypeWeights().get(instance.getInstanceType()); - final int effectiveNumExecutors; - if (scaleExecutorsByWeight && instanceTypeWeight != null) { - effectiveNumExecutors = (int) Math.max(Math.round(numExecutors * instanceTypeWeight), 1); - } else { - effectiveNumExecutors = numExecutors; - } + int effectiveNumExecutors = this.executorScaler.scale(instance.getInstanceType(), stats, ec2); final FleetAutoResubmitComputerLauncher computerLauncher = new FleetAutoResubmitComputerLauncher( computerConnector.launch(address, TaskListener.NULL)); @@ -945,6 +931,10 @@ public List getComputerConnectorDescriptors() { return Jenkins.get().getDescriptorList(ComputerConnector.class); } + public List getExecutorScalerDescriptors() { + return Jenkins.get().getDescriptorList(ExecutorScaler.class); + } + public ListBoxModel doFillAwsCredentialsIdItems() { return AWSCredentialsHelper.doFillCredentialsIdItems(Jenkins.get()); } @@ -1044,4 +1034,123 @@ public boolean configure(final StaplerRequest req, final JSONObject formData) th return super.configure(req, formData); } } + + public static abstract class ExecutorScaler extends AbstractDescribableImpl implements ExtensionPoint { + + protected int numExecutors = 1; + + protected ExecutorScaler(){} + + public abstract int scale(String instanceType, FleetStateStats stats, AmazonEC2 ec2); + + public ExecutorScaler withNumExecutors(int numExecutors) { + setNumExecutors(numExecutors); + return this; + } + + private void setNumExecutors(int numExecutors) { + this.numExecutors = numExecutors; + } + + public ExecutorScaleDescriptor getDescriptor() { return (ExecutorScaleDescriptor) super.getDescriptor();} + } + + public static class ExecutorScaleDescriptor extends Descriptor {} + + public static class NoScaler extends ExecutorScaler { + + @DataBoundConstructor + public NoScaler() {} + + @Override + public int scale(String instanceType, FleetStateStats stats, AmazonEC2 ec2) { return numExecutors; } + + @Extension + public static class DescriptorImpl extends ExecutorScaleDescriptor { + @Override + public String getDisplayName() { return "No scaling"; } + } + } + + public static class WeightedScaler extends ExecutorScaler { + + @DataBoundConstructor + public WeightedScaler() {} + @Override + public int scale(String instanceType, FleetStateStats stats, AmazonEC2 ec2) { + if(stats == null) { + return numExecutors; + } + + final Double instanceTypeWeight = stats.getInstanceTypeWeights().get(instanceType); + if (instanceTypeWeight == null) { + return numExecutors; + } + return (int) Math.max(Math.round(numExecutors * instanceTypeWeight), 1); + } + + @Extension + public static class DescriptorImpl extends ExecutorScaleDescriptor { + @Override + public String getDisplayName() { return "Scale by weight";} + } + } + + public static class NodeHardwareScaler extends ExecutorScaler { + private int vCpuPerExecutor; + private int memoryGiBPerExecutor; + + @DataBoundConstructor + public NodeHardwareScaler(int vCpuPerExecutor, int memoryGiBPerExecutor) { + this.vCpuPerExecutor = vCpuPerExecutor; + this.memoryGiBPerExecutor = memoryGiBPerExecutor; + } + + @DataBoundSetter + public void setvCpuPerExecutor(int value) { this.vCpuPerExecutor = value; } + + @DataBoundSetter + public void setMemoryGiBPerExecutor(int value) { this.memoryGiBPerExecutor = value; } + + public int getvCpuPerExecutor() { + return vCpuPerExecutor; + } + + public int getMemoryGiBPerExecutor() { + return memoryGiBPerExecutor; + } + + @Override + public int scale(final String instanceType, final FleetStateStats stats, final AmazonEC2 ec2) { + if(this.vCpuPerExecutor == 0 && this.memoryGiBPerExecutor == 0) { + return numExecutors; + } + + int vCPUNumExecutors = Integer.MAX_VALUE; + int memoryNumExecutors = Integer.MAX_VALUE; + InstanceTypeInfo instanceTypeInfo = getInstanceTypeInfo(ec2, instanceType); + if(this.vCpuPerExecutor != 0) { + int instanceVCPUs = instanceTypeInfo.getVCpuInfo().getDefaultVCpus(); + vCPUNumExecutors = Math.max(instanceVCPUs/this.vCpuPerExecutor, 1); + } + + if(this.memoryGiBPerExecutor != 0) { + long instanceMemory = instanceTypeInfo.getMemoryInfo().getSizeInMiB()/1024; + memoryNumExecutors = Math.max(Math.round((float) instanceMemory /this.memoryGiBPerExecutor), 1); + } + return Math.min(vCPUNumExecutors, memoryNumExecutors); + } + + @Extension + public static class DescriptorImpl extends ExecutorScaleDescriptor { + @Override + public String getDisplayName() { return "Scale by node hardware";} + } + + private InstanceTypeInfo getInstanceTypeInfo(final AmazonEC2 ec2, final String instanceType) { + DescribeInstanceTypesRequest request = new DescribeInstanceTypesRequest().withInstanceTypes(instanceType); + DescribeInstanceTypesResult result = ec2.describeInstanceTypes(request); + return result.getInstanceTypes().get(0); + } + } } diff --git a/src/main/java/com/amazon/jenkins/ec2fleet/NoDelayProvisionStrategy.java b/src/main/java/com/amazon/jenkins/ec2fleet/NoDelayProvisionStrategy.java index d059d478..cbeb3049 100644 --- a/src/main/java/com/amazon/jenkins/ec2fleet/NoDelayProvisionStrategy.java +++ b/src/main/java/com/amazon/jenkins/ec2fleet/NoDelayProvisionStrategy.java @@ -58,7 +58,7 @@ public NodeProvisioner.StrategyDecision apply(final NodeProvisioner.StrategyStat Cloud.CloudState cloudState = new Cloud.CloudState(label, strategyState.getAdditionalPlannedCapacity()); if (!c.canProvision(cloudState)) { - LOGGER.log(Level.INFO, "label [{0}]: cloud {1} can not provision for this label, continuing...", + LOGGER.log(Level.FINE, "label [{0}]: cloud {1} can not provision for this label, continuing...", new Object[]{label, c.getDisplayName()}); continue; } diff --git a/src/main/java/com/amazon/jenkins/ec2fleet/aws/AwsPermissionChecker.java b/src/main/java/com/amazon/jenkins/ec2fleet/aws/AwsPermissionChecker.java index 92c94488..4bae3e03 100644 --- a/src/main/java/com/amazon/jenkins/ec2fleet/aws/AwsPermissionChecker.java +++ b/src/main/java/com/amazon/jenkins/ec2fleet/aws/AwsPermissionChecker.java @@ -32,9 +32,10 @@ public enum FleetAPI { ModifySpotFleetRequest, DescribeSpotFleetRequests, DescribeAutoScalingGroups, - DescribeEC2FleetRequests, - DescribeEC2FleetInstances, - ModifyEC2FleetRequest, + DescribeFleets, + DescribeFleetInstances, + ModifyFleet, + DescribeInstanceTypes, TerminateInstances, // TODO: Dry-run throws invalid instanceID first then AuthZ error. We need to find a better way to test UpdateAutoScalingGroup; // TODO: There is no dry-run for AutoScalingClient }; @@ -78,6 +79,9 @@ private List getMissingCommonPermissions(final AmazonEC2 ec2Client) { if(!hasCreateTagsPermissions(ec2Client)) { missingCommonPermissions.add(FleetAPI.CreateTags.name()); } + if(!hasDescribeInstanceTypesPermission(ec2Client)) { + missingCommonPermissions.add(FleetAPI.DescribeInstanceTypes.name()); + } return missingCommonPermissions; } @@ -93,13 +97,13 @@ private List getMissingPermissionsForASG() { private List getMissingPermissionsForEC2Fleet(final AmazonEC2 ec2Client, final String fleet) { final List missingFleetPermissions = new ArrayList<>(); if(!hasDescribeEC2FleetRequestsPermission(ec2Client, fleet)) { - missingFleetPermissions.add(FleetAPI.DescribeEC2FleetRequests.name()); + missingFleetPermissions.add(FleetAPI.DescribeFleets.name()); } if(!hasDescribeEC2FleetInstancesPermission(ec2Client, fleet)) { - missingFleetPermissions.add(FleetAPI.DescribeEC2FleetInstances.name()); + missingFleetPermissions.add(FleetAPI.DescribeFleetInstances.name()); } if(!hasModifyEC2FleetRequestPermission(ec2Client, fleet)) { - missingFleetPermissions.add(FleetAPI.ModifyEC2FleetRequest.name()); + missingFleetPermissions.add(FleetAPI.ModifyFleet.name()); } return missingFleetPermissions; } @@ -152,4 +156,9 @@ private boolean hasDescribeInstancePermission(final AmazonEC2 ec2Client) { final DryRunResult dryRunResult = ec2Client.dryRun(new DescribeInstancesRequest()); return dryRunResult.getDryRunResponse().getStatusCode() != UNAUTHORIZED_STATUS_CODE; } + + private boolean hasDescribeInstanceTypesPermission(final AmazonEC2 ec2Client) { + final DryRunResult dryRunResult = ec2Client.dryRun(new DescribeInstanceTypesRequest()); + return dryRunResult.getDryRunResponse().getStatusCode() != UNAUTHORIZED_STATUS_CODE; + } } diff --git a/src/main/resources/com/amazon/jenkins/ec2fleet/FleetCloud/NodeHardwareScaler/config.jelly b/src/main/resources/com/amazon/jenkins/ec2fleet/FleetCloud/NodeHardwareScaler/config.jelly new file mode 100644 index 00000000..ca96cab4 --- /dev/null +++ b/src/main/resources/com/amazon/jenkins/ec2fleet/FleetCloud/NodeHardwareScaler/config.jelly @@ -0,0 +1,9 @@ + + + + + + + + + \ No newline at end of file diff --git a/src/main/resources/com/amazon/jenkins/ec2fleet/FleetCloud/NodeHardwareScaler/help-memoryGiBPerExecutor.html b/src/main/resources/com/amazon/jenkins/ec2fleet/FleetCloud/NodeHardwareScaler/help-memoryGiBPerExecutor.html new file mode 100644 index 00000000..a91903c8 --- /dev/null +++ b/src/main/resources/com/amazon/jenkins/ec2fleet/FleetCloud/NodeHardwareScaler/help-memoryGiBPerExecutor.html @@ -0,0 +1,17 @@ +

Used to set the number of executors based on the Node's memory(GiB)

+ +
+

Number of Executors = + + + GiB of memory + Memory(GiB) Per Executor + + +

+
+ +

For example, if an EC2 Instance with 8GiB of memory is used as a Node, and the Memory(GiB) Per Executor is set to 2, then +the number of executors will be set to 4.

+ + diff --git a/src/main/resources/com/amazon/jenkins/ec2fleet/FleetCloud/NodeHardwareScaler/help-vCpuPerExecutor.html b/src/main/resources/com/amazon/jenkins/ec2fleet/FleetCloud/NodeHardwareScaler/help-vCpuPerExecutor.html new file mode 100644 index 00000000..c90bade7 --- /dev/null +++ b/src/main/resources/com/amazon/jenkins/ec2fleet/FleetCloud/NodeHardwareScaler/help-vCpuPerExecutor.html @@ -0,0 +1,15 @@ +

Used to set the number of executors based on the Node's memory(GiB)

+ +
+

Number of Executors = + + + Number of vCPUs + vCPU(s) Per Executor + + +

+
+ +

For example, if an EC2 Instance with a vCPU count of 8 is used as a Node, and the vCPU(s) Per Executor is set to 4, then + the number of executors will be set to 2.

\ No newline at end of file diff --git a/src/main/resources/com/amazon/jenkins/ec2fleet/FleetCloud/config.jelly b/src/main/resources/com/amazon/jenkins/ec2fleet/FleetCloud/config.jelly index 87e5af98..19501437 100644 --- a/src/main/resources/com/amazon/jenkins/ec2fleet/FleetCloud/config.jelly +++ b/src/main/resources/com/amazon/jenkins/ec2fleet/FleetCloud/config.jelly @@ -79,14 +79,13 @@ - Number of executors per instance + Testing Number of executors per instance - - - + + How long to keep an idle node. If set to 0, never scale down diff --git a/src/main/resources/com/amazon/jenkins/ec2fleet/FleetCloud/help-executorScaler.html b/src/main/resources/com/amazon/jenkins/ec2fleet/FleetCloud/help-executorScaler.html new file mode 100644 index 00000000..a229e132 --- /dev/null +++ b/src/main/resources/com/amazon/jenkins/ec2fleet/FleetCloud/help-executorScaler.html @@ -0,0 +1,71 @@ +

Plugin always set number of executors to at least one.

+
No Scaling
+

Doesn't scale. Uses set number of executors

+ +
Scale by node hardware
+

Determine number of executors by node hardware: Memory and vCPU count. Specify the quantity of each resource to be + allocated per executor. The plugin sources the resources of the node and calculates the desired number of executors. + The lower number of executors calculated will be chosen to prevent over provisioning. +

+ +
Scale by weight
+ +

+ The plugin consumes instance + weight information provided by a Launch Specification + and uses it to scale the node's number of executors from configuration field Number of Executors. + Note: current implementation doesn't support Launch Template, only Launch Specification. +

+ +

+ Example (here instance type from launch specification matches with + launched instance type): + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Number of ExecutorsInstance WeightEffective
Number of Executors
111
10.51
10.11
100.11
11.52
11.441
+

+ +

+ If the launched instance type doesn't match any weight in launch specification, + regular number of executors will be used without any scaling. +

+ diff --git a/src/test/java/com/amazon/jenkins/ec2fleet/AutoResubmitIntegrationTest.java b/src/test/java/com/amazon/jenkins/ec2fleet/AutoResubmitIntegrationTest.java index 73c4141b..06a9a537 100644 --- a/src/test/java/com/amazon/jenkins/ec2fleet/AutoResubmitIntegrationTest.java +++ b/src/test/java/com/amazon/jenkins/ec2fleet/AutoResubmitIntegrationTest.java @@ -43,6 +43,8 @@ @SuppressWarnings({"deprecation"}) public class AutoResubmitIntegrationTest extends IntegrationTest { + private FleetCloud.ExecutorScaler noScaling; + @Before public void before() { Fleet fleet = mock(Fleet.class); @@ -69,6 +71,8 @@ public void before() { new Reservation().withInstances( instance ))); + + noScaling = new FleetCloud.NoScaler(); } @Test @@ -76,8 +80,8 @@ public void should_successfully_resubmit_freestyle_task() throws Exception { FleetCloud cloud = new FleetCloud("TestCloud", "credId", null, "region", null, "fId", "momo", null, new LocalComputerConnector(j), false, false, 0, 0, 10, 0, 1, false, true, - "-1", false, 0, 0, false, - 10, false); + "-1", false, 0, 0, + 10, false, noScaling); j.jenkins.clouds.add(cloud); List rs = enqueTask(1); @@ -112,8 +116,8 @@ public void should_successfully_resubmit_parametrized_task() throws Exception { FleetCloud cloud = new FleetCloud("TestCloud", "credId", null, "region", null, "fId", "momo", null, new LocalComputerConnector(j), false, false, 0, 0, 10, 0, 1, false, true, - "-1", false, 0, 0, false, - 10, false); + "-1", false, 0, 0, + 10, false, noScaling); j.jenkins.clouds.add(cloud); List rs = new ArrayList<>(); @@ -168,7 +172,7 @@ public void should_not_resubmit_if_disabled() throws Exception { FleetCloud cloud = new FleetCloud("TestCloud", "credId", null, "region", null, "fId", "momo", null, new LocalComputerConnector(j), false, false, 0, 0, 10, 0, 1, false, true, - "-1", true, 0, 0, false, 10, false); + "-1", true, 0, 0, 10, false, noScaling); j.jenkins.clouds.add(cloud); List rs = enqueTask(1); diff --git a/src/test/java/com/amazon/jenkins/ec2fleet/CloudNamesTest.java b/src/test/java/com/amazon/jenkins/ec2fleet/CloudNamesTest.java index b3c4d845..80d8cdfb 100644 --- a/src/test/java/com/amazon/jenkins/ec2fleet/CloudNamesTest.java +++ b/src/test/java/com/amazon/jenkins/ec2fleet/CloudNamesTest.java @@ -9,13 +9,15 @@ public class CloudNamesTest { @Rule public JenkinsRule j = new JenkinsRule(); + private final FleetCloud.ExecutorScaler noScaling = new FleetCloud.NoScaler(); + @Test public void isUnique_true() { j.jenkins.clouds.add(new FleetCloud("SomeDefaultName", null, null, null, null, null, "test-label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false)); + "-1", false, 0, 0, + 10, false, noScaling)); Assert.assertTrue(CloudNames.isUnique("TestCloud")); } @@ -25,8 +27,8 @@ public void isUnique_false() { j.jenkins.clouds.add(new FleetCloud("SomeDefaultName", null, null, null, null, null, "test-label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false)); + "-1", false, 0, 0, + 10, false, noScaling)); Assert.assertFalse(CloudNames.isUnique("SomeDefaultName")); } @@ -36,14 +38,14 @@ public void isDuplicated_false() { j.jenkins.clouds.add(new FleetCloud("TestCloud", null, null, null, null, null, "test-label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false)); + "-1", false, 0, 0, + 10, false, noScaling)); j.jenkins.clouds.add(new FleetCloud("TestCloud2", null, null, null, null, null, "test-label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false)); + "-1", false, 0, 0, + 10, false, noScaling)); Assert.assertFalse(CloudNames.isDuplicated("TestCloud")); } @@ -53,14 +55,14 @@ public void isDuplicated_true() { j.jenkins.clouds.add(new FleetCloud("TestCloud", null, null, null, null, null, "test-label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false)); + "-1", false, 0, 0, + 10, false, noScaling)); j.jenkins.clouds.add(new FleetCloud("TestCloud", null, null, null, null, null, "test-label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false)); + "-1", false, 0, 0, + 10, false, noScaling)); Assert.assertTrue(CloudNames.isDuplicated("TestCloud")); } @@ -75,8 +77,8 @@ public void generateUnique_addsSuffixOnlyWhenNeeded() { j.jenkins.clouds.add(new FleetCloud("UniqueCloud-1", null, null, null, null, null, "test-label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false)); + "-1", false, 0, 0, + 10, false, noScaling)); Assert.assertEquals("UniqueCloud", CloudNames.generateUnique("UniqueCloud")); } @@ -86,13 +88,14 @@ public void generateUnique_addsSuffixCorrectly() { j.jenkins.clouds.add(new FleetCloud("UniqueCloud", null, null, null, null, null, "test-label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false)); + "-1", false, 0, 0, + 10, false, noScaling)); + j.jenkins.clouds.add(new FleetCloud("UniqueCloud-1", null, null, null, null, null, "test-label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false)); + "-1", false, 0, 0, + 10, false, noScaling)); String actual = CloudNames.generateUnique("UniqueCloud"); Assert.assertTrue(actual.length() == ("UniqueCloud".length() + CloudNames.SUFFIX_LENGTH + 1)); @@ -104,8 +107,9 @@ public void generateUnique_emptyStringInConstructor() { FleetCloud fleetCloud = new FleetCloud("", null, null, null, null, null, "test-label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false); + "-1", false, 0, 0, + 10, false, noScaling); + FleetLabelCloud fleetLabelCloud = new FleetLabelCloud("", null, null, null, null, new LocalComputerConnector(j), false, false, 0, 0, 0, 1, false, @@ -123,8 +127,9 @@ public void generateUnique_nonEmptyStringInConstructor() { FleetCloud fleetCloud = new FleetCloud("UniqueCloud", null, null, null, null, null, "test-label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false); + "-1", false, 0, 0, + 10, false, noScaling); + FleetLabelCloud fleetLabelCloud = new FleetLabelCloud("UniqueLabelCloud", null, null, null, null, new LocalComputerConnector(j), false, false, 0, 0, 0, 1, false, diff --git a/src/test/java/com/amazon/jenkins/ec2fleet/EC2RetentionStrategyIntegrationTest.java b/src/test/java/com/amazon/jenkins/ec2fleet/EC2RetentionStrategyIntegrationTest.java index e4b218f8..ce472743 100644 --- a/src/test/java/com/amazon/jenkins/ec2fleet/EC2RetentionStrategyIntegrationTest.java +++ b/src/test/java/com/amazon/jenkins/ec2fleet/EC2RetentionStrategyIntegrationTest.java @@ -37,6 +37,8 @@ public class EC2RetentionStrategyIntegrationTest extends IntegrationTest { + private final FleetCloud.ExecutorScaler noScaling = new FleetCloud.NoScaler(); + private AmazonEC2 amazonEC2; @Before @@ -72,7 +74,7 @@ public void before() { public void shouldTerminateNodeMarkedForDeletion() throws Exception { final FleetCloud cloud = new FleetCloud("TestCloud", "credId", null, "region", null, "fId", "momo", null, new LocalComputerConnector(j), false, false, - 1, 0, 0, 0, 1, false, true, "-1", false, 0, 0, false, 999, false); + 1, 0, 0, 0, 1, false, true, "-1", false, 0, 0, 999, false, noScaling); // Set initial jenkins nodes cloud.update(); j.jenkins.clouds.add(cloud); @@ -96,7 +98,7 @@ public void shouldTerminateNodeMarkedForDeletion() throws Exception { public void shouldTerminateExcessCapacity() throws Exception { final FleetCloud cloud = new FleetCloud("TestCloud", "credId", null, "region", null, "fId", "momo", null, new LocalComputerConnector(j), false, false, - 1, 0, 0, 0, 1, false, true, "-1", false, 0, 0, false, 999, false); + 1, 0, 0, 0, 1, false, true, "-1", false, 0, 0, 999, false, noScaling); // Set initial jenkins nodes cloud.update(); j.jenkins.clouds.add(cloud); @@ -135,14 +137,14 @@ public void shouldNotTerminateExcessCapacityWhenNodeIsBusy() throws Exception { FleetCloud cloud = new FleetCloud("TestCloud", "credId", null, "region", null, "fId", "momo", null, new LocalComputerConnector(j), false, false, - 1, 2, 2, 0, 1, false, true, "-1", false, 0, 0, false, 999, false); + 1, 2, 2, 0, 1, false, true, "-1", false, 0, 0, 999, false, noScaling); j.jenkins.clouds.add(cloud); cloud.update(); assertAtLeastOneNode(); cloud = new FleetCloud("TestCloud", "credId", null, "region", null, "fId", "momo", null, new LocalComputerConnector(j), false, false, - 1, 0, 0, 0, 1, false, true, "-1", false, 0, 0, false, 99, false); + 1, 0, 0, 0, 1, false, true, "-1", false, 0, 0, 99, false, noScaling); j.jenkins.clouds.clear(); j.jenkins.clouds.add(cloud); assertAtLeastOneNode(); @@ -167,7 +169,7 @@ public void shouldNotTerminateExcessCapacityWhenNodeIsBusy() throws Exception { public void shouldTerminateIdleNodesAfterIdleTimeout() throws Exception { final FleetCloud cloud = new FleetCloud("TestCloud", "credId", null, "region", null, "fId", "momo", null, new LocalComputerConnector(j), false, false, - 1, 0, 2, 0, 1, false, true, "-1", false, 0, 0, false, 99, false); + 1, 0, 2, 0, 1, false, true, "-1", false, 0, 0, 99, false, noScaling); j.jenkins.clouds.add(cloud); cloud.update(); @@ -198,7 +200,7 @@ public void shouldTerminateIdleNodesAfterIdleTimeout() throws Exception { public void shouldNotTerminateBelowMinSize() throws Exception { final FleetCloud cloud = new FleetCloud("TestCloud", "credId", null, "region", null, "fId", "momo", null, new LocalComputerConnector(j), false, false, - 1, 2, 5, 0, 1, false, true, "-1", false, 0, 0, false, 30, false); + 1, 2, 5, 0, 1, false, true, "-1", false, 0, 0, 30, false, noScaling); j.jenkins.clouds.add(cloud); cloud.update(); @@ -222,7 +224,7 @@ public void shouldNotTerminateBelowMinSize() throws Exception { public void shouldNotTerminateBelowMinSpareSize() throws Exception { final FleetCloud cloud = new FleetCloud("TestCloud", "credId", null, "region", null, "fId", "momo", null, new LocalComputerConnector(j), false, false, - 1, 0, 5, 2, 1, false, true, "-1", false, 0, 0, false, 30, false); + 1, 0, 5, 2, 1, false, true, "-1", false, 0, 0, 30, false, noScaling); j.jenkins.clouds.add(cloud); cloud.update(); @@ -252,7 +254,7 @@ public void shouldTerminateWhenMaxTotalUsesIsExhausted() throws Exception { FleetCloud cloud = spy(new FleetCloud("testCloud", "credId", null, "region", null, "fId", label, null, new LocalComputerConnector(j), false, false, 0, 0, 10, 0, 1, false, true, - String.valueOf(maxTotalUses), true, 0, 0, false, 10, false)); + String.valueOf(maxTotalUses), true, 0, 0, 10, false, noScaling)); j.jenkins.clouds.add(cloud); cloud.update(); assertAtLeastOneNode(); diff --git a/src/test/java/com/amazon/jenkins/ec2fleet/FleetCloudConfigurationAsCodeTest.java b/src/test/java/com/amazon/jenkins/ec2fleet/FleetCloudConfigurationAsCodeTest.java index 23d2c0b7..24ce863a 100644 --- a/src/test/java/com/amazon/jenkins/ec2fleet/FleetCloudConfigurationAsCodeTest.java +++ b/src/test/java/com/amazon/jenkins/ec2fleet/FleetCloudConfigurationAsCodeTest.java @@ -64,7 +64,7 @@ public void shouldCreateCloudFromMinConfiguration() { assertEquals(cloud.getNumExecutors(), 1); assertEquals(cloud.isAddNodeOnlyIfRunning(), false); assertEquals(cloud.isRestrictUsage(), false); - assertEquals(cloud.isScaleExecutorsByWeight(), false); + assertEquals(cloud.getExecutorScaler().getClass(), FleetCloud.NoScaler.class); assertEquals(cloud.getInitOnlineTimeoutSec(), 180); assertEquals(cloud.getInitOnlineCheckIntervalSec(), 15); assertEquals(cloud.getCloudStatusIntervalSec(), 10); @@ -92,7 +92,7 @@ public void shouldCreateCloudFromMaxConfiguration() { assertEquals(cloud.getNumExecutors(), 12); assertEquals(cloud.isAddNodeOnlyIfRunning(), true); assertEquals(cloud.isRestrictUsage(), true); - assertEquals(cloud.isScaleExecutorsByWeight(), true); + assertEquals(cloud.getExecutorScaler().getClass(), FleetCloud.WeightedScaler.class); assertEquals(cloud.getInitOnlineTimeoutSec(), 181); assertEquals(cloud.getInitOnlineCheckIntervalSec(), 13); assertEquals(cloud.getCloudStatusIntervalSec(), 11); diff --git a/src/test/java/com/amazon/jenkins/ec2fleet/FleetCloudTest.java b/src/test/java/com/amazon/jenkins/ec2fleet/FleetCloudTest.java index 5a32df6f..0faff254 100644 --- a/src/test/java/com/amazon/jenkins/ec2fleet/FleetCloudTest.java +++ b/src/test/java/com/amazon/jenkins/ec2fleet/FleetCloudTest.java @@ -9,13 +9,7 @@ import com.amazonaws.regions.RegionUtils; import com.amazonaws.services.ec2.AmazonEC2; import com.amazonaws.services.ec2.AmazonEC2Client; -import com.amazonaws.services.ec2.model.BatchState; -import com.amazonaws.services.ec2.model.DescribeRegionsResult; -import com.amazonaws.services.ec2.model.FleetType; -import com.amazonaws.services.ec2.model.Instance; -import com.amazonaws.services.ec2.model.Region; -import com.amazonaws.services.ec2.model.SpotFleetRequestConfig; -import com.amazonaws.services.ec2.model.SpotFleetRequestConfigData; +import com.amazonaws.services.ec2.model.*; import hudson.ExtensionList; import hudson.model.Computer; import hudson.model.Label; @@ -109,6 +103,12 @@ public class FleetCloudTest { @Mock private FleetNodeComputer busyComputer; + private FleetCloud.ExecutorScaler noScaling; + + private FleetCloud.ExecutorScaler weightedScaling; + + private int MiB_TO_GiB_MULTIPLIER = 1024; + @Before public void before() { spotFleetRequestConfig1 = new SpotFleetRequestConfig(); @@ -149,6 +149,10 @@ public void before() { PowerMockito.when(idleComputer.isIdle()).thenReturn(true); PowerMockito.when(busyComputer.isIdle()).thenReturn(false); + + noScaling = new FleetCloud.NoScaler(); + weightedScaling = new FleetCloud.WeightedScaler(); + } @After @@ -161,8 +165,8 @@ public void canProvision_fleetIsNull(){ FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", "", null, "", null, null, false, false, 0, 0, 10, 0, 1, true, - false, "-1", false, 0, 0, false, - 10, false); + false, "-1", false, 0, 0, + 10, false, noScaling); Label label = new LabelAtom("momo"); boolean result = fleetCloud.canProvision(new Cloud.CloudState(label, 0)); @@ -174,8 +178,8 @@ public void canProvision_restrictUsageLabelIsNull(){ FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", "", "", "", null, null, false, false, 0, 0, 10, 0, 1, true, - true, "-1", false, 0, 0, false, - 10, false); + true, "-1", false, 0, 0, + 10, false, noScaling); Label label = null; boolean result = fleetCloud.canProvision(new Cloud.CloudState(label, 0)); @@ -187,8 +191,8 @@ public void canProvision_LabelNotInLabelString(){ FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", "", "", "", null, null, false, false, 0, 0, 10, 0, 1, true, - false, "-1", false, 0, 0, false, - 10, false); + false, "-1", false, 0, 0, + 10, false, noScaling); Label label = new LabelAtom("momo"); boolean result = fleetCloud.canProvision(new Cloud.CloudState(label, 0)); @@ -200,8 +204,8 @@ public void canProvision_LabelInLabelString(){ FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", "", "", "label1 momo", null, null, false, false, 0, 0, 10, 0, 1, true, - false, "-1", false, 0, 0, false, - 10, false); + false, "-1", false, 0, 0, + 10, false, noScaling); // have to mock these for the Label.parse(...) call otherwise we get an NPE when(jenkins.getLabelAtom("momo")).thenReturn(new LabelAtom("momo")); @@ -224,8 +228,8 @@ public void provision_shouldProvisionNoneWhenMaxReached() { FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", "", "", "", null, null, false, false, 0, 0, 10, 0, 1, true, - false, "-1", false, 0, 0, false, - 10, false); + false, "-1", false, 0, 0, + 10, false, noScaling); fleetCloud.setStats(new FleetStateStats("", 10, FleetStateStats.State.active(), Collections.emptySet(), Collections.emptyMap())); @@ -250,8 +254,8 @@ public void provision_shouldProvisionNoneWhenMaxReachedAndNumExecutorsMoreOne() FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", "", "", "", null, null, false, false, 0, 1, 8, 0, 3, true, - false, "-1", false, 0, 0, false, - 10, false); + false, "-1", false, 0, 0, + 10, false, noScaling); fleetCloud.setStats(new FleetStateStats("", 1, FleetStateStats.State.active(), Collections.emptySet(), Collections.emptyMap())); @@ -276,8 +280,8 @@ public void provision_shouldProvisionNoneWhenMaxReachedAndNumExecutorsMoreOne1() FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", "", "", "", null, null, false, false, 0, 1, 8, 0, 3, true, - false, "-1", false, 0, 0, false, - 10, false); + false, "-1", false, 0, 0, + 10, false, noScaling); fleetCloud.setStats(new FleetStateStats("", 7, FleetStateStats.State.active(), Collections.emptySet(), Collections.emptyMap())); @@ -302,8 +306,8 @@ public void provision_shouldProvisionNoneWhenExceedMax() { FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", "", "", "", null, null, false, false, 0, 0, 9, 0, 1, true, - false, "-1", false, 0, 0, false, - 10, false); + false, "-1", false, 0, 0, + 10, false, noScaling); fleetCloud.setStats(new FleetStateStats("", 10, FleetStateStats.State.active(), Collections.emptySet(), Collections.emptyMap())); @@ -328,8 +332,8 @@ public void provision_shouldProvisionIfBelowMax() { FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", "", "", "", null, null, false, false, 0, 0, 10, 0, 1, true, - false, "-1", false, 0, 0, false, - 10, false); + false, "-1", false, 0, 0, + 10, false, noScaling); fleetCloud.setStats(new FleetStateStats("", 5, FleetStateStats.State.active(), Collections.emptySet(), Collections.emptyMap())); @@ -354,8 +358,8 @@ public void provision_shouldProvisionNoMoreMax() { FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", "", "", "", null, null, false, false, 0, 0, 10, 0, 1, true, - false, "-1", false, 0, 0, false, - 10, false); + false, "-1", false, 0, 0, + 10, false, noScaling); fleetCloud.setStats(new FleetStateStats("", 5, FleetStateStats.State.active(), Collections.emptySet(), Collections.emptyMap())); @@ -380,8 +384,8 @@ public void provision_shouldProvisionNoMoreMaxWhenMultipleCallBeforeUpdate() { FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", "", "", "", null, null, false, false, 0, 0, 10, 0, 1, true, - false, "-1", false, 0, 0, false, - 10, false); + false, "-1", false, 0, 0, + 10, false, noScaling); fleetCloud.setStats(new FleetStateStats("", 5, FleetStateStats.State.active(), Collections.emptySet(), Collections.emptyMap())); @@ -410,8 +414,8 @@ public void provision_shouldProvisionNoneIfNotYetUpdated() { FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", "", "", "", null, null, false, false, 0, 0, 1, 0, 1, true, - false, "-1", false, 0, 0, false, - 10, false); + false, "-1", false, 0, 0, + 10, false, noScaling); // when Collection r = fleetCloud.provision(new Cloud.CloudState(null, 0), 1); @@ -433,8 +437,8 @@ public void scheduleToTerminate_shouldNotRemoveIfStatsNotUpdated() { FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", "", "", "", null, null, false, false, 0, 0, 1, 0, 1, true, - false, "-1", false, 0, 0, false, - 10, false); + false, "-1", false, 0, 0, + 10, false, noScaling); // when boolean r = fleetCloud.scheduleToTerminate("z", false, EC2AgentTerminationReason.IDLE_FOR_TOO_LONG); @@ -455,8 +459,8 @@ public void scheduleToTerminate_notRemoveIfBelowMin() { FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", "", "", "", null, null, false, false, 0, 1, 1, 0, 1, true, - false, "-1", false, 0, 0, false, - 10, false); + false, "-1", false, 0, 0, + 10, false, noScaling); fleetCloud.setStats(new FleetStateStats("", 0, FleetStateStats.State.active(), Collections.emptySet(), Collections.emptyMap())); @@ -480,8 +484,8 @@ public void scheduleToTerminate_notRemoveIfEqualMin() { FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", "", "", "", null, null, false, false, 0, 1, 1, 0, 1, true, - false, "-1", false, 0, 0, false, - 10, false); + false, "-1", false, 0, 0, + 10, false, noScaling); fleetCloud.setStats(new FleetStateStats("", 1, FleetStateStats.State.active(), Collections.singleton("z"), Collections.emptyMap())); @@ -506,8 +510,8 @@ public void scheduleToTerminate_notRemoveIfEqualMinSpare() { FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", "", "", "", null, null, false, false, 0, 0, 5, 1, 1, true, - false, "-1", false, 0, 0, false, - 10, false); + false, "-1", false, 0, 0, + 10, false, noScaling); fleetCloud.setStats(new FleetStateStats("", 1, FleetStateStats.State.active(), Collections.singleton("z"), Collections.emptyMap())); @@ -531,8 +535,8 @@ public void scheduleToTerminate_remove() { FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", "", "", "", null, null, false, false, 0, 1, 1, 0, 1, true, - false, "-1", false, 0, 0, false, - 10, false); + false, "-1", false, 0, 0, + 10, false, noScaling); fleetCloud.setStats(new FleetStateStats("", 2, FleetStateStats.State.active(), new HashSet<>(Arrays.asList("z", "z1")), Collections.emptyMap())); @@ -557,8 +561,8 @@ public void scheduleToTerminate_upToZeroNodes() { FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", "", "", "", null, null, false, false, 0, 0, 1, 0, 1, true, - false, "-1", false, 0, 0, false, - 10, false); + false, "-1", false, 0, 0, + 10, false, noScaling); fleetCloud.setStats(new FleetStateStats("", 2, FleetStateStats.State.active(), new HashSet<>(Arrays.asList("z-1", "z-2")), Collections.emptyMap())); @@ -588,8 +592,8 @@ public void scheduleToTerminate_removeNoMoreMinIfCalledMultipleBeforeUpdate() { FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", "", "", "", null, null, false, false, 0, 1, 1, 0, 1, true, - false, "-1", false, 0, 0, false, - 10, false); + false, "-1", false, 0, 0, + 10, false, noScaling); fleetCloud.setStats(new FleetStateStats("", 3, FleetStateStats.State.active(), new HashSet<>(Arrays.asList("z1", "z2", "z3")), Collections.emptyMap())); @@ -622,7 +626,7 @@ public void update_shouldDoNothingIfNoTerminationOrProvisionAndFleetIsEmpty() { "", "fleetId", "", null, null, false, false, 0, 0, 1, 0, 1, true, false, "-1", false, 0, - 0, false, 10, false); + 0, 10, false, noScaling); // when FleetStateStats stats = fleetCloud.update(); @@ -646,7 +650,7 @@ public void update_shouldIncreaseTargetCapacityWhenProvisioned() { "", "fleetId", "", null, null, false, false, 0, 0, 10, 0, 1, true, false, "-1", false, 0, - 0, false, 10, false); + 0, 10, false, noScaling); fleetCloud.setStats(new FleetStateStats("", 0, FleetStateStats.State.active(), Collections.emptySet(), Collections.emptyMap())); @@ -674,7 +678,7 @@ public void update_shouldResetTerminateAndProvision() { "", "fleetId", "", null, null, false, false, 0, 0, 10, 0, 1, true, false, "-1", false, 0, - 0, false, 10, false); + 0, 10, false, noScaling); fleetCloud.setStats(currentState); @@ -703,7 +707,7 @@ public void update_shouldNotIncreaseMoreThenMax() { "", "fleetId", "", null, null, false, false, 0, 0, 10, 0, 1, true, false, "-1", false, 0, - 0, false, 10, false); + 0, 10, false, noScaling); fleetCloud.setStats(new FleetStateStats("", 5, FleetStateStats.State.active(), Collections.emptySet(), Collections.emptyMap())); @@ -734,7 +738,7 @@ public void update_shouldNotCountScheduledToTerminateWhenScaleUp() { "", "fleetId", "", null, null, false, false, 0, 0, 10, 0, 1, true, false, "-1", false, 0, - 0, false, 10, false); + 0, 10, false, noScaling); fleetCloud.setStats(new FleetStateStats("", 5, FleetStateStats.State.active(), Collections.emptySet(), Collections.emptyMap())); @@ -764,7 +768,7 @@ public void update_shouldDecreaseTargetCapacityAndTerminateInstancesIfScheduled( "", "fleetId", "", null, null, false, false, 0, 0, 10, 0, 1, true, false, "-1", false, 0, - 0, false, 10, false); + 0, 10, false, noScaling); fleetCloud.setStats(new FleetStateStats("", 4, FleetStateStats.State.active(), Collections.emptySet(), Collections.emptyMap())); @@ -805,7 +809,7 @@ public void update_shouldAddNodeIfAnyNewDescribed() throws IOException { "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, false, 0, 0, 1, 0, 1, false, false, "-1", false, - 0, 0, false, 10, false); + 0, 0, 10, false, noScaling); ArgumentCaptor nodeCaptor = ArgumentCaptor.forClass(Node.class); doNothing().when(jenkins).addNode(nodeCaptor.capture()); @@ -847,7 +851,7 @@ public void update_shouldTagNewNodesBeforeAdding() throws IOException { "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, false, 0, 0, 2, 0, 1, false, false, "-1", false, - 0, 0, false, 10, false); + 0, 0, 10, false, noScaling); ArgumentCaptor nodeCaptor = ArgumentCaptor.forClass(Node.class); doNothing().when(jenkins).addNode(nodeCaptor.capture()); @@ -883,7 +887,7 @@ public void update_shouldTagNewNodesBeforeAddingWithFleetName() throws IOExcepti "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, false, 0, 0, 1, 0, 1, false, false, "-1", false, - 0, 0, false, 10, false); + 0, 0, 10, false, noScaling); ArgumentCaptor nodeCaptor = ArgumentCaptor.forClass(Node.class); doNothing().when(jenkins).addNode(nodeCaptor.capture()); @@ -924,7 +928,7 @@ public void update_givenFailedTaggingShouldIgnoreExceptionAndAddNode() throws IO "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, false, 0, 0, 1, 0, 1, false, false, "-1", false, - 0, 0, false, 10, false); + 0, 0, 10, false, noScaling); ArgumentCaptor nodeCaptor = ArgumentCaptor.forClass(Node.class); doNothing().when(jenkins).addNode(nodeCaptor.capture()); @@ -962,7 +966,7 @@ public void update_shouldAddNodeIfAnyNewDescribed_restrictUsage() throws IOExcep "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, false, 0, 0, 1, 0, 1, false, true, "-1", false, - 0, 0, false, 10, false); + 0, 0, 10, false, noScaling); ArgumentCaptor nodeCaptor = ArgumentCaptor.forClass(Node.class); doNothing().when(jenkins).addNode(nodeCaptor.capture()); @@ -1007,7 +1011,7 @@ public void update_shouldAddNodeWithNumExecutors_whenWeightProvidedButNotEnabled "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, false, 0, 0, 1, 0, 1, false, true, "-1", false, - 0, 0, false, 10, false); + 0, 0, 10, false, noScaling); ArgumentCaptor nodeCaptor = ArgumentCaptor.forClass(Node.class); doNothing().when(jenkins).addNode(nodeCaptor.capture()); @@ -1040,7 +1044,7 @@ public void update_givenManuallyUpdatedFleetShouldCorrectLocalTargetCapacityToKe "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, false, 0, 0, 10, 0, 1, false, true, "-1", false, - 0, 0, false, 10, false); + 0, 0, 10, false, noScaling); fleetCloud.setStats(initState); doNothing().when(jenkins).addNode(any(Node.class)); @@ -1077,7 +1081,7 @@ public void update_shouldTrimPlannedNodesIfExceedTargetCapacity() throws IOExcep "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, false, 0, 0, 10, 0, 1, false, true, "-1", false, - 0, 0, false, 10, false); + 0, 0, 10, false, noScaling); fleetCloud.setStats(initState); doNothing().when(jenkins).addNode(any(Node.class)); @@ -1125,7 +1129,7 @@ public void update_shouldTrimPlannedNodesBasedOnUpdatedTargetCapacityIfProvision "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, false, 0, 0, 10, 0, 1, false, true, "-1", false, - 0, 0, false, 10, false); + 0, 0, 10, false, noScaling); fleetCloud.setStats(initState); doNothing().when(jenkins).addNode(any(Node.class)); @@ -1177,7 +1181,7 @@ public void update_shouldUpdateStateWithFleetTargetCapacityPlusToAdd() throws IO "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, false, 0, 0, 10, 0,1, false, true, "-1", false, - 0, 0, false, 10, false); + 0, 0, 10, false, noScaling); fleetCloud.setStats(initState); doNothing().when(jenkins).addNode(any(Node.class)); @@ -1223,7 +1227,7 @@ public void update_shouldUpdateStateWithFleetTargetCapacityMinusToTerminate() th "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, false, 0, 0, 10,0, 1, false, true, "-1", false, - 0, 0, false, 10, false); + 0, 0, 10, false, noScaling); fleetCloud.setStats(initState); doNothing().when(jenkins).addNode(any(Node.class)); @@ -1258,7 +1262,7 @@ public void update_shouldTerminateIdleOrNullInstancesOnly() { "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, false, 0, 0, 2, 0, 1, false, false, "-1", false, - 0, 0, false, 10, false); + 0, 0, 10, false, noScaling); when(jenkins.getComputer("i-1")).thenReturn(idleComputer); when(jenkins.getComputer("i-2")).thenReturn(busyComputer); @@ -1302,7 +1306,7 @@ public void update_shouldUpdateStateWithMinSpare() throws IOException { "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, false, 0, 0, 10, minSpareSize, 1, false, true, "-1", false, - 0, 0, false, 10, false); + 0, 0, 10, false, noScaling); fleetCloud.setStats(initState); doNothing().when(jenkins).addNode(any(Node.class)); @@ -1343,7 +1347,7 @@ public void update_shouldAddNodeWithScaledNumExecutors_whenWeightPresentAndEnabl "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, false, 0, 0, 1, 0, 1, false, true, "-1", false, - 0, 0, true, 10, false); + 0, 0, 10, false, weightedScaling); ArgumentCaptor nodeCaptor = ArgumentCaptor.forClass(Node.class); doNothing().when(jenkins).addNode(nodeCaptor.capture()); @@ -1385,7 +1389,7 @@ public void update_shouldAddNodeWithNumExecutors_whenWeightPresentAndEnabledButF "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, false, 0, 0, 1, 0, 1, false, true, "-1", false, - 0, 0, true, 10, false); + 0, 0, 10, false, weightedScaling); ArgumentCaptor nodeCaptor = ArgumentCaptor.forClass(Node.class); doNothing().when(jenkins).addNode(nodeCaptor.capture()); @@ -1427,7 +1431,7 @@ public void update_shouldAddNodeWithRoundToLowScaledNumExecutors_whenWeightPrese "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, false, 0, 0, 1, 0, 1, false, true, "-1", false, - 0, 0, true, 10, false); + 0, 0, 10, false, weightedScaling); ArgumentCaptor nodeCaptor = ArgumentCaptor.forClass(Node.class); doNothing().when(jenkins).addNode(nodeCaptor.capture()); @@ -1469,7 +1473,7 @@ public void update_shouldAddNodeWithRoundToLowScaledNumExecutors_whenWeightPrese "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, false, 0, 0, 1, 0, 1, false, true, "-1", false, - 0, 0, true, 10, false); + 0, 0, 10, false, weightedScaling); ArgumentCaptor nodeCaptor = ArgumentCaptor.forClass(FleetNode.class); doNothing().when(jenkins).addNode(nodeCaptor.capture()); @@ -1512,7 +1516,7 @@ public void update_shouldAddNodeWithScaledToOneNumExecutors_whenWeightPresentBut "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, false, 0, 0, 1, 0, 1, false, true, "-1", false, - 0, 0, true, 10, false); + 0, 0, 10, false, weightedScaling); // set init state so we can do provision fleetCloud.setStats(new FleetStateStats("", 0, FleetStateStats.State.active(), Collections.emptySet(), Collections.emptyMap())); @@ -1557,7 +1561,7 @@ public void update_givenFailedModifyShouldNotUpdateToAddToDelete() throws IOExce "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, false, 0, 0, 1, 0, 1, false, true, "-1", false, - 0, 0, true, 10, false); + 0, 0, 10, false, weightedScaling); ArgumentCaptor nodeCaptor = ArgumentCaptor.forClass(Node.class); doNothing().when(jenkins).addNode(nodeCaptor.capture()); @@ -1605,7 +1609,7 @@ public void update_givenFleetInModifyingShouldNotDoAnyUpdates() throws IOExcepti "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, false, 0, 0, 1, 0, 1, false, true, "-1", false, - 0, 0, true, 10, false); + 0, 0, 10, false, weightedScaling); fleetCloud.setStats(initialState); doNothing().when(jenkins).addNode(any(Node.class)); @@ -1634,8 +1638,8 @@ public void update_scheduledFuturesExecutesAfterTimeout() throws IOException, In FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", "", "", "", null, null, false, false, 0, 0, 10, 0,1, true, - false, "-1", false, timeout, 0, false, - 1, false); + false, "-1", false, timeout, 0, + 1, false, noScaling); fleetCloud.setStats(new FleetStateStats("", 5, FleetStateStats.State.active(), Collections.emptySet(), Collections.emptyMap())); @@ -1665,8 +1669,8 @@ public void update_scheduledFuturesIsCancelledAfterUpdate() throws IOException, FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", "", "", "", null, null, false, false, 0, 0, 10, 0,1, true, - false, "-1", false, timeout, 0, false, - 10, false); + false, "-1", false, timeout, 0, + 10, false, noScaling); fleetCloud.setStats(new FleetStateStats("", 5, FleetStateStats.State.active(), Collections.emptySet(), Collections.emptyMap())); @@ -1693,7 +1697,7 @@ public void update_shouldScaleUpToMinSize() { "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, false, 0, 1, 1, 0,1, false, true, "-1", false, - 0, 0, true, 10, false); + 0, 0, 10, false, weightedScaling); // when fleetCloud.update(); @@ -1702,6 +1706,370 @@ public void update_shouldScaleUpToMinSize() { Assert.assertEquals(fleetCloud.getStats().getNumDesired(), 1); } + @Test + public void update_whenScalingByNodeHardwareWithLessVCPUs_shouldScaleExecutorsByVCPUs() throws IOException { + when(amazonEC2.describeInstanceTypes(any(DescribeInstanceTypesRequest.class))) + .thenReturn(new DescribeInstanceTypesResult() + .withInstanceTypes( + new InstanceTypeInfo() + .withMemoryInfo( + new MemoryInfo() + .withSizeInMiB((long)4*MiB_TO_GiB_MULTIPLIER)) + .withVCpuInfo( + new VCpuInfo() + .withDefaultVCpus(2)))); + + when(ec2Api.connect(any(String.class), any(String.class), anyString())).thenReturn(amazonEC2); + + final Instance instance = new Instance() + .withPublicIpAddress("p-ip") + .withInstanceId("i-0") + .withState(new InstanceState() + .withName(InstanceStateName.Running)); + + final HashMap instanceIdMap = new HashMap<>(); + instanceIdMap.put("i-0", instance); + + when(ec2Api.describeInstances(any(AmazonEC2.class), any(Set.class))).thenReturn( + instanceIdMap); + + PowerMockito.when(fleet.getState(anyString(), anyString(), anyString(), anyString())) + .thenReturn(new FleetStateStats("fleetId", 0, FleetStateStats.State.active(), + Collections.singleton("i-0"), Collections.emptyMap())); + + mockNodeCreatingPart(); + + FleetCloud.NodeHardwareScaler nodeHardwareScaler = new FleetCloud.NodeHardwareScaler(1, 1); + + FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", + "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, + false, 0, 0, 10, 0, 1, true, + false, "-1", false, 0, + 0, 10, false, nodeHardwareScaler); + + ArgumentCaptor nodeCaptor = ArgumentCaptor.forClass(Node.class); + doNothing().when(jenkins).addNode(nodeCaptor.capture()); + + // when + fleetCloud.update(); + + // then + Node actualFleetNode = nodeCaptor.getValue(); + assertEquals(2, actualFleetNode.getNumExecutors()); + } + + @Test + public void update_whenScalingByNodeHardwareWithLessMemory_shouldScaleExecutorsByMemory() throws IOException { + when(amazonEC2.describeInstanceTypes(any(DescribeInstanceTypesRequest.class))) + .thenReturn(new DescribeInstanceTypesResult() + .withInstanceTypes( + new InstanceTypeInfo() + .withMemoryInfo( + new MemoryInfo() + .withSizeInMiB((long)6*MiB_TO_GiB_MULTIPLIER)) + .withVCpuInfo( + new VCpuInfo() + .withDefaultVCpus(8)))); + + when(ec2Api.connect(any(String.class), any(String.class), anyString())).thenReturn(amazonEC2); + + final Instance instance = new Instance() + .withPublicIpAddress("p-ip") + .withInstanceId("i-0") + .withState(new InstanceState() + .withName(InstanceStateName.Running)); + + final HashMap instanceIdMap = new HashMap<>(); + instanceIdMap.put("i-0", instance); + + when(ec2Api.describeInstances(any(AmazonEC2.class), any(Set.class))).thenReturn( + instanceIdMap); + + PowerMockito.when(fleet.getState(anyString(), anyString(), anyString(), anyString())) + .thenReturn(new FleetStateStats("fleetId", 0, FleetStateStats.State.active(), + Collections.singleton("i-0"), Collections.emptyMap())); + + mockNodeCreatingPart(); + + FleetCloud.NodeHardwareScaler nodeHardwareScaler = new FleetCloud.NodeHardwareScaler(2, 2); + + FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", + "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, + false, 0, 0, 10, 0, 1, true, + false, "-1", false, 0, + 0, 10, false, nodeHardwareScaler); + + ArgumentCaptor nodeCaptor = ArgumentCaptor.forClass(Node.class); + doNothing().when(jenkins).addNode(nodeCaptor.capture()); + + // when + fleetCloud.update(); + + // then + Node actualFleetNode = nodeCaptor.getValue(); + assertEquals(3, actualFleetNode.getNumExecutors()); + } + + @Test + public void update_whenScalingByNodeHardwareWithNoVCPUs_shouldScaleExecutorsByMemory() throws IOException { + when(amazonEC2.describeInstanceTypes(any(DescribeInstanceTypesRequest.class))) + .thenReturn(new DescribeInstanceTypesResult() + .withInstanceTypes( + new InstanceTypeInfo() + .withMemoryInfo( + new MemoryInfo() + .withSizeInMiB((long)4*MiB_TO_GiB_MULTIPLIER)) + .withVCpuInfo( + new VCpuInfo() + .withDefaultVCpus(2)))); + + when(ec2Api.connect(any(String.class), any(String.class), anyString())).thenReturn(amazonEC2); + + final Instance instance = new Instance() + .withPublicIpAddress("p-ip") + .withInstanceId("i-0") + .withState(new InstanceState() + .withName(InstanceStateName.Running)); + + final HashMap instanceIdMap = new HashMap<>(); + instanceIdMap.put("i-0", instance); + + when(ec2Api.describeInstances(any(AmazonEC2.class), any(Set.class))).thenReturn( + instanceIdMap); + + PowerMockito.when(fleet.getState(anyString(), anyString(), anyString(), anyString())) + .thenReturn(new FleetStateStats("fleetId", 0, FleetStateStats.State.active(), + Collections.singleton("i-0"), Collections.emptyMap())); + + mockNodeCreatingPart(); + + FleetCloud.NodeHardwareScaler nodeHardwareScaler = new FleetCloud.NodeHardwareScaler(0, 1); + + FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", + "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, + false, 0, 0, 10, 0, 1, true, + false, "-1", false, 0, + 0, 10, false, nodeHardwareScaler); + + ArgumentCaptor nodeCaptor = ArgumentCaptor.forClass(Node.class); + doNothing().when(jenkins).addNode(nodeCaptor.capture()); + + // when + fleetCloud.update(); + + // then + Node actualFleetNode = nodeCaptor.getValue(); + assertEquals(4, actualFleetNode.getNumExecutors()); + } + + @Test + public void update_whenScalingByNodeHardwareWithNoMemory_shouldScaleExecutorsByVCPUs() throws IOException { + when(amazonEC2.describeInstanceTypes(any(DescribeInstanceTypesRequest.class))) + .thenReturn(new DescribeInstanceTypesResult() + .withInstanceTypes( + new InstanceTypeInfo() + .withMemoryInfo( + new MemoryInfo() + .withSizeInMiB((long)3*MiB_TO_GiB_MULTIPLIER)) + .withVCpuInfo( + new VCpuInfo() + .withDefaultVCpus(8)))); + + when(ec2Api.connect(any(String.class), any(String.class), anyString())).thenReturn(amazonEC2); + + final Instance instance = new Instance() + .withPublicIpAddress("p-ip") + .withInstanceId("i-0") + .withState(new InstanceState() + .withName(InstanceStateName.Running)); + + final HashMap instanceIdMap = new HashMap<>(); + instanceIdMap.put("i-0", instance); + + when(ec2Api.describeInstances(any(AmazonEC2.class), any(Set.class))).thenReturn( + instanceIdMap); + + PowerMockito.when(fleet.getState(anyString(), anyString(), anyString(), anyString())) + .thenReturn(new FleetStateStats("fleetId", 0, FleetStateStats.State.active(), + Collections.singleton("i-0"), Collections.emptyMap())); + + mockNodeCreatingPart(); + + FleetCloud.NodeHardwareScaler nodeHardwareScaler = new FleetCloud.NodeHardwareScaler(2, 0); + + FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", + "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, + false, 0, 0, 10, 0, 1, true, + false, "-1", false, 0, + 0, 10, false, nodeHardwareScaler); + + ArgumentCaptor nodeCaptor = ArgumentCaptor.forClass(Node.class); + doNothing().when(jenkins).addNode(nodeCaptor.capture()); + + // when + fleetCloud.update(); + + // then + Node actualFleetNode = nodeCaptor.getValue(); + assertEquals(4, actualFleetNode.getNumExecutors()); + } + + @Test + public void update_whenScalingByNodeHardwareByMemoryWithLowMemory_shouldSetOneExecutor() throws IOException { + when(amazonEC2.describeInstanceTypes(any(DescribeInstanceTypesRequest.class))) + .thenReturn(new DescribeInstanceTypesResult() + .withInstanceTypes( + new InstanceTypeInfo() + .withMemoryInfo( + new MemoryInfo() + .withSizeInMiB((long)2*MiB_TO_GiB_MULTIPLIER)) + .withVCpuInfo( + new VCpuInfo() + .withDefaultVCpus(2)))); + + when(ec2Api.connect(any(String.class), any(String.class), anyString())).thenReturn(amazonEC2); + + final Instance instance = new Instance() + .withPublicIpAddress("p-ip") + .withInstanceId("i-0") + .withState(new InstanceState() + .withName(InstanceStateName.Running)); + + final HashMap instanceIdMap = new HashMap<>(); + instanceIdMap.put("i-0", instance); + + when(ec2Api.describeInstances(any(AmazonEC2.class), any(Set.class))).thenReturn( + instanceIdMap); + + PowerMockito.when(fleet.getState(anyString(), anyString(), anyString(), anyString())) + .thenReturn(new FleetStateStats("fleetId", 0, FleetStateStats.State.active(), + Collections.singleton("i-0"), Collections.emptyMap())); + + mockNodeCreatingPart(); + + FleetCloud.NodeHardwareScaler nodeHardwareScaler = new FleetCloud.NodeHardwareScaler(0, 4); + + FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", + "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, + false, 0, 0, 10, 0, 1, true, + false, "-1", false, 0, + 0, 10, false, nodeHardwareScaler); + + ArgumentCaptor nodeCaptor = ArgumentCaptor.forClass(Node.class); + doNothing().when(jenkins).addNode(nodeCaptor.capture()); + + // when + fleetCloud.update(); + + // then + Node actualFleetNode = nodeCaptor.getValue(); + assertEquals(1, actualFleetNode.getNumExecutors()); + } + + @Test + public void update_whenScalingByNodeHardwareByVCPUsWithLowVCPUCount_shouldSetOneExecutor() throws IOException { + when(amazonEC2.describeInstanceTypes(any(DescribeInstanceTypesRequest.class))) + .thenReturn(new DescribeInstanceTypesResult() + .withInstanceTypes( + new InstanceTypeInfo() + .withMemoryInfo( + new MemoryInfo() + .withSizeInMiB((long)4*MiB_TO_GiB_MULTIPLIER)) + .withVCpuInfo( + new VCpuInfo() + .withDefaultVCpus(2)))); + + when(ec2Api.connect(any(String.class), any(String.class), anyString())).thenReturn(amazonEC2); + + final Instance instance = new Instance() + .withPublicIpAddress("p-ip") + .withInstanceId("i-0") + .withState(new InstanceState() + .withName(InstanceStateName.Running)); + + final HashMap instanceIdMap = new HashMap<>(); + instanceIdMap.put("i-0", instance); + + when(ec2Api.describeInstances(any(AmazonEC2.class), any(Set.class))).thenReturn( + instanceIdMap); + + PowerMockito.when(fleet.getState(anyString(), anyString(), anyString(), anyString())) + .thenReturn(new FleetStateStats("fleetId", 0, FleetStateStats.State.active(), + Collections.singleton("i-0"), Collections.emptyMap())); + + mockNodeCreatingPart(); + + FleetCloud.NodeHardwareScaler nodeHardwareScaler = new FleetCloud.NodeHardwareScaler(5, 0); + + FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", + "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, + false, 0, 0, 10, 0, 1, true, + false, "-1", false, 0, + 0, 10, false, nodeHardwareScaler); + + ArgumentCaptor nodeCaptor = ArgumentCaptor.forClass(Node.class); + doNothing().when(jenkins).addNode(nodeCaptor.capture()); + + // when + fleetCloud.update(); + + // then + Node actualFleetNode = nodeCaptor.getValue(); + assertEquals(1, actualFleetNode.getNumExecutors()); + } + + @Test + public void update_whenScalingByNodeHardwareWithNoVCPUsAndNoMemory_shouldSetExecutorsToNumExecutors() throws IOException { + when(amazonEC2.describeInstanceTypes(any(DescribeInstanceTypesRequest.class))) + .thenReturn(new DescribeInstanceTypesResult() + .withInstanceTypes( + new InstanceTypeInfo() + .withMemoryInfo( + new MemoryInfo() + .withSizeInMiB((long)4*MiB_TO_GiB_MULTIPLIER)) + .withVCpuInfo( + new VCpuInfo() + .withDefaultVCpus(2)))); + + when(ec2Api.connect(any(String.class), any(String.class), anyString())).thenReturn(amazonEC2); + + final Instance instance = new Instance() + .withPublicIpAddress("p-ip") + .withInstanceId("i-0") + .withState(new InstanceState() + .withName(InstanceStateName.Running)); + + final HashMap instanceIdMap = new HashMap<>(); + instanceIdMap.put("i-0", instance); + + when(ec2Api.describeInstances(any(AmazonEC2.class), any(Set.class))).thenReturn( + instanceIdMap); + + PowerMockito.when(fleet.getState(anyString(), anyString(), anyString(), anyString())) + .thenReturn(new FleetStateStats("fleetId", 0, FleetStateStats.State.active(), + Collections.singleton("i-0"), Collections.emptyMap())); + + mockNodeCreatingPart(); + + FleetCloud.NodeHardwareScaler nodeHardwareScaler = new FleetCloud.NodeHardwareScaler(0, 0); + + FleetCloud fleetCloud = new FleetCloud("TestCloud", "credId", null, "region", + "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, + false, 0, 0, 10, 0, 3, true, + false, "-1", false, 0, + 0, 10, false, nodeHardwareScaler); + + ArgumentCaptor nodeCaptor = ArgumentCaptor.forClass(Node.class); + doNothing().when(jenkins).addNode(nodeCaptor.capture()); + + // when + fleetCloud.update(); + + // then + Node actualFleetNode = nodeCaptor.getValue(); + assertEquals(fleetCloud.getNumExecutors(), actualFleetNode.getNumExecutors()); + } + @Test public void removeScheduledFutures_success() { // given @@ -1709,7 +2077,7 @@ public void removeScheduledFutures_success() { "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, false, 0, 0, 1, 0, 1, false, true, "-1", false, - 0, 0, true, 10, false); + 0, 0, 10, false, weightedScaling); ArrayList> scheduledFutures = new ArrayList<>(); scheduledFutures.add(mock(ScheduledFuture.class)); @@ -1730,7 +2098,7 @@ public void removeScheduledFutures_scheduledFutureIsEmpty() { "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, false, 0, 0, 1, 0, 1, false, true, "-1", false, - 0, 0, true, 10, false); + 0, 0, 10, false, weightedScaling); ArrayList> scheduledFutures = new ArrayList<>(); fleetCloud.setPlannedNodeScheduledFutures(scheduledFutures); @@ -1749,7 +2117,7 @@ public void removeScheduledFutures_numToRemoveIsZero() { "", "fleetId", "", null, PowerMockito.mock(ComputerConnector.class), false, false, 0, 0, 1, 0, 1, false, true, "-1", false, - 0, 0, true, 10, false); + 0, 0, 10, false, weightedScaling); ArrayList> scheduledFutures = new ArrayList<>(); scheduledFutures.add(mock(ScheduledFuture.class)); @@ -1954,140 +2322,140 @@ public void descriptorImpl_doCheckFleet_nonDefault() { @Test public void getDisplayName_returnDisplayName() { - FleetCloud ec2FleetCloud = new FleetCloud( + FleetCloud fleetCloud = new FleetCloud( "CloudName", null, null, null, null, null, null, null, null, false, false, null, 0, 1, 0, 1, true, false, "-1", false - , 0, 0, false, - 10, false); - assertEquals(ec2FleetCloud.getDisplayName(), "CloudName"); + , 0, 0, + 10, false, noScaling); + assertEquals(fleetCloud.getDisplayName(), "CloudName"); } @Test public void getAwsCredentialsId_returnNull_whenNoCredentialsIdOrAwsCredentialsId() { - FleetCloud ec2FleetCloud = new FleetCloud( + FleetCloud fleetCloud = new FleetCloud( "TestCloud", null, null, null, null, null, null, null, null, false, false, null, 0, 1, 0, 1, true, false, "-1", false, - 0, 0, false, - 10, false); - Assert.assertNull(ec2FleetCloud.getAwsCredentialsId()); + 0, 0, + 10, false, noScaling); + Assert.assertNull(fleetCloud.getAwsCredentialsId()); } @Test public void getAwsCredentialsId_returnValue_whenCredentialsIdPresent() { - FleetCloud ec2FleetCloud = new FleetCloud( + FleetCloud fleetCloud = new FleetCloud( "TestCloud", null, "Opa", null, null, null, null, null, null, false, false, null, 0, 1, 0, 1, true, false, "-1", false - , 0, 0, false, - 10, false); - assertEquals("Opa", ec2FleetCloud.getAwsCredentialsId()); + , 0, 0, + 10, false, noScaling); + assertEquals("Opa", fleetCloud.getAwsCredentialsId()); } @Test public void getAwsCredentialsId_returnValue_whenAwsCredentialsIdPresent() { - FleetCloud ec2FleetCloud = new FleetCloud( + FleetCloud fleetCloud = new FleetCloud( "TestCloud", "Opa", null, null, null, null, null, null, null, false, false, null, 0, 1, 0, 1, true, false, "-1", false - , 0, 0, false, - 10, false); - assertEquals("Opa", ec2FleetCloud.getAwsCredentialsId()); + , 0, 0, + 10, false, noScaling); + assertEquals("Opa", fleetCloud.getAwsCredentialsId()); } @Test public void getAwsCredentialsId_returnAwsCredentialsId_whenAwsCredentialsIdAndCredentialsIdPresent() { - FleetCloud ec2FleetCloud = new FleetCloud( + FleetCloud fleetCloud = new FleetCloud( "TestCloud", "A", "B", null, null, null, null, null, null, false, false, null, 0, 1, 0, 1, true, false, "-1", false - , 0, 0, false, - 10, false); - assertEquals("A", ec2FleetCloud.getAwsCredentialsId()); + , 0, 0, + 10, false, noScaling); + assertEquals("A", fleetCloud.getAwsCredentialsId()); } // todo create test cases update failed to modify fleet @Test public void getCloudStatusInterval_returnCloudStatusInterval() { - FleetCloud ec2FleetCloud = new FleetCloud( + FleetCloud fleetCloud = new FleetCloud( "CloudName", null, null, null, null, null, null, null, null, false, false, null, 0, 1, 0, 1, true, false, "-1", false - , 0, 0, false, - 45, false); - assertEquals(45, ec2FleetCloud.getCloudStatusIntervalSec()); + , 0, 0, + 45, false, noScaling); + assertEquals(45, fleetCloud.getCloudStatusIntervalSec()); } @Test public void create_numExecutorsLessThenOneShouldUpgradedToOne() { - FleetCloud ec2FleetCloud = new FleetCloud( + FleetCloud fleetCloud = new FleetCloud( "CloudName", null, null, null, null, null, null, null, null, false, false, null, 0, 1, 0, 0, true, false, "-1", false - , 0, 0, false, - 45, false); - assertEquals(1, ec2FleetCloud.getNumExecutors()); + , 0, 0, + 45, false, noScaling); + assertEquals(1, fleetCloud.getNumExecutors()); } @Test public void hasUnlimitedUsesForNodes_shouldReturnTrueWhenUnlimited() { final int maxTotalUses = -1; - FleetCloud ec2FleetCloud = new FleetCloud( + FleetCloud fleetCloud = new FleetCloud( "CloudName", null, null, null, null, null, null, null, null, false, false, null, 0, 1, 0, 0, true, false, String.valueOf(maxTotalUses), false - , 0, 0, false, - 45, false); - assertTrue(ec2FleetCloud.hasUnlimitedUsesForNodes()); + , 0, 0, + 45, false, noScaling); + assertTrue(fleetCloud.hasUnlimitedUsesForNodes()); } @Test public void hasUnlimitedUsesForNodes_shouldReturnDefaultTrueForNull() { final String maxTotalUses = null; - FleetCloud ec2FleetCloud = new FleetCloud( + FleetCloud fleetCloud = new FleetCloud( "CloudName", null, null, null, null, null, null, null, null, false, false, null, 0, 1, 0, 0, true, false, maxTotalUses, false - , 0, 0, false, - 45, false); - assertTrue(ec2FleetCloud.hasUnlimitedUsesForNodes()); + , 0, 0, + 45, false, noScaling); + assertTrue(fleetCloud.hasUnlimitedUsesForNodes()); } @Test public void hasUnlimitedUsesForNodes_shouldReturnDefaultTrueForEmptyString() { final String maxTotalUses = ""; - FleetCloud ec2FleetCloud = new FleetCloud( + FleetCloud fleetCloud = new FleetCloud( "CloudName", null, null, null, null, null, null, null, null, false, false, null, 0, 1, 0, 0, true, false, maxTotalUses, false - , 0, 0, false, - 45, false); - assertTrue(ec2FleetCloud.hasUnlimitedUsesForNodes()); + , 0, 0, + 45, false, noScaling); + assertTrue(fleetCloud.hasUnlimitedUsesForNodes()); } @Test public void hasUnlimitedUsesForNodes_shouldReturnFalseWhenLimited() { final int maxTotalUses = 5; - FleetCloud ec2FleetCloud = new FleetCloud( + FleetCloud fleetCloud = new FleetCloud( "CloudName", null, null, null, null, null, null, null, null, false, false, null, 0, 1, 0, 0, true, false, String.valueOf(maxTotalUses), false - , 0, 0, false, - 45, false); - assertFalse(ec2FleetCloud.hasUnlimitedUsesForNodes()); + , 0, 0, + 45, false, noScaling); + assertFalse(fleetCloud.hasUnlimitedUsesForNodes()); } private void mockNodeCreatingPart() { diff --git a/src/test/java/com/amazon/jenkins/ec2fleet/FleetCloudWithHistory.java b/src/test/java/com/amazon/jenkins/ec2fleet/FleetCloudWithHistory.java index 3a9814a8..81f4400e 100644 --- a/src/test/java/com/amazon/jenkins/ec2fleet/FleetCloudWithHistory.java +++ b/src/test/java/com/amazon/jenkins/ec2fleet/FleetCloudWithHistory.java @@ -17,12 +17,12 @@ public FleetCloudWithHistory( String endpoint, String fleet, String labelString, String fsRoot, ComputerConnector computerConnector, boolean privateIpUsed, boolean alwaysReconnect, Integer idleMinutes, Integer minSize, Integer maxSize, Integer minSpareSize, Integer numExecutors, boolean addNodeOnlyIfRunning, boolean restrictUsage, boolean disableTaskResubmit, - Integer initOnlineTimeoutSec, Integer initOnlineCheckIntervalSec, boolean scaleExecutorsByWeight, - Integer cloudStatusIntervalSec, boolean immediatelyProvision) { + Integer initOnlineTimeoutSec, Integer initOnlineCheckIntervalSec, Integer cloudStatusIntervalSec, + boolean immediatelyProvision, ExecutorScaler executorScaler) { super(name, awsCredentialsId, credentialsId, region, endpoint, fleet, labelString, fsRoot, computerConnector, privateIpUsed, alwaysReconnect, idleMinutes, minSize, maxSize, minSpareSize, numExecutors, addNodeOnlyIfRunning, restrictUsage, "-1", disableTaskResubmit, initOnlineTimeoutSec, - initOnlineCheckIntervalSec, scaleExecutorsByWeight, cloudStatusIntervalSec, immediatelyProvision); + initOnlineCheckIntervalSec, cloudStatusIntervalSec, immediatelyProvision, executorScaler); } @Override diff --git a/src/test/java/com/amazon/jenkins/ec2fleet/FleetCloudWithMeter.java b/src/test/java/com/amazon/jenkins/ec2fleet/FleetCloudWithMeter.java index 184d90c6..a730ea26 100644 --- a/src/test/java/com/amazon/jenkins/ec2fleet/FleetCloudWithMeter.java +++ b/src/test/java/com/amazon/jenkins/ec2fleet/FleetCloudWithMeter.java @@ -18,12 +18,12 @@ public FleetCloudWithMeter( String endpoint, String fleet, String labelString, String fsRoot, ComputerConnector computerConnector, boolean privateIpUsed, boolean alwaysReconnect, Integer idleMinutes, Integer minSize, Integer maxSize, Integer minSpareSize, Integer numExecutors, boolean addNodeOnlyIfRunning, boolean restrictUsage, boolean disableTaskResubmit, - Integer initOnlineTimeoutSec, Integer initOnlineCheckIntervalSec, boolean scaleExecutorsByWeight, - Integer cloudStatusIntervalSec, boolean immediatelyProvision) { + Integer initOnlineTimeoutSec, Integer initOnlineCheckIntervalSec, Integer cloudStatusIntervalSec, + boolean immediatelyProvision, ExecutorScaler executorScaler) { super(name, awsCredentialsId, credentialsId, region, endpoint, fleet, labelString, fsRoot, computerConnector, privateIpUsed, alwaysReconnect, idleMinutes, minSize, maxSize, minSpareSize, numExecutors, addNodeOnlyIfRunning, restrictUsage, "-1", disableTaskResubmit, initOnlineTimeoutSec, initOnlineCheckIntervalSec, - scaleExecutorsByWeight, cloudStatusIntervalSec, immediatelyProvision); + cloudStatusIntervalSec, immediatelyProvision, executorScaler); } @Override diff --git a/src/test/java/com/amazon/jenkins/ec2fleet/NoDelayProvisionStrategyPerformanceTest.java b/src/test/java/com/amazon/jenkins/ec2fleet/NoDelayProvisionStrategyPerformanceTest.java index 92286db6..350f0876 100644 --- a/src/test/java/com/amazon/jenkins/ec2fleet/NoDelayProvisionStrategyPerformanceTest.java +++ b/src/test/java/com/amazon/jenkins/ec2fleet/NoDelayProvisionStrategyPerformanceTest.java @@ -25,6 +25,7 @@ */ @Ignore public class NoDelayProvisionStrategyPerformanceTest extends IntegrationTest { + private final FleetCloud.ExecutorScaler noScaling = new FleetCloud.NoScaler(); @BeforeClass public static void beforeClass() { @@ -55,8 +56,8 @@ private void test(final boolean noDelay) throws IOException, InterruptedExceptio final FleetCloudWithHistory cloud = new FleetCloudWithHistory(null, "credId", null, "region", null, "fId", label, null, computerConnector, false, false, 1, 0, maxWorkers, 0, 1, true, false, - false, 0, 0, false, - 15, noDelay); + false, 0, 0, + 15, noDelay, noScaling); j.jenkins.clouds.add(cloud); System.out.println("waiting cloud start"); diff --git a/src/test/java/com/amazon/jenkins/ec2fleet/ProvisionIntegrationTest.java b/src/test/java/com/amazon/jenkins/ec2fleet/ProvisionIntegrationTest.java index 46904e16..9abd9606 100644 --- a/src/test/java/com/amazon/jenkins/ec2fleet/ProvisionIntegrationTest.java +++ b/src/test/java/com/amazon/jenkins/ec2fleet/ProvisionIntegrationTest.java @@ -49,6 +49,8 @@ public class ProvisionIntegrationTest extends IntegrationTest { + private final FleetCloud.ExecutorScaler noScaling = new FleetCloud.NoScaler(); + @BeforeClass public static void beforeClass() { System.setProperty("jenkins.test.timeout", "720"); @@ -68,8 +70,8 @@ public void dont_provide_any_planned_if_empty_and_reached_max_capacity() throws FleetCloud cloud = new FleetCloud("TestCloud", "credId", null, "region", null, "fId", "momo", null, computerConnector, false, false, 0, 0, 0, 0, 1, true, false, - "-1", false, 0, 0, false, - 2, false); + "-1", false, 0, 0, + 2, false, noScaling); j.jenkins.clouds.add(cloud); final EC2Api ec2Api = spy(EC2Api.class); @@ -102,8 +104,8 @@ public void should_add_planned_if_capacity_required_but_not_described_yet() thro FleetCloud cloud = new FleetCloud("TestCloud", "credId", null, "region", null, "fId", "momo", null, computerConnector, false, false, 0, 0, 10, 0, 1, true, false, - "-1", false, 0, 0, false, - 2, false); + "-1", false, 0, 0, + 2, false, noScaling); j.jenkins.clouds.add(cloud); List rs = enqueTask(1); @@ -137,8 +139,8 @@ public void should_keep_planned_node_until_node_will_not_be_online_so_jenkins_wi FleetCloud cloud = spy(new FleetCloud("TestCloud", "credId", null, "region", null, "fId", "momo", null, computerConnector, false, false, 0, 0, 10, 0, 1, true, false, - "-1", false, 300, 15, false, - 2, false)); + "-1", false, 300, 15, + 2, false, noScaling)); j.jenkins.clouds.add(cloud); @@ -171,8 +173,8 @@ public void should_not_keep_planned_node_if_configured_so_jenkins_will_overprovi final FleetCloud cloud = spy(new FleetCloud("TestCloud", "credId", null, "region", null, "fId", "momo", null, computerConnector, false, false, 0, 0, 10, 0, 1, true, false, - "-1", false, 0, 0, false, - 10, false)); + "-1", false, 0, 0, + 10, false, noScaling)); j.jenkins.clouds.add(cloud); mockFleetApiToSpotFleet(InstanceStateName.Running); @@ -199,8 +201,8 @@ public void should_not_allow_jenkins_to_provision_if_address_not_available() thr FleetCloud cloud = spy(new FleetCloud("TestCloud", "credId", null, "region", null, "fId", "momo", null, computerConnector, false, false, 0, 0, 10, 0, 1, true, false, - "-1", false, 0, 0, false, - 10, false)); + "-1", false, 0, 0, + 10, false, noScaling)); cloud.setStats(new FleetStateStats("", 0, FleetStateStats.State.active(), Collections.emptySet(), Collections.emptyMap())); @@ -260,8 +262,8 @@ public void should_not_convert_planned_to_node_if_state_is_not_running_and_check FleetCloud cloud = new FleetCloud("TestCloud", "credId", null, "region", null, "fId", "momo", null, computerConnector, false, false, 0, 0, 10, 0, 1, true, false, - "-1", false, 0, 0, false, - 2, false); + "-1", false, 0, 0, + 2, false, noScaling); j.jenkins.clouds.add(cloud); mockFleetApiToSpotFleet(InstanceStateName.Pending); @@ -297,8 +299,8 @@ public void should_successfully_create_nodes() throws Exception { FleetCloud cloud = new FleetCloud("TestCloud", "credId", null, "region", null, "fId", "momo", null, computerConnector, false, false, 0, 0, 2, 0, 1, true, false, - "-1", false, 0, 0, false, - 2, false); + "-1", false, 0, 0, + 2, false, noScaling); j.jenkins.clouds.add(cloud); mockFleetApiToSpotFleet(InstanceStateName.Running); @@ -328,8 +330,8 @@ public void should_continue_update_after_termination() throws IOException { final FleetCloud cloud = new FleetCloud("TestCloud", "credId", null, "region", null, "fId", "momo", null, computerConnector, false, false, 1, 0, 5, 0, 1, true, false, - "-1", false, 0, 0, false, - 10, false); + "-1", false, 0, 0, + 10, false, noScaling); j.jenkins.clouds.add(cloud); waitFirstStats(cloud); diff --git a/src/test/java/com/amazon/jenkins/ec2fleet/ProvisionPerformanceTest.java b/src/test/java/com/amazon/jenkins/ec2fleet/ProvisionPerformanceTest.java index 9b1539d2..fb22746c 100644 --- a/src/test/java/com/amazon/jenkins/ec2fleet/ProvisionPerformanceTest.java +++ b/src/test/java/com/amazon/jenkins/ec2fleet/ProvisionPerformanceTest.java @@ -20,6 +20,8 @@ @Ignore public class ProvisionPerformanceTest extends IntegrationTest { + private final FleetCloud.ExecutorScaler noScaling = new FleetCloud.NoScaler(); + @BeforeClass public static void beforeClass() { System.setProperty("jenkins.test.timeout", "720"); @@ -42,8 +44,7 @@ private void test(int workers, int maxTasks) throws IOException, InterruptedExce final FleetCloudWithMeter cloud = new FleetCloudWithMeter(null, "credId", null, "region", null, "fId", "momo", null, computerConnector, false, false, 1, 0, workers, 0, 1, true, false, - false, 0, 0, false, - 2, false); + false, 0, 0, 2, false, noScaling); j.jenkins.clouds.add(cloud); // updated plugin requires some init time to get first update diff --git a/src/test/java/com/amazon/jenkins/ec2fleet/RealTest.java b/src/test/java/com/amazon/jenkins/ec2fleet/RealTest.java index 292854a0..3c4b38b2 100644 --- a/src/test/java/com/amazon/jenkins/ec2fleet/RealTest.java +++ b/src/test/java/com/amazon/jenkins/ec2fleet/RealTest.java @@ -71,12 +71,14 @@ public static void beforeClass() { private List credentialLines; private String privateKeyName; private AWSCredentialsProvider awsCredentialsProvider; + private FleetCloud.ExecutorScaler noScaling; @Before public void before() throws IOException { credentialLines = FileUtils.readLines(new File("credentials.txt")); privateKeyName = getPrivateKeyName(credentialLines); awsCredentialsProvider = getAwsCredentialsProvider(credentialLines); + noScaling = new FleetCloud.NoScaler(); } @Ignore("for manual run as you need to provide real AWS credentials") @@ -134,8 +136,8 @@ public void run() { autoScalingGroupName, "momo", null, computerConnector, false, false, 1, 0, 5, 0, 1, true, false, - "-1", false, 180, 15, false, - 10, true); + "-1", false, 180, 15, + 10, true, noScaling); j.jenkins.clouds.add(cloud); final List tasks = enqueTask(2); @@ -186,8 +188,8 @@ public void givenEc2SpotFleet_shouldScaleUpExecuteTaskAndScaleDown() throws Exce requestSpotFleetResult.getSpotFleetRequestId(), "momo", null, computerConnector, false, false, 1, 0, 5, 0, 1, true, false, - "-1", false, 180, 15, false, - 10, true); + "-1", false, 180, 15, + 10, true, noScaling); j.jenkins.clouds.add(cloud); final List tasks = enqueTask(2); diff --git a/src/test/java/com/amazon/jenkins/ec2fleet/UiIntegrationTest.java b/src/test/java/com/amazon/jenkins/ec2fleet/UiIntegrationTest.java index caa04283..f5756267 100644 --- a/src/test/java/com/amazon/jenkins/ec2fleet/UiIntegrationTest.java +++ b/src/test/java/com/amazon/jenkins/ec2fleet/UiIntegrationTest.java @@ -54,6 +54,8 @@ public class UiIntegrationTest { @Rule public JenkinsRule j = new JenkinsRule(); + private final FleetCloud.ExecutorScaler noScaling = new FleetCloud.NoScaler(); + @Before public void before() { final Fleet fleet = mock(Fleet.class); @@ -79,8 +81,8 @@ public void shouldShowNodeConfigurationPage() throws Exception { FleetCloud cloud = new FleetCloud("test-cloud", null, null, null, null, null, "test-label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false); + "-1", false, 0, 0, + 10, false, noScaling); j.jenkins.clouds.add(cloud); j.jenkins.addNode(new FleetNode(nodeName, "", "", 1, @@ -97,8 +99,8 @@ public void shouldReplaceCloudForNodesAfterConfigurationSave() throws Exception FleetCloud cloud = new FleetCloud("test-cloud", null, null, null, null, "", "label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false); + "-1", false, 0, 0, + 10, false, noScaling); j.jenkins.clouds.add(cloud); j.jenkins.addNode(new FleetNode("mock", "", "", 1, @@ -123,8 +125,8 @@ public void shouldShowInConfigurationClouds() throws IOException, SAXException { Cloud cloud = new FleetCloud("TestCloud", null, null, null, null, null, null, null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false); + "-1", false, 0, 0, + 10, false, noScaling); j.jenkins.clouds.add(cloud); HtmlPage page = j.createWebClient().goTo("configureClouds"); @@ -137,15 +139,15 @@ public void shouldShowMultipleClouds() throws IOException, SAXException { Cloud cloud1 = new FleetCloud("a", null, null, null, null, null, "label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false); + "-1", false, 0, 0, + 10, false, noScaling); j.jenkins.clouds.add(cloud1); Cloud cloud2 = new FleetCloud("b", null, null, null, null, null, "label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false); + "-1", false, 0, 0, + 10, false, noScaling); j.jenkins.clouds.add(cloud2); HtmlPage page = j.createWebClient().goTo("configureClouds"); @@ -161,15 +163,15 @@ public void shouldShowMultipleCloudsWithDefaultName() throws IOException, SAXExc Cloud cloud1 = new FleetCloud("TestCloud1", null, null, null, null, null, "label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false); + "-1", false, 0, 0, + 10, false, noScaling); j.jenkins.clouds.add(cloud1); Cloud cloud2 = new FleetCloud("TestCloud2", null, null, null, null, null, "label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false); + "-1", false, 0, 0, + 10, false, noScaling); j.jenkins.clouds.add(cloud2); HtmlPage page = j.createWebClient().goTo("configureClouds"); @@ -185,15 +187,15 @@ public void shouldUpdateProperCloudWhenMultiple() throws Exception { FleetCloud cloud1 = new FleetCloud("TestCloud1", null, null, null, null, null, "label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false); + "-1", false, 0, 0, + 10, false, noScaling); j.jenkins.clouds.add(cloud1); FleetCloud cloud2 = new FleetCloud("TestCloud2", null, null, null, null, null, "label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false); + "-1", false, 0, 0, + 10, false, noScaling); j.jenkins.clouds.add(cloud2); HtmlPage page = j.createWebClient().goTo("configureClouds"); @@ -211,8 +213,8 @@ public void shouldContainRegionValueInRegionLabel() throws IOException, SAXExcep FleetCloud cloud1 = new FleetCloud("TestCloud", "uh", null, null, null, null, "label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false); + "-1", false, 0, 0, + 10, false, noScaling); j.jenkins.clouds.add(cloud1); HtmlPage page = j.createWebClient().goTo("configureClouds"); @@ -233,8 +235,8 @@ public void shouldHaveRegionCodeAndRegionDescriptionInRegionLabel() throws IOExc FleetCloud cloud1 = new FleetCloud("TestCloud", "uh", null, null, null, null, "label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false); + "-1", false, 0, 0, + 10, false, noScaling); j.jenkins.clouds.add(cloud1); HtmlPage page = j.createWebClient().goTo("configureClouds"); @@ -261,15 +263,15 @@ public void shouldGetFirstWhenMultipleCloudWithSameName() { FleetCloud cloud1 = new FleetCloud("TestCloud", null, null, null, null, null, "label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false); + "-1", false, 0, 0, + 10, false, noScaling); j.jenkins.clouds.add(cloud1); FleetCloud cloud2 = new FleetCloud("TestCloud", null, null, null, null, null, "label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false); + "-1", false, 0, 0, + 10, false, noScaling); j.jenkins.clouds.add(cloud2); assertSame(cloud1, j.jenkins.getCloud("TestCloud")); @@ -280,15 +282,15 @@ public void shouldGetProperWhenMultipleWithDiffName() { FleetCloud cloud1 = new FleetCloud("a", null, null, null, null, null, null, null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false); + "-1", false, 0, 0, + 10, false, noScaling); j.jenkins.clouds.add(cloud1); FleetCloud cloud2 = new FleetCloud("b", null, null, null, null, null, null, null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false); + "-1", false, 0, 0, + 10, false, noScaling); j.jenkins.clouds.add(cloud2); assertSame(cloud1, j.jenkins.getCloud("a")); @@ -300,8 +302,8 @@ public void verifyCloudNameReadOnlyAfterCloudCreated() throws Exception { FleetCloud cloud = new FleetCloud("test-cloud", null, null, null, null, "", "label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false); + "-1", false, 0, 0, + 10, false, noScaling); j.jenkins.clouds.add(cloud); HtmlPage page = j.createWebClient().goTo("configureClouds"); @@ -315,13 +317,14 @@ public void verifyExistingDuplicateCloudNamesEditable() throws Exception { j.jenkins.clouds.add(new FleetCloud("test-cloud", null, null, null, null, "", "label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false)); + "-1", false, 0, 0, + 10, false, noScaling)); + j.jenkins.clouds.add(new FleetCloud("test-cloud", null, null, null, null, "", "label", null, null, false, false, 0, 0, 0, 0, 0, true, false, - "-1", false, 0, 0, false, - 10, false)); + "-1", false, 0, 0, + 10, false, noScaling)); HtmlPage page = j.createWebClient().goTo("configureClouds"); diff --git a/src/test/resources/com/amazon/jenkins/ec2fleet/FleetCloud/max-configuration-as-code.yml b/src/test/resources/com/amazon/jenkins/ec2fleet/FleetCloud/max-configuration-as-code.yml index e8a6cc3d..2205201f 100644 --- a/src/test/resources/com/amazon/jenkins/ec2fleet/FleetCloud/max-configuration-as-code.yml +++ b/src/test/resources/com/amazon/jenkins/ec2fleet/FleetCloud/max-configuration-as-code.yml @@ -21,9 +21,9 @@ jenkins: numExecutors: 12 addNodeOnlyIfRunning: true restrictUsage: true - scaleExecutorsByWeight: true initOnlineTimeoutSec: 181 initOnlineCheckIntervalSec: 13 cloudStatusIntervalSec: 11 disableTaskResubmit: true - noDelayProvision: true \ No newline at end of file + noDelayProvision: true + executorScaler: weightedScaler \ No newline at end of file