[Discussion] sst.aws.Cluster() in dev mode

[abgutierrez]

Hi there,

First of all, thank you for building SST — I currently have several projects running in production using sst/sst. I’m now trying to adopt more native SST components beyond the lower-level Pulumi ones. I really enjoy working with sst.aws.Service, and I’m trying to better understand the design decisions around local development for ECS-based services.

Context

I’m using sst.aws.Service() to deploy ECS services on AWS. When running sst dev, the workflow still creates or references an ECS cluster in AWS, because the Service constructor requires a cluster to be provided via args.cluster.

From what I’ve been able to observe, there doesn’t seem to be an alternative way to run local development without also provisioning (or referencing) an ECS cluster in AWS — even though this is only for local development. In production, of course, requiring a real cluster makes complete sense.

Why I’m confused about needing a cluster up front

Looking at the constructor implementation, it appears that the Service exits early when running in dev mode:

constructor(
  name: string,
  args: ServiceArgs,
  opts: ComponentResourceOptions = {},
) {
  super(__pulumiType, name, args, opts);
  this._name = name;

  const self = this;
  const clusterArn = args.cluster.nodes.cluster.arn;
  const clusterName = args.cluster.nodes.cluster.name;
  const region = getRegionOutput({}, opts).name;
  const dev = normalizeDev();
  const wait = output(args.wait ?? false);
  const architecture = normalizeArchitecture(args);
  const cpu = normalizeCpu(args);
  const memory = normalizeMemory(cpu, args);
  const storage = normalizeStorage(args);
  const containers = normalizeContainers("service", args, name, architecture);
  const lbArgs = normalizeLoadBalancer();
  const scaling = normalizeScaling();
  const capacity = normalizeCapacity();
  const vpc = normalizeVpc();

  const taskRole = createTaskRole(name, args, opts, self, !!dev);

  this.dev = !!dev;
  this.cloudmapNamespace = vpc.cloudmapNamespaceName;
  this.taskRole = taskRole;

  if (dev) {
    this.devUrl = !lbArgs ? undefined : dev.url;
    registerReceiver();
    return;
  }

  [...]
}

Because the if (dev) { ... return; } branch exits relatively early, I’m trying to understand why a fully defined ECS cluster is still required upfront, along with all the associated normalization logic.

Observations

• Many of the normalization steps (region, VPC, load balancer, CPU/memory, etc.) are clearly necessary for a real ECS deployment.
• However, for local development, it seems that several of these validations could potentially be skipped or deferred.
• Requiring an ECS cluster during sst dev increases the initial startup time and can introduce unnecessary AWS cost if clusters are kept running only for development purposes.

Questions

• Is it intentional that sst.aws.Service requires a real ECS cluster even when running in dev mode?

• Is there an architectural reason why cluster-related normalization must happen before the early dev return?

• Is there any plan to introduce something like:

  • a lightweight or mocked ClusterArgs for dev, or
  • an extension of Cluster with explicit dev-only capabilities (similar in spirit to how EKS local workflows are handled)?

If this behavior is by design, I’d really appreciate any insight into the rationale behind it, or pointers to documentation or code paths that explain the constraints.

Thanks again for the project, and happy New Year! 🎉

[lachlanmacphee]

Happy new year all and +1 to this discussion, thanks for raising @abgutierrez

I'm also wondering why sst needs to deploy a Cluster (and VPC) when all I really need to develop locally is my docker compose stack (including Postgres) and my fullstack app (+ maybe an S3 bucket - but ideally I'd be able to swap this out for a local equivalent). Now that I've adopted SST throughout my app it makes it hard to go back.

Ideally I'd just like to use sst for production deploys but still be able to pass env variables in through Resource locally - can't see anything in docs that show how to do that.

Config for reference:

const isProd = ['production'].includes($app.stage);
const vpc = new sst.aws.Vpc('VPC', { bastion: true });

const database = new sst.aws.Postgres('Database', {
	instance: 't4g.micro',
	version: '18.1',
	vpc,
	dev: {
		username: 'postgres',
		password: 'password',
		database: 'eddi',
		host: 'localhost',
		port: 5432
	},
	proxy: false
});

new sst.x.DevCommand('Studio', {
	link: [database],
	dev: {
		command: 'npx drizzle-kit studio'
	}
});

const bucket = new sst.aws.Bucket('BucketSchools');

const cluster = new sst.aws.Cluster('Cluster', {
	vpc
});

// sst secret definitions here

const GEMINI_DEFAULT_MODEL = 'gemini-2.5-flash';
const GEMINI_DEFAULT_IMAGE_MODEL = 'gemini-2.5-flash-image-generation';

const app = new sst.aws.Service('EddiApp', {
	cluster,
	capacity: 'spot',
	link: [
		bucket,
		database,
		// secret links here
	],
	environment: {
		GEMINI_DEFAULT_MODEL,
		GEMINI_DEFAULT_IMAGE_MODEL
	},
	image: {
		context: '.',
		dockerfile: 'infra/Dockerfile'
	},
	loadBalancer: {
		domain: {
			name: 'eddi.com.au',
			aliases: ['*.eddi.com.au']
		},
		rules: [
			{ listen: '80/http', redirect: '443/https' },
			{ listen: '443/https', forward: '3000/http' }
		]
	},
	// Set to defaults for now
	scaling: {
		min: 1,
		max: 1
	},
	dev: {
		command: 'npm run dev'
	}
});

const router = isProd
	? new sst.aws.Router('DomainRedirects', {
		routes: {
			'/*': app.url
		},
		domain: {
			name: 'www.eddi.com.au',
			aliases: ['eddi.au', 'www.eddi.au']
		}
	})
	: null;

return {
	resources: { vpc, database, bucket, cluster, app, router }
};

[abgutierrez]

@lachlanmacphee
I follow a similar approach to yours, but I include both the VPC and the ECS cluster in the development stage to achieve better isolation.

That said, I would like to improve the sst dev lifecycle. In my workflow, developers use their own development stage with docker-compose watch and SST Live mode (hot reload). As a result, for day-to-day work I usually run:

bunx sst deploy --stage dev in the morning

bunx sst remove --stage dev at the end of the day

(Occasionally I forget to remove the stack, which means the cluster remains running, incurs unnecessary costs, and is not actually used, since all development and testing are performed locally.)

I was unable to find clear documentation explaining the differences between sst dev and sst deploy --stage dev. However, in my case this approach works well, as $dev is set to false, and we can rely on patterns like the following (based on a similar snippet from the official documentation):

const cluster = $dev
  ? sst.aws.Cluster.get("cluster-dev", {
      id: "arn:aws:ecs:us-east-1:123456789012:cluster/app-dev-MyCluster",
      vpc,
    })
  : new sst.aws.Cluster("cluster-production", { vpc });

This is essentially the extent of the infrastructure cost savings I was able to achieve. However, it still feels somewhat unintuitive to reuse AWS resources solely because they are required to satisfy parameter normalization.

[lachlanmacphee]

@abgutierrez

I include both the VPC and the ECS cluster in the development stage to achieve better isolation.

I assume by your code snippet that you force sst to utilise the predefined dedicated VPC and Cluster (in dev) that you have defined, so that rather than each dev having their own VPC and Cluster they utilise the shared ones? I've now done the same, thanks for the idea, appreciate that!

I added this snippet so that each dev is forced to use the sst dev --stage dev command when developing locally (to prevent unnecessary VPCs/Clusters)

const isDev = $app.stage === 'dev';
const isProd = $app.stage === 'production';
if (!isDev && !isProd) {
	throw new Error(`Invalid stage: ${$app.stage}. Must be 'dev' or 'production'.`);
}

I was unable to find clear documentation explaining the differences between sst dev and sst deploy --stage dev

Does this help at all? I believe they are actually quite different in their behaviour. One utilises a locally deployed stack rather than the AWS one. If you're not running Postgres or the app locally then it probably wouldn't make a difference to you.

https://sst.dev/docs/#dev
https://sst.dev/docs/reference/cli/#flags-2

[abgutierrez]

I assume by your code snippet that you force sst to utilise the predefined dedicated VPC and Cluster (in dev) that you have defined, so that rather than each dev having their own VPC and Cluster they utilise the shared ones? I've now done the same, thanks for the idea, appreciate that!

Exactly. Sharing the VPC and cluster is not only about cost reduction; it also helps avoid the initial deployment delay that each developer would otherwise experience when provisioning their own infrastructure.

I believe they are actually quite different in their behaviour.

You’re right — I may not have been completely clear. I do understand that sst dev and sst deploy --stage dev behave differently, particularly in where and how resources are deployed.

From my understanding, sst deploy --stage dev behaves much like a production-grade deployment, just under a different stage name. On the other hand, sst dev sets $dev = true at runtime and skips actual deployment for components that define a dev attribute, relying instead on local execution.

Regarding this snippet:

const isDev = $app.stage === 'dev';
const isProd = $app.stage === 'production';

I think this logic could be simplified. Based on the documentation (https://sst.dev/docs/#dev), using the $dev flag alone is usually sufficient to distinguish between sst dev and sst deploy.

For example, when running sst dev, the username is used as the stage name by default. Relying strictly on isDev / isProd checks may limit flexibility in the future, especially if you later want to support additional ephemeral stages such as staging, per-PR deployments, or branch-based stages. This could pair nicely with features like auto-deploys:
https://sst.dev/docs/console/#autodeploy

Another potential issue arises when the same dev stage is shared by multiple developers. Ideally, each developer should use a distinct stage to avoid SST unintentionally refreshing or mutating shared AWS resources. For example, if two developers run sst dev --stage dev simultaneously, changes made by one could conflict with or overwrite the other’s resources during development.