make aws.batch.platformType dynamic

[kubacech]

I have usecase when I would like to use both ec2 and fargate queues and compute environments for one workflow.

To be specific, I am running nf-core/sarek in aws batch. After year of usage I came across data that I am not able to process because of fargate limit of 200gb ephemeral storage. This limit is hit in one of processes. Long story short, at the end I had to split sarek into running in until incriminated job, then run this job on ec2, then continue sarek with outputs of this ec2 job. It took a lot of time and orchestration around it. Except this I am very happy with fargate.

Point is, I believe there should be no obstacle on aws to run some processes on fargate and then some on ec2 compute environments. Maybe correct me, but I think that only obstacle is nexflow doesn't allow configuring this per process or dynamically in any other way. So if I configure fatgate, and set ec2 queue to process I get following error:

Job Queue is attached to Compute Environment that can not run Jobs with capability FARGATE.

I have several ideas, how to solve this, but I don't know implementation details yet and firstly I would like to discuss if this isn't bad idea in first place and potentially what would be best way how to implement it.