Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions docs/azure.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -619,6 +619,16 @@ Nextflow can submit tasks to existing pools that are already attached to a virtu
Virtual networks require Microsoft Entra authentication (service principal or managed identity).
:::

**Node communication mode**

Pools can be configured to use [simplified compute node communication](https://learn.microsoft.com/en-us/azure/batch/simplified-compute-node-communication), which changes how the Batch service communicates with the compute nodes and reduces the outbound network access required by the nodes.

```groovy
azure.batch.pools.<pool-name>.targetCommunicationMode = 'simplified' // 'simplified' or 'classic'
```

When this option is not set, the Azure Batch service selects the communication mode.
Comment on lines +624 to +630

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Pools can be configured to use [simplified compute node communication](https://learn.microsoft.com/en-us/azure/batch/simplified-compute-node-communication), which changes how the Batch service communicates with the compute nodes and reduces the outbound network access required by the nodes.
```groovy
azure.batch.pools.<pool-name>.targetCommunicationMode = 'simplified' // 'simplified' or 'classic'
```
When this option is not set, the Azure Batch service selects the communication mode.
To use [simplified compute node communication](https://learn.microsoft.com/en-us/azure/batch/simplified-compute-node-communication), set `targetCommunicationMode` to `simplified`. This changes how the Batch service communicates with the compute nodes and reduces the outbound network access the nodes require.
```groovy
azure.batch.pools.<pool-name>.targetCommunicationMode = 'simplified' // 'simplified' or 'classic'
```
When this option is not set, the Azure Batch service selects the communication mode.


**Start tasks**

Start tasks are optional commands that run when a node joins the pool. They are useful for setting up the node environment or for running other commands that need to be executed when the node is created.
Expand Down
4 changes: 4 additions & 0 deletions docs/reference/config.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -577,6 +577,10 @@ Enable the `startTask` to run with elevated access (default`false`).

The `startTask` that is executed as the node joins the Azure Batch node pool.

##### `azure.batch.pools.<name>.targetCommunicationMode`

The node communication mode used by the pool. Can be either `classic` or `simplified`. When not set, the Azure Batch service selects the mode. In `simplified` mode the Batch service initiates communication with the compute nodes, reducing the outbound network access required by the nodes.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The node communication mode used by the pool. Can be either `classic` or `simplified`. When not set, the Azure Batch service selects the mode. In `simplified` mode the Batch service initiates communication with the compute nodes, reducing the outbound network access required by the nodes.
The node communication mode used by the pool. One of `classic` or `simplified`. When not set, the Azure Batch service selects the mode. In `simplified` mode, the Batch service initiates communication with the compute nodes, which reduces the outbound network access the nodes require.


##### `azure.batch.pools.<name>.virtualNetwork`

<AddedInVersion version="23.04" />
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -952,6 +952,10 @@ class AzBatchService implements Closeable {
poolParams.setTaskSchedulingPolicy( new BatchTaskSchedulingPolicy(pol) )
}

// node communication mode
if( spec.opts.targetCommunicationMode )
poolParams.setTargetNodeCommunicationMode( spec.opts.targetCommunicationMode )

// mount points
if ( config.storage().fileShares ) {
List<MountConfiguration> mountConfigs = new ArrayList(config.storage().fileShares.size())
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@

package nextflow.cloud.azure.config

import com.azure.compute.batch.models.BatchNodeCommunicationMode
import com.azure.compute.batch.models.ImageVerificationType
import com.azure.compute.batch.models.OSType
import com.google.common.hash.Hasher
Expand Down Expand Up @@ -144,6 +145,12 @@ class AzPoolOpts implements CacheFunnel, ConfigScope {
""")
final String vmType

@ConfigOption
@Description("""
The node communication mode used by the pool. Can be either `classic` or `simplified`. When not set, the Azure Batch service selects the mode. In `simplified` mode the Batch service initiates communication with the compute nodes, reducing the outbound network access required by the nodes.
""")
final BatchNodeCommunicationMode targetCommunicationMode

OSType osType = DEFAULT_OS_TYPE
ImageVerificationType verification = ImageVerificationType.VERIFIED

Expand Down Expand Up @@ -175,6 +182,18 @@ class AzPoolOpts implements CacheFunnel, ConfigScope {
this.password = opts.password
this.virtualNetwork = opts.virtualNetwork
this.lowPriority = opts.lowPriority as boolean
this.targetCommunicationMode = parseCommunicationMode(opts.targetCommunicationMode)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Nextflow team have the final say on style, but it may be more disciplined to only use parseCommunicationMode when targetCommunicationMode is populated:

Suggested change
this.targetCommunicationMode = parseCommunicationMode(opts.targetCommunicationMode)
this.targetCommunicationMode = opts.targetCommunicationMode != null ?parseCommunicationMode(opts.targetCommunicationMode) : null

Style choice, it's been a while since I've contributed so I want one of the core developers to look at it!

}

protected static BatchNodeCommunicationMode parseCommunicationMode(value) {
if( value == null )
return null
if( value instanceof BatchNodeCommunicationMode )
return value
final str = value.toString().toLowerCase()
if( str == 'simplified' ) return BatchNodeCommunicationMode.SIMPLIFIED
if( str == 'classic' ) return BatchNodeCommunicationMode.CLASSIC
throw new IllegalArgumentException("Invalid azure.batch.pools.<name>.targetCommunicationMode value: '$value' - expected 'simplified' or 'classic'")
Comment on lines +188 to +196

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could include an explicit "default" here:

I've also removed the using BatchNodeCommunication mode because I think it makes the method confusing.

Suggested change
protected static BatchNodeCommunicationMode parseCommunicationMode(value) {
if( value == null )
return null
if( value instanceof BatchNodeCommunicationMode )
return value
final str = value.toString().toLowerCase()
if( str == 'simplified' ) return BatchNodeCommunicationMode.SIMPLIFIED
if( str == 'classic' ) return BatchNodeCommunicationMode.CLASSIC
throw new IllegalArgumentException("Invalid azure.batch.pools.<name>.targetCommunicationMode value: '$value' - expected 'simplified' or 'classic'")
protected static BatchNodeCommunicationMode parseCommunicationMode(value) {
final str = value.toString().toLowerCase()
if( str == 'simplified' ) return BatchNodeCommunicationMode.SIMPLIFIED
if( str == 'classic' ) return BatchNodeCommunicationMode.CLASSIC
if( str == 'default' ) return BatchNodeCommunicationMode.DEFAULT
throw new IllegalArgumentException("Invalid azure.batch.pools.<name>.targetCommunicationMode value: '$value' - expected 'simplified', 'classic' or 'default'")
}

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: default is likely functionally equivalent to leaving the option unset but I'm not certain.

I'm about 99% certain that that's the case. However, I can't find any clear documentation on that...

However, given that this option was never explicitly set up until now, we can safely conclude that leaving it empty means it takes the 'Default' option.

}

@Override
Expand All @@ -195,6 +214,7 @@ class AzPoolOpts implements CacheFunnel, ConfigScope {
hasher.putUnencodedChars(schedulePolicy ?: '')
hasher.putUnencodedChars(virtualNetwork ?: '')
hasher.putBoolean(lowPriority)
hasher.putUnencodedChars(targetCommunicationMode?.toString() ?: '')
hasher.putUnencodedChars(startTask.script ?: '')
hasher.putBoolean(startTask.privileged)
return hasher
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@

package nextflow.cloud.azure.config

import com.azure.compute.batch.models.BatchNodeCommunicationMode
import com.google.common.hash.Hashing
import nextflow.util.CacheHelper
import nextflow.util.Duration
import spock.lang.Specification
/**
Expand Down Expand Up @@ -48,6 +51,39 @@ class AzPoolOptsTest extends Specification {
!opts.lowPriority
!opts.startTask.script
!opts.startTask.privileged
opts.targetCommunicationMode == null
}

def 'should parse the target communication mode' () {
expect:
new AzPoolOpts([targetCommunicationMode: VALUE]).targetCommunicationMode == EXPECTED
where:
VALUE | EXPECTED
null | null
'simplified' | BatchNodeCommunicationMode.SIMPLIFIED
'classic' | BatchNodeCommunicationMode.CLASSIC
'SIMPLIFIED' | BatchNodeCommunicationMode.SIMPLIFIED
'CLASSIC' | BatchNodeCommunicationMode.CLASSIC
}

def 'should reject an invalid target communication mode' () {
when:
new AzPoolOpts([targetCommunicationMode: 'bogus'])
then:
def e = thrown(IllegalArgumentException)
e.message.contains('targetCommunicationMode')
}

private static String hash(AzPoolOpts opts) {
opts.funnel(Hashing.murmur3_128().newHasher(), CacheHelper.HashMode.STANDARD).hash().toString()
}

def 'pool hash should differ when communication mode differs' () {
given:
def base = new AzPoolOpts()
def classic = new AzPoolOpts([targetCommunicationMode: 'classic'])
expect:
hash(base) != hash(classic)
}

def 'should create pool with custom options' () {
Expand Down
Loading