You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have a service that runs on a single GPU. If I have multiple GPUs available, I would like to create one replica of this service for each GPU available.
Currently, I can only do this by explicitly creating the service multiple times in the docker-compose file and changing the device_ids section of the resources
Having to create 8 near-identical replicas of the same service in the config file is unwieldy.
I would like to specify the service once and set the replicas with
replicas: ${GPU_COUNT}
However this requires some way for each replica to know what GPU to use. My understanding is currently there is no way to do this, and there isn't a good way to go off a "replica index" for each container (see #9153).
Some method of mapping replicas of a service to different GPUs would be helpful.
The text was updated successfully, but these errors were encountered:
In an ideal world the engine (or nvidia driver) would manage this as a pool of resources, just like you can declare a service to bind port with a range, and let the engine select available port in the range, so that scaling is not an issue.
About #9153, I would not be comfortable we rely on container number used to index replicas. While we try to make this somehow sequential there are many corner cases and no guarantee you would always get value within the [1..GPU_COUNT] interval
Currently this is the best solution I've found solely using the docker compose file and not different environment variables in each container. But something akin to the way port ranges are assigned would be nice.
Description
I have a service that runs on a single GPU. If I have multiple GPUs available, I would like to create one replica of this service for each GPU available.
Currently, I can only do this by explicitly creating the service multiple times in the docker-compose file and changing the
device_ids
section of the resourcesHaving to create 8 near-identical replicas of the same service in the config file is unwieldy.
I would like to specify the service once and set the replicas with
However this requires some way for each replica to know what GPU to use. My understanding is currently there is no way to do this, and there isn't a good way to go off a "replica index" for each container (see #9153).
Some method of mapping replicas of a service to different GPUs would be helpful.
The text was updated successfully, but these errors were encountered: