How to pass a Public Docker Image to the Estimater

[kaushikb11]

Motivation

I would like to pass a public Docker Hub Image through the image_uri parameter to the Estimator(Pytorch)

[ChoiByungWook]

Hey @kaushikb11,

I believe SageMaker doesn't support pubic docker images currently and only through ECR. Most likely what you will have to do is pull the public docker image and push it to a ECR repository in your AWS account.

For more information see: https://sagemaker-workshop.com/custom/containers.html

[kaushikb11]

Got it! @ChoiByungWook

Is there a way to run smdistributed data-parallel code without the Estimator for debugging?
As it is only available when you run it with the Estimator.

[akinolawilson]

@kaushikb11

I am facing the same problem.

There is little to no documentation on how exactly this module smdistributed is handled by the Sagemaker training API.

I am currently attempting to use the model parallelism feature of the API and now, following the creation of a custom container based on the Sagemaker PyTorch training toolkit, I receive an error informing that the library; smdistributed is not found.

If there was a way to debug a training script, prior to submitting it to the estimator, it would be very helpful.

[asigiuk]

I'm also having issues passing a public Docker Image on ECR through the image_uri parameter to the Estimator. The training job fails to find this image.
@ChoiByungWook Does this include public docker images on ECR?

This is my code

from sagemaker.estimator import Estimator

byoc_image_uri = 'public.ecr.aws/v8x6t9d9/train-image-classifier-firefly'

estimator = Estimator(image_uri=byoc_image_uri,
                      role=role,
                      base_job_name='tf-custom-container-test-job',
                      instance_count=1,
                      hyperparameters=hyperparameters,
                      output_path=output_path,
                      instance_type='ml.p2.xlarge')