Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updated Pytorch image changes for ppc64le #3

Closed
wants to merge 8 commits into from
Closed

Updated Pytorch image changes for ppc64le #3

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
2a8404a
Updated Pytorch image changes for ppc64le
bhagyashrigai Jul 3, 2024
1f57851
updated code with enabling codeflare-sdk
bhagyashrigai Jul 12, 2024
4a529fc
updated sha values
bhagyashrigai Jul 12, 2024
3bbd4ea
removed extra line
bhagyashrigai Jul 12, 2024
e2a3fef
updated code
bhagyashrigai Jul 12, 2024
861089b
updated code for root user
bhagyashrigai Jul 12, 2024
f4a6161
revert back changes
bhagyashrigai Jul 12, 2024
6e22a62
updated code for datascience image
bhagyashrigai Jul 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -176,9 +176,14 @@ cuda-jupyter-datascience-ubi9-python-3.9: cuda-jupyter-minimal-ubi9-python-3.9
cuda-jupyter-tensorflow-ubi9-python-3.9: cuda-jupyter-datascience-ubi9-python-3.9
$(call image,$@,jupyter/tensorflow/ubi9-python-3.9,$<)

# Build and push jupyter-pytorch-ubi9-python-3.9 image to the registry
.PHONY: cuda-jupyter-pytorch-ubi9-python-3.9
cuda-jupyter-pytorch-ubi9-python-3.9: cuda-jupyter-datascience-ubi9-python-3.9
$(call image,$@,jupyter/pytorch/ubi9-python-3.9,$<)

# Build and push jupyter-pytorch-ubi9-python-3.9 image to the registry
.PHONY: jupyter-pytorch-ubi9-python-3.9
jupyter-pytorch-ubi9-python-3.9: cuda-jupyter-datascience-ubi9-python-3.9
jupyter-pytorch-ubi9-python-3.9: jupyter-datascience-ubi9-python-3.9
bhagyashrigai marked this conversation as resolved.
Show resolved Hide resolved
$(call image,$@,jupyter/pytorch/ubi9-python-3.9,$<)

# Build and push jupyter-trustyai-ubi9-python-3.9 image to the registry
Expand Down
6 changes: 6 additions & 0 deletions jupyter/pytorch/ubi9-python-3.9/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ LABEL name="odh-notebook-jupyter-pytorch-ubi9-python-3.9" \
# Install Python packages and Jupyterlab extensions from Pipfile.lock
COPY Pipfile.lock ./

# Switch to root user
USER root
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any specific reason for switching to root here?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There was a permission issue when attempting to write the pipfile.lock file to the container image. [Error:-Failed to write lockfile to container image: [Errno 13] Permission denied: 'Pipfile.lock']
hence above changes were done.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets debug that instead of making root context here, use podman.. or check if you are facing any selinux related issue etc..

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mkumatag @npanpaliya
I haven't found any other error in the logs besides this 'permission denied'( [Error:-Failed to write lockfile to container image: [Errno 13] Permission denied: 'Pipfile.lock']) while building the image.
Instead of using the root user, can we use then:-
COPY --chown=1001:0 Pipfile.lock ./
Could you please guide me further on that

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my point is if this docker file is built on x86 then same file should on ppc64le as well, I don't see a point in modifying just for power!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this because of the different base image? 🤔 if yes - lets fix the base image accordingly..

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably this is because of using a non root user.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mkumatag @npanpaliya

  1. After updating the notebook repository to the latest tag v1.20.0, I have successfully built the PyTorch image.
    No root user changes are required inside this Dockerfile. All packages installed successfully into the container.
    Snap:-
powercommand_snap
  1. Additionally, I tested the base image building on an x86 VM. It was observed that the build encounters the same error "failed to write lockfile to container image." However, this error does not impact any functionality and can be safely ignored.

Snap:-
x86_baseimage

  1. I need to close this PR because the base branch has been updated with the latest changes. I will create a new PR for this pytorch and datascience image builds changes."


RUN echo "Installing softwares and packages" && \
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && \
source "$HOME/.cargo/env" && \
Expand All @@ -28,3 +31,6 @@ RUN echo "Installing softwares and packages" && \
# Fix permissions to support pip in Openshift environments
RUN chmod -R g+w /opt/app-root/lib/python3.9/site-packages && \
fix-permissions /opt/app-root -P

# Switch back to non-root user
USER 1001
13 changes: 11 additions & 2 deletions jupyter/pytorch/ubi9-python-3.9/Pipfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,22 @@ url = "https://download.pytorch.org/whl/cu121"
verify_ssl = true
name = "pytorch"

[[source]]
url = "http://10.20.177.222:9000/"
verify_ssl = false
name = "localpi"

[dev-packages]

[packages]
# PyTorch packages
tensorboard = "~=2.16.2"
torch = {version = "~=2.2.1", index = "pytorch"}
torchvision = {version = "~=0.17.1", index = "pytorch"}
torch = {version = "2.2.1a0", index = "localpi"}
torchvision = {version = "0.17.*", index = "localpi"}
ray = {version = "2.7.0.*", index="localpi"}
py-spy = {version = "0.3.14.*", index="localpi"}
pyarrow = {version = "15.0.1.dev0+g5ce6ff434.d20240426", index="localpi"}

# Datascience and useful extensions
boto3 = "~=1.34.50"
kafka-python = "~=2.0.2"
Expand Down
Loading