-
Notifications
You must be signed in to change notification settings - Fork 49
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
nsys-jax: re-work to be more pip-install-able #1165
Open
olupton
wants to merge
37
commits into
main
Choose a base branch
from
olupton/pip-install-nsys-jax
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
olupton
force-pushed
the
olupton/pip-install-nsys-jax
branch
from
November 22, 2024 11:01
9520dee
to
d2f268d
Compare
olupton
force-pushed
the
olupton/pip-install-nsys-jax
branch
from
November 22, 2024 13:59
1d0b327
to
80c4ca0
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The overarching goal of this PR is to get closer to a world where the
nsys-jax
tooling is straightforwardlypip install
-able. While the diff looks scary, it's mostly re-organisation.Substantive changes:
nsys-jax
no longer bundles Python code in the output archives, theinstall.sh
script provided for users to run on local machines becomes, loosely,install 'pip nsys-jax[jupyter] @ git+https://github.com/NVIDIA/JAX-Toolbox.git@COMMIT#subdirectory=.github/container/nsys_jax'
, whereCOMMIT
corresponds to thensys-jax
command that produced the archive. For theghcr.io/nvidia/jax
containers, this is the commit of JAX-Toolbox that triggered the container build.Changes included:
/opt/pip-tools-post-install.d
, whichpip-finalize.sh
will execute the contents of after installing thepip
-managed worldinstall-protoc
to use this, sopip-finalize.sh
can forget about that detail.nvtx_gpu_proj_trace
Python code in Nsight Systems 2024.5 and 2024.6 via this.nsys-jax
installation (specifically for the containers) intoinstall-nsys-jax.sh
and thereby clean upinstall-nsight.sh
. The new script has to be told the git commit hash of JAX-Toolbox that is being built, becausensys-jax
bakes this into an installation script in its output.zip
archives to ensure the local environment matches the profile-collection environment.nsys-jax
,nsys-jax-combine
andinstall-protoc
are now handled via[project.scripts]
inpyproject.toml
instead of being standalone Python scripts. This is "more standard", and also makes it easier to share code betweennsys-jax
andnsys-jax-combine
.jax_nsys
tonsys_jax
for consistency.NSYS_JAX_DEFAULT_PREFIX
environment variable; previously the default was the current working directory, but that can be inconvenient to steer in Jupyter environments.