Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

utils_misc can't find freer GPU #4

Open
corey-cole opened this issue Dec 26, 2022 · 0 comments
Open

utils_misc can't find freer GPU #4

corey-cole opened this issue Dec 26, 2022 · 0 comments

Comments

@corey-cole
Copy link

The nvidia-smi parsing code results in an empty sequence.

>>> import utils_misc
>>> utils_misc.select_freer_gpu()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/corey/workspace/keep_it_simple/utils_misc.py", line 24, in select_freer_gpu
    freer_gpu = str(get_freer_gpu())
  File "/home/corey/workspace/keep_it_simple/utils_misc.py", line 11, in get_freer_gpu
    return np.argmax(memory_available)
  File "<__array_function__ internals>", line 200, in argmax
  File "/home/corey/workspace/keep_it_simple/.venv/lib/python3.10/site-packages/numpy/core/fromnumeric.py", line 1242, in argmax
    return _wrapfunc(a, 'argmax', axis=axis, out=out, **kwds)
  File "/home/corey/workspace/keep_it_simple/.venv/lib/python3.10/site-packages/numpy/core/fromnumeric.py", line 54, in _wrapfunc
    return _wrapit(obj, method, *args, **kwds)
  File "/home/corey/workspace/keep_it_simple/.venv/lib/python3.10/site-packages/numpy/core/fromnumeric.py", line 43, in _wrapit
    result = getattr(asarray(obj), method)(*args, **kwds)
ValueError: attempt to get argmax of an empty sequence

Changing to 'grep -A5' appears to work for running the sample on a system with one consumer GPU, but I'm not equipped to evaluate the overall impact of the change.

os.system('nvidia-smi -q -d Memory |grep -A5 GPU|grep Free >tmp_smi')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant