Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

./TensorNVMe/include/aio.h:3:10: fatal error: libaio.h: No such file or directory #43

Open
marsggbo opened this issue Mar 1, 2023 · 3 comments
Open

./TensorNVMe/include/aio.h:3:10: fatal error: libaio.h: No such file or directory #43

marsggbo opened this issue Mar 1, 2023 · 3 comments

Comments

@marsggbo
Copy link

marsggbo commented Mar 1, 2023

how to solve this problem without root priority when installing tensornvme?
@Angelconan
Copy link

mark，I have the same problem！

@Angelconan
Copy link

I solved this problem by compiling locally

@marsggbo
Copy link
Author

marsggbo commented Mar 16, 2023 •
edited
Loading

I solved this problem by compiling locally

Hi @Angelconan , could you share your installation steps?

  • cmake version: 3.22.1
  • gcc version: 7.3.0

I tried to install liburing and libaio locally by using their source code, but when I ran pip install tensornvme, I got the error below:

Processing /home/xihe/.tensornvme/TensorNVMe
  Running command python setup.py egg_info
  running egg_info
  creating /tmp/pip-pip-egg-info-hbii2su8/tensornvme.egg-info
  writing /tmp/pip-pip-egg-info-hbii2su8/tensornvme.egg-info/PKG-INFO
  writing dependency_links to /tmp/pip-pip-egg-info-hbii2su8/tensornvme.egg-info/dependency_links.txt
  writing entry points to /tmp/pip-pip-egg-info-hbii2su8/tensornvme.egg-info/entry_points.txt
  writing requirements to /tmp/pip-pip-egg-info-hbii2su8/tensornvme.egg-info/requires.txt
  writing top-level names to /tmp/pip-pip-egg-info-hbii2su8/tensornvme.egg-info/top_level.txt
  writing manifest file '/tmp/pip-pip-egg-info-hbii2su8/tensornvme.egg-info/SOURCES.txt'
  reading manifest file '/tmp/pip-pip-egg-info-hbii2su8/tensornvme.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  writing manifest file '/tmp/pip-pip-egg-info-hbii2su8/tensornvme.egg-info/SOURCES.txt'
  Preparing metadata (setup.py) ... done
Requirement already satisfied: packaging in /datasets/xihe/miniconda3/envs/colossal/lib/python3.9/site-packages (from tensornvme==0.1.0) (21.3)
Requirement already satisfied: click in /datasets/xihe/miniconda3/envs/colossal/lib/python3.9/site-packages (from tensornvme==0.1.0) (8.1.3)
Requirement already satisfied: torch in /datasets/xihe/miniconda3/envs/colossal/lib/python3.9/site-packages (from tensornvme==0.1.0) (1.12.1+cu113)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /datasets/xihe/miniconda3/envs/colossal/lib/python3.9/site-packages (from packaging->tensornvme==0.1.0) (3.0.9)
Requirement already satisfied: typing-extensions in /datasets/xihe/miniconda3/envs/colossal/lib/python3.9/site-packages (from torch->tensornvme==0.1.0) (4.3.0)
Building wheels for collected packages: tensornvme
  Running command python setup.py bdist_wheel
  -- The CXX compiler identification is GNU 7.3.0
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: /cm/extra/apps/GCC/7.3.0/GCC-4.8.5_GMP-6.1.2_MPFR-4.0.1_MPC-1.1.0/bin/c++ - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  liburing is found in /home/xihe/.tensornvme/libs/liburing/lib/liburing.so
  libaio is found in /home/xihe/.tensornvme/libs/libaio/lib/libaio.so
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /home/xihe/.tensornvme/TensorNVMe/cmake-build
  make: *** No rule to make target `extern_aio'.  Stop.
  No CUDA runtime is found, using CUDA_HOME='/cm/extra/Utils/CUDA/11.1.0.0_455.23.05'
  running bdist_wheel
  running build
  running build_py
  running build_ext
  building 'tensornvme._C' extension
  Emitting ninja build file /home/xihe/.tensornvme/TensorNVMe/build/temp.linux-x86_64-cpython-39/build.ninja...
  Compiling objects...
  Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
  ninja: no work to do.
  g++ -pthread -B /datasets/xihe/miniconda3/envs/colossal/compiler_compat -shared -Wl,-rpath,/datasets/xihe/miniconda3/envs/colossal/lib -Wl,-rpath-link,/datasets/xihe/miniconda3/envs/colossal/lib -L/datasets/xihe/miniconda3/envs/colossal/lib -L/datasets/xihe/miniconda3/envs/colossal/lib -Wl,-rpath,/datasets/xihe/miniconda3/envs/colossal/lib -Wl,-rpath-link,/datasets/xihe/miniconda3/envs/colossal/lib -L/datasets/xihe/miniconda3/envs/colossal/lib /home/xihe/.tensornvme/TensorNVMe/build/temp.linux-x86_64-cpython-39/home/xihe/.tensornvme/TensorNVMe/csrc/aio.o /home/xihe/.tensornvme/TensorNVMe/build/temp.linux-x86_64-cpython-39/home/xihe/.tensornvme/TensorNVMe/csrc/offload.o /home/xihe/.tensornvme/TensorNVMe/build/temp.linux-x86_64-cpython-39/home/xihe/.tensornvme/TensorNVMe/csrc/space_mgr.o -L/home/xihe/.tensornvme/lib -L/datasets/xihe/miniconda3/envs/colossal/lib/python3.9/site-packages/torch/lib -laio -lc10 -ltorch -ltorch_cpu -ltorch_python -o build/lib.linux-x86_64-cpython-39/tensornvme/_C.cpython-39-x86_64-linux-gnu.so
  /datasets/xihe/miniconda3/envs/colossal/compiler_compat/ld: cannot find -laio: No such file or directory
  collect2: error: ld returned 1 exit status
  error: command '/cm/extra/apps/GCC/7.3.0/GCC-4.8.5_GMP-6.1.2_MPFR-4.0.1_MPC-1.1.0/bin/g++' failed with exit code 1
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /datasets/xihe/miniconda3/envs/colossal/bin/python -u -c '
  exec(compile('"'"''"'"''"'"'
  # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
  #
  # - It imports setuptools before invoking setup.py, to enable projects that directly
  #   import from `distutils.core` to work with newer packaging standards.
  # - It provides a clear error message when setuptools is not installed.
  # - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
  #   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
  #     manifest_maker: standard file '"'"'-c'"'"' not found".
  # - It generates a shim setup.py, for handling setup.cfg-only projects.
  import os, sys, tokenize
  
  try:
      import setuptools
  except ImportError as error:
      print(
          "ERROR: Can not execute `setup.py` since setuptools is not available in "
          "the build environment.",
          file=sys.stderr,
      )
      sys.exit(1)
  
  __file__ = %r
  sys.argv[0] = __file__
  
  if os.path.exists(__file__):
      filename = __file__
      with tokenize.open(__file__) as f:
          setup_py_code = f.read()
  else:
      filename = "<auto-generated setuptools caller>"
      setup_py_code = "from setuptools import setup; setup()"
  
  exec(compile(setup_py_code, filename, "exec"))
  '"'"''"'"''"'"' % ('"'"'/home/xihe/.tensornvme/TensorNVMe/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' bdist_wheel -d /tmp/pip-wheel-dhb1gpbd
  cwd: /home/xihe/.tensornvme/TensorNVMe/
  Building wheel for tensornvme (setup.py) ... error
  ERROR: Failed building wheel for tensornvme
  Running setup.py clean for tensornvme
  Running command python setup.py clean
  running clean
  removing 'build/temp.linux-x86_64-cpython-39' (and everything under it)
  'build/lib' does not exist -- can't clean it
  'build/bdist.linux-x86_64' does not exist -- can't clean it
  'build/scripts-3.9' does not exist -- can't clean it
Failed to build tensornvme
Installing collected packages: tensornvme
  Running command Running setup.py install for tensornvme
  liburing is found in /home/xihe/.tensornvme/libs/liburing/lib/liburing.so
  libaio is found in /home/xihe/.tensornvme/libs/libaio/lib/libaio.so
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /home/xihe/.tensornvme/TensorNVMe/cmake-build
  make: *** No rule to make target `extern_aio'.  Stop.
  No CUDA runtime is found, using CUDA_HOME='/cm/extra/Utils/CUDA/11.1.0.0_455.23.05'
  running install
  /datasets/xihe/miniconda3/envs/colossal/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
    warnings.warn(
  running build
  running build_py
  running build_ext
  building 'tensornvme._C' extension
  creating /home/xihe/.tensornvme/TensorNVMe/build/temp.linux-x86_64-cpython-39
  creating /home/xihe/.tensornvme/TensorNVMe/build/temp.linux-x86_64-cpython-39/home
  creating /home/xihe/.tensornvme/TensorNVMe/build/temp.linux-x86_64-cpython-39/home/xihe
  creating /home/xihe/.tensornvme/TensorNVMe/build/temp.linux-x86_64-cpython-39/home/xihe/.tensornvme
  creating /home/xihe/.tensornvme/TensorNVMe/build/temp.linux-x86_64-cpython-39/home/xihe/.tensornvme/TensorNVMe
  creating /home/xihe/.tensornvme/TensorNVMe/build/temp.linux-x86_64-cpython-39/home/xihe/.tensornvme/TensorNVMe/csrc
  Emitting ninja build file /home/xihe/.tensornvme/TensorNVMe/build/temp.linux-x86_64-cpython-39/build.ninja...
  Compiling objects...
  Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
  [1/3] c++ -MMD -MF /home/xihe/.tensornvme/TensorNVMe/build/temp.linux-x86_64-cpython-39/home/xihe/.tensornvme/TensorNVMe/csrc/space_mgr.o.d -pthread -B /datasets/xihe/miniconda3/envs/colossal/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /datasets/xihe/miniconda3/envs/colossal/include -I/datasets/xihe/miniconda3/envs/colossal/include -fPIC -O2 -isystem /datasets/xihe/miniconda3/envs/colossal/include -fPIC -DDISABLE_URING -I/home/xihe/.tensornvme/TensorNVMe/csrc -I/home/xihe/.tensornvme/TensorNVMe/include -I/home/xihe/.tensornvme/include -I/cm/extra/apps/GCC/7.3.0/GCC-4.8.5_GMP-6.1.2_MPFR-4.0.1_MPC-1.1.0/include -I/datasets/xihe/miniconda3/envs/colossal/lib/python3.9/site-packages/torch/include -I/datasets/xihe/miniconda3/envs/colossal/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -I/datasets/xihe/miniconda3/envs/colossal/lib/python3.9/site-packages/torch/include/TH -I/datasets/xihe/miniconda3/envs/colossal/lib/python3.9/site-packages/torch/include/THC -I/datasets/xihe/miniconda3/envs/colossal/include/python3.9 -c -c /home/xihe/.tensornvme/TensorNVMe/csrc/space_mgr.cpp -o /home/xihe/.tensornvme/TensorNVMe/build/temp.linux-x86_64-cpython-39/home/xihe/.tensornvme/TensorNVMe/csrc/space_mgr.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
  [2/3] c++ -MMD -MF /home/xihe/.tensornvme/TensorNVMe/build/temp.linux-x86_64-cpython-39/home/xihe/.tensornvme/TensorNVMe/csrc/aio.o.d -pthread -B /datasets/xihe/miniconda3/envs/colossal/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /datasets/xihe/miniconda3/envs/colossal/include -I/datasets/xihe/miniconda3/envs/colossal/include -fPIC -O2 -isystem /datasets/xihe/miniconda3/envs/colossal/include -fPIC -DDISABLE_URING -I/home/xihe/.tensornvme/TensorNVMe/csrc -I/home/xihe/.tensornvme/TensorNVMe/include -I/home/xihe/.tensornvme/include -I/cm/extra/apps/GCC/7.3.0/GCC-4.8.5_GMP-6.1.2_MPFR-4.0.1_MPC-1.1.0/include -I/datasets/xihe/miniconda3/envs/colossal/lib/python3.9/site-packages/torch/include -I/datasets/xihe/miniconda3/envs/colossal/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -I/datasets/xihe/miniconda3/envs/colossal/lib/python3.9/site-packages/torch/include/TH -I/datasets/xihe/miniconda3/envs/colossal/lib/python3.9/site-packages/torch/include/THC -I/datasets/xihe/miniconda3/envs/colossal/include/python3.9 -c -c /home/xihe/.tensornvme/TensorNVMe/csrc/aio.cpp -o /home/xihe/.tensornvme/TensorNVMe/build/temp.linux-x86_64-cpython-39/home/xihe/.tensornvme/TensorNVMe/csrc/aio.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
  [3/3] c++ -MMD -MF /home/xihe/.tensornvme/TensorNVMe/build/temp.linux-x86_64-cpython-39/home/xihe/.tensornvme/TensorNVMe/csrc/offload.o.d -pthread -B /datasets/xihe/miniconda3/envs/colossal/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /datasets/xihe/miniconda3/envs/colossal/include -I/datasets/xihe/miniconda3/envs/colossal/include -fPIC -O2 -isystem /datasets/xihe/miniconda3/envs/colossal/include -fPIC -DDISABLE_URING -I/home/xihe/.tensornvme/TensorNVMe/csrc -I/home/xihe/.tensornvme/TensorNVMe/include -I/home/xihe/.tensornvme/include -I/cm/extra/apps/GCC/7.3.0/GCC-4.8.5_GMP-6.1.2_MPFR-4.0.1_MPC-1.1.0/include -I/datasets/xihe/miniconda3/envs/colossal/lib/python3.9/site-packages/torch/include -I/datasets/xihe/miniconda3/envs/colossal/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -I/datasets/xihe/miniconda3/envs/colossal/lib/python3.9/site-packages/torch/include/TH -I/datasets/xihe/miniconda3/envs/colossal/lib/python3.9/site-packages/torch/include/THC -I/datasets/xihe/miniconda3/envs/colossal/include/python3.9 -c -c /home/xihe/.tensornvme/TensorNVMe/csrc/offload.cpp -o /home/xihe/.tensornvme/TensorNVMe/build/temp.linux-x86_64-cpython-39/home/xihe/.tensornvme/TensorNVMe/csrc/offload.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
  g++ -pthread -B /datasets/xihe/miniconda3/envs/colossal/compiler_compat -shared -Wl,-rpath,/datasets/xihe/miniconda3/envs/colossal/lib -Wl,-rpath-link,/datasets/xihe/miniconda3/envs/colossal/lib -L/datasets/xihe/miniconda3/envs/colossal/lib -L/datasets/xihe/miniconda3/envs/colossal/lib -Wl,-rpath,/datasets/xihe/miniconda3/envs/colossal/lib -Wl,-rpath-link,/datasets/xihe/miniconda3/envs/colossal/lib -L/datasets/xihe/miniconda3/envs/colossal/lib /home/xihe/.tensornvme/TensorNVMe/build/temp.linux-x86_64-cpython-39/home/xihe/.tensornvme/TensorNVMe/csrc/aio.o /home/xihe/.tensornvme/TensorNVMe/build/temp.linux-x86_64-cpython-39/home/xihe/.tensornvme/TensorNVMe/csrc/offload.o /home/xihe/.tensornvme/TensorNVMe/build/temp.linux-x86_64-cpython-39/home/xihe/.tensornvme/TensorNVMe/csrc/space_mgr.o -L/home/xihe/.tensornvme/lib -L/datasets/xihe/miniconda3/envs/colossal/lib/python3.9/site-packages/torch/lib -laio -lc10 -ltorch -ltorch_cpu -ltorch_python -o build/lib.linux-x86_64-cpython-39/tensornvme/_C.cpython-39-x86_64-linux-gnu.so
  /datasets/xihe/miniconda3/envs/colossal/compiler_compat/ld: cannot find -laio: No such file or directory
  collect2: error: ld returned 1 exit status
  error: command '/cm/extra/apps/GCC/7.3.0/GCC-4.8.5_GMP-6.1.2_MPFR-4.0.1_MPC-1.1.0/bin/g++' failed with exit code 1
  error: subprocess-exited-with-error
  
  × Running setup.py install for tensornvme did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /datasets/xihe/miniconda3/envs/colossal/bin/python -u -c '
  exec(compile('"'"''"'"''"'"'
  # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
  #
  # - It imports setuptools before invoking setup.py, to enable projects that directly
  #   import from `distutils.core` to work with newer packaging standards.
  # - It provides a clear error message when setuptools is not installed.
  # - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
  #   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
  #     manifest_maker: standard file '"'"'-c'"'"' not found".
  # - It generates a shim setup.py, for handling setup.cfg-only projects.
  import os, sys, tokenize
  
  try:
      import setuptools
  except ImportError as error:
      print(
          "ERROR: Can not execute `setup.py` since setuptools is not available in "
          "the build environment.",
          file=sys.stderr,
      )
      sys.exit(1)
  
  __file__ = %r
  sys.argv[0] = __file__
  
  if os.path.exists(__file__):
      filename = __file__
      with tokenize.open(__file__) as f:
          setup_py_code = f.read()
  else:
      filename = "<auto-generated setuptools caller>"
      setup_py_code = "from setuptools import setup; setup()"
  
  exec(compile(setup_py_code, filename, "exec"))
  '"'"''"'"''"'"' % ('"'"'/home/xihe/.tensornvme/TensorNVMe/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' install --record /tmp/pip-record-v1xk0mb3/install-record.txt --single-version-externally-managed --compile --install-headers /datasets/xihe/miniconda3/envs/colossal/include/python3.9/tensornvme
  cwd: /home/xihe/.tensornvme/TensorNVMe/
  Running setup.py install for tensornvme ... error
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> tensornvme

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Angelconan @marsggbo