Does PyAV-CUDA work on windows?

[AgainsTurb]

I wonder if PyAV-CUDA work on windows?

[materight]

I haven't tested it but if you have a valid CUDA and ffmpeg installation it should. Have you tried it?

[AgainsTurb]

@materight I shall give it a try till I'm available

[AgainsTurb]

I try my best to make it install on my windows, but it gives me missing dlls error when running python script. I have no clue how to resolve this. I shall try rewrite your lib to directly target windows platform :)

[AgainsTurb]

@materight Have you gotten into such error?
(.venv) PS D:\o4> python test.py
[AVHWDeviceContext @ 0000019d93678600] Using current CUDA context.
Traceback (most recent call last):
File "D:\o4\test.py", line 15, in
frame_tensor = avcuda.to_tensor(avframe, CUDA_DEVICE)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "avcuda\context.pyx", line 79, in avcuda.context.to_tensor
raise ValueError(f"Input frame must be in CUDA format, got {frame.format.name}.")
ValueError: Input frame must be in CUDA format, got yuv420p.

[materight]

Looks like the decoder wasn't initialized properly. How does test.py look?

[AgainsTurb]

Slightly modified your benchmark_decode.py:

import os
os.add_dll_directory("C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v12.4\\bin")
os.add_dll_directory("C:\\Users\\Administrator\\Downloads\\Programs\\ffmpeg-n6.1-latest-win64-gpl-shared-6.1\\bin")

import time
from pathlib import Path

import av
import av.datasets
import avcuda
import cv2
import torch

OUT_DIR = Path(__file__).parent / "out" / "decode"
DEVICE = torch.device("cuda", index=0)
N_RUNS = 10


def main() -> None:
    OUT_DIR.mkdir(exist_ok=True, parents=True)

    # Pre-initialize context to avoid measuring the initialization time
    torch.cuda.init()

    # Test GPU decoding
    print(f"Running GPU decoding {N_RUNS} times...", end=" ", flush=True)
    gpu_start_time = time.perf_counter()

    for i in range(N_RUNS):
        with av.open("C:\\Users\\Administrator\\Downloads\\Video\\test.mp4") as container:
            stream = container.streams.video[0]

            avcuda.init_hwcontext(stream.codec_context, DEVICE.index)

            for frame_idx, frame in enumerate(container.decode(stream)):
                frame_tensor = avcuda.to_tensor(frame, DEVICE.index, format="rgb24")

                if i == 0 and frame_idx == 0:
                    img = cv2.cvtColor(frame_tensor.cpu().numpy(), cv2.COLOR_RGB2BGR)
                    cv2.imwrite(str(OUT_DIR / "gpu.png"), img)
            stream.close()

    gpu_elapsed = time.perf_counter() - gpu_start_time
    print(f"took {gpu_elapsed:.2f}s")

    # Test CPU decoding
    print(f"Running CPU decoding {N_RUNS} times...", end=" ", flush=True)
    cpu_start_time = time.perf_counter()

    for i in range(N_RUNS):
        with av.open("C:\\Users\\Administrator\\Downloads\\Video\\test.mp4") as container:
            stream = container.streams.video[0]

            for frame_idx, frame in enumerate(container.decode(stream)):
                frame_ndarray = frame.to_ndarray(format="rgb24")
                frame_tensor = torch.from_numpy(frame_ndarray).to(DEVICE)

                if i == 0 and frame_idx == 0:
                    img = cv2.cvtColor(frame_ndarray, cv2.COLOR_RGB2BGR)
                    cv2.imwrite(str(OUT_DIR / "cpu.png"), img)
            stream.close()

    cpu_elapsed = time.perf_counter() - cpu_start_time
    print(f"took {cpu_elapsed:.2f}s")

    # Test difference in decoded images
    cpu_frame = cv2.imread(str(OUT_DIR / "cpu.png"))
    gpu_frame = cv2.imread(str(OUT_DIR / "gpu.png"))
    diff = cv2.absdiff(cpu_frame, gpu_frame)
    print(f"Max diff in px values between CPU and GPU decoded frames: {diff.max()}")


if __name__ == "__main__":
    main()

[AgainsTurb]

@materight It seems I finally make it work on my windows pc but with some warnings:) Your code is excellent and a great help to me. Should I post my method to make it work on windows?

[materight]

Nice! Sure might be helpful to others

[omaiyiwa]

@materight It seems I finally make it work on my windows pc but with some warnings:) Your code is excellent and a great help to me. Should I post my method to make it work on windows?看来我终于可以在我的 Windows PC 上运行了，但还是有一些警告:) 你的代码很棒，对我帮助很大。我应该把我的方法发布出来让它在 Windows 上运行吗？

hello, when I run your setup-win.py it comes up with

print(CUDA_HOME, CUDA_ARCH_LIST)
print(include_dirs, library_dirs, libraries)

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8 75,86
['D:\\peng\\python_project\\K21\\water\\glkj_project\\azhu\\baorui_dian\\test\\ffmpeg-7.1-full_build-shared\\include', 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.8\\include'] ['D:\\peng\\python_project\\K21\\water\\glkj_project\\azhu\\baorui_dian\\test\\ffmpeg-7.1-full_build-shared\\lib', 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.8\\lib\\x64'] ['avcodec', 'avutil', 'nppicc', 'cudart', 'nppif', 'nppc', 'nppig', 'nppim', 'nppist', 'nppisu']
Compiling avcuda\context.pyx because it changed.
[1/1] Cythonizing avcuda\context.pyx

Error compiling Cython file:
------------------------------------------------------------
...
cimport libav
        ^
------------------------------------------------------------

avcuda\context.pyx:1:8: 'libav.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
cimport libav as lib


cdef class Codec:

    cdef const lib.AVCodec *ptr
               ^
------------------------------------------------------------

D:\peng\Anaconda\envs\yolo\lib\site-packages\av\codec\codec.pxd:6:15: 'AVCodec' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...

---

Is my av not installed right?

[AgainsTurb]

@omaiyiwa Sorry I've not checked my mail recently. It seems your ffmpeg is not compiled with proper support. You can try this one: ffmpeg-n6.1-latest-win64-gpl-shared-6.1.

[omaiyiwa]

@omaiyiwa Sorry I've not checked my mail recently. It seems your ffmpeg is not compiled with proper support. You can try this one: ffmpeg-n6.1-latest-win64-gpl-shared-6.1.抱歉，我最近没查看邮件。看来你的 ffmpeg 编译时没有提供正确的支持。你可以试试这个：ffmpeg-n6.1-latest-win64-gpl-shared-6.1。

• I tried ffmpeg-n6.1-latest-win64-gpl-shared-6.1,but the pyav14.* version seems to support only ffmpeg7.version,even though I tried to install av as 12.3.0,it seems that the compiled av is also missing something
• I got it to work by ffmpeg-n7.1-latest- win64-gpl-shared-7.1 with pyav==14.4.0 using pip install av --no-binary av --no-cache, success
• But when I run setup-win.py I found that pyav14.* doesn't seem to have libav.pxd, I copied libav.pxd from somewhere else and encountered the following problem, have you ever encountered it, please?

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8 75,86
['D:\\ffmpeg-n7.1-latest-win64-gpl-shared-7.1\\include', 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.8\\include'] ['D:\\ffmpeg-n7.1-latest-win64-gpl-shared-7.1\\lib', 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.8\\lib\\x64'] ['avcodec', 'avutil', 'nppicc', 'cudart', 'nppif', 'nppc', 'nppig', 'nppim', 'nppist', 'nppisu']
running build_ext
"C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -Iavcuda -ID:\peng\Anaconda\envs\yolo\lib\site-packages\av\..\include -ID:\ffmpeg-n7.1-latest-win64-gpl-shared-7.1\include "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\include" -ID:\peng\Anaconda\envs\yolo\include -ID:\peng\Anaconda\envs\yolo\Include "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\ATLMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" -ID:\ffmpeg-n7.1-latest-win64-gpl-shared-7.1\include /Tcbuild\avcuda\npp.c /Fobuild\temp.win-amd64-3.9\Release\build\avcuda\npp.obj msvc nvcc
cl: 命令行 warning D9024 :无法识别的源文件类型“msvc”，假定为对象文件
cl: 命令行 warning D9027 :源文件“msvc”被忽略
cl: 命令行 warning D9024 :无法识别的源文件类型“nvcc”，假定为对象文件
cl: 命令行 warning D9027 :源文件“nvcc”被忽略
npp.c
D:\ffmpeg-n7.1-latest-win64-gpl-shared-7.1\include\libavutil\rational.h(1): warning C4819: 该文件包含不能在当前代码页(936)中表示的字符。请将该文件保存为 Unicode 格式以防止数据丢失
build\avcuda\npp.c(2385): warning C4013: “nppiNV12ToRGB_8u_ColorTwist32f_P2C3R_Ctx”未定义；假设外部返回 int
build\avcuda\npp.c(3169): warning C4305: “=”: 从“double”到“Npp32f”截断
build\avcuda\npp.c(3171): warning C4305: “=”: 从“double”到“Npp32f”截断
build\avcuda\npp.c(3172): warning C4305: “=”: 从“double”到“Npp32f”截断
build\avcuda\npp.c(3181): warning C4305: “=”: 从“double”到“Npp32f”截断
build\avcuda\npp.c(3182): warning C4305: “=”: 从“double”到“Npp32f”截断
build\avcuda\npp.c(3183): warning C4305: “=”: 从“double”到“Npp32f”截断
build\avcuda\npp.c(3184): warning C4305: “=”: 从“double”到“Npp32f”截断
build\avcuda\npp.c(3193): warning C4305: “=”: 从“double”到“Npp32f”截断
build\avcuda\npp.c(3194): warning C4305: “=”: 从“double”到“Npp32f”截断
build\avcuda\npp.c(3196): warning C4305: “=”: 从“double”到“Npp32f”截断
build\avcuda\npp.c(3219): warning C4305: “=”: 从“double”到“Npp32f”截断
build\avcuda\npp.c(3220): warning C4305: “=”: 从“double”到“Npp32f”截断
build\avcuda\npp.c(3230): warning C4305: “=”: 从“double”到“Npp32f”截断
build\avcuda\npp.c(3231): warning C4305: “=”: 从“double”到“Npp32f”截断
build\avcuda\npp.c(3232): warning C4305: “=”: 从“double”到“Npp32f”截断
build\avcuda\npp.c(3242): warning C4305: “=”: 从“double”到“Npp32f”截断
build\avcuda\npp.c(3244): warning C4305: “=”: 从“double”到“Npp32f”截断
"C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\bin\HostX86\x64\link.exe" /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:D:\ffmpeg-n7.1-latest-win64-gpl-shared-7.1\lib "/LIBPATH:C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\lib\x64" /LIBPATH:D:\peng\Anaconda\envs\yolo\libs /LIBPATH:D:\peng\Anaconda\envs\yolo /LIBPATH:D:\peng\Anaconda\envs\yolo\PCbuild\amd64 "/LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\ATLMFC\lib\x64" "/LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64" /LIBPATH:D:\ffmpeg-n7.1-latest-win64-gpl-shared-7.1\lib avcodec.lib avutil.lib nppicc.lib cudart.lib nppif.lib nppc.lib nppig.lib nppim.lib nppist.lib nppisu.lib /EXPORT:PyInit_npp build\temp.win-amd64-3.9\Release\build\avcuda\npp.obj /OUT:build\lib.win-amd64-3.9\avcuda\npp.cp39-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.9\Release\build\avcuda\npp.cp39-win_amd64.lib
  正在创建库 build\temp.win-amd64-3.9\Release\build\avcuda\npp.cp39-win_amd64.lib 和对象 build\temp.win-amd64-3.9\Release\build\avcuda\npp.cp39-win_amd64.exp
npp.obj : error LNK2001: 无法解析的外部符号 nppiNV12ToRGB_8u_ColorTwist32f_P2C3R_Ctx
build\lib.win-amd64-3.9\avcuda\npp.cp39-win_amd64.pyd : fatal error LNK1120: 1 个无法解析的外部命令
error: command 'C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.41.34120\\bin\\HostX86\\x64\\link.exe' failed with exit code 1120

[AgainsTurb]

@omaiyiwa Maybe I suggest you try building the PyAV from source. You may clone the official PyAV repo and navigate to the directory, then open powershell to run this command:
$env:INCLUDE="D:\ffmpeg-n7.1-latest-win64-gpl-shared-7.1\include"; $env:LIB="D:\ffmpeg-n7.1-latest-win64-gpl-shared-7.1\lib"; $env:PATH="D:\ffmpeg-n7.1-latest-win64-gpl-shared-7.1\bin;$env:PATH"; python setup.py bdist_wheel --ffmpeg-dir="D:\ffmpeg-n7.1-latest-win64-gpl-shared-7.1"
If it builds successfully, you may find the PyAV whl file inside dist folder which you can directly pip install. Then you may be able to build avcuda from source (I personally suggest you build avcude to a folder where you can directly import instead of building into a whl file). I'm not sure if it works for you, but let me know If you make any progress

[omaiyiwa]

This is the latest version I downloaded from the official PyAv repository

(yolo) PS C:\Users\Administrator> $env:INCLUDE="D:\ffmpeg-n7.1-latest-win64-gpl-shared-7.1\include"; $env:LIB="D:\ffmpeg-n7.1-latest-win64-gpl-shared-7.1\lib"; $env:PATH="D:\ffmpeg-n7.1-latest-win64-gpl-shared-7.1\bin;$env:PATH";
(yolo) PS C:\Users\Administrator> cd D:\PyAV-main\
(yolo) PS D:\PyAV-main> python setup.py bdist_wheel --ffmpeg-dir="D:\ffmpeg-n7.1-latest-win64-gpl-shared-7.1"

�[1;91mWarning!�[0m You are installing from source.
It is �[1;37mEXPECTED�[0m that it will fail. You are �[1;37mREQUIRED�[0m to use ffmpeg 7.
You �[1;37mMUST�[0m have Cython, pkg-config, and a C compiler.

�[1;91mWarning!�[0m You are not using a virtual environment
Compiling av/filter/loudnorm.py because it changed.
[1/1] Cythonizing av/filter/loudnorm.py

Error compiling Cython file:
------------------------------------------------------------
...
    format_ptr: cython.pointer[AVFormatContext] = container.ptr
    container.ptr = cython.NULL  # Prevent double-free

    stream_index: cython.int = stream.index
    py_args: bytes = loudnorm_args.encode("utf-8")
    c_args: cython.p_const_char = py_args
                  ^
------------------------------------------------------------

av\filter\loudnorm.py:31:18: Unknown type declaration 'cython.p_const_char' in annotation

Error compiling Cython file:
------------------------------------------------------------
...

    # Save log level since C function overwrite it.
    level = get_level()

    with cython.nogil:
        result = loudnorm_get_stats(format_ptr, stream_index, c_args)
                                                              ^
------------------------------------------------------------

av\filter\loudnorm.py:38:62: Coercion from Python not allowed without the GIL
Traceback (most recent call last):
  File "D:\PyAV-main\setup.py", line 202, in <module>
    ext_modules = cythonize(
  File "D:\peng\Anaconda\envs\yolo\lib\site-packages\Cython\Build\Dependencies.py", line 1154, in cythonize
    cythonize_one(*args)
  File "D:\peng\Anaconda\envs\yolo\lib\site-packages\Cython\Build\Dependencies.py", line 1321, in cythonize_one
    raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: av/filter/loudnorm.py

@omaiyiwa Maybe I suggest you try building the PyAV from source. You may clone the official PyAV repo and navigate to the directory, then open powershell to run this command:我建议您尝试从源代码构建 PyAV。您可以克隆官方 PyAV 仓库并导航到该目录，然后打开 powershell 运行以下命令： $env:INCLUDE="D:\ffmpeg-n7.1-latest-win64-gpl-shared-7.1\include"; $env:LIB="D:\ffmpeg-n7.1-latest-win64-gpl-shared-7.1\lib"; $env:PATH="D:\ffmpeg-n7.1-latest-win64-gpl-shared-7.1\bin;$env:PATH"; python setup.py bdist_wheel --ffmpeg-dir="D:\ffmpeg-n7.1-latest-win64-gpl-shared-7.1"$env:INCLUDE="D:\ffmpeg-n7.1-latest-win64-gpl-shared-7.1\include"; $env:LIB="D:\ffmpeg-n7.1-latest-win64-gpl-shared-7.1\lib"; $env：PATH="D:\ffmpeg-n7.1-latest-win64-gpl-shared-7.1\bin;$env：PATH"; python setup.py bdist_wheel --ffmpeg-dir="D:\ffmpeg-n7.1-latest-win64-gpl-shared-7.1" If it builds successfully, you may find the PyAV whl file inside dist folder which you can directly pip install. Then you may be able to build avcuda from source (I personally suggest you build avcude to a folder where you can directly import instead of building into a whl file). I'm not sure if it works for you, but let me know If you make any progress如果构建成功，您可能会在 dist 文件夹中找到 PyAV whl 文件，可以直接使用 pip 安装。然后，您就可以从源代码构建 avcuda 了（我个人建议您将 avcude 构建到可以直接导入的文件夹中，而不是构建到 whl 文件中）。我不确定这是否适合您，但如果您有任何进展，请告诉我。

[AgainsTurb]

@omaiyiwa I am sorry to hear that there is still error persisting. And from your reply I may suggest you're from china and I'm too. If you want to resolve this problem (or maybe you have found some equivalents to this project), we can exchange contact and discuss it in private if you don't mind :). You can email me here againsturb123@gmail.com