Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(server): better transcoding logs #13000

Merged
merged 2 commits into from
Sep 27, 2024
Merged

feat(server): better transcoding logs #13000

merged 2 commits into from
Sep 27, 2024

Conversation

mertalev
Copy link
Contributor

@mertalev mertalev commented Sep 27, 2024 •
edited
Loading

Description

  • Adds progress logs with 5% intervals and estimated remaining time
    • Many videos don't have frame count metadata available, which would limit the usefulness of this feature. To remedy this, I added a setting to make FFprobe count frames when using debug logs. It's otherwise disabled since it adds some overhead to probe
  • Encoding start log explicitly mentions status of hardware acceleration
  • Logs the raw FFmpeg command instead of a JSON object, making it much easier to copy/paste
  • Clarifies OpenCL debug log that only affects RKMPP
  • Adds verbose log when skipping an asset because it doesn't need to be transcoded
  • Fixes handling of N/A that resulted in frameCount being NaN

How Has This Been Tested?

CPU:

immich_server            | [Nest] 7  - 09/27/2024, 7:59:22 PM     LOG [Microservices:MediaService] Encoding video c3acdaf0-3a86-4e54-950f-a33a8241dcaf without hardware acceleration
immich_server            | [Nest] 7  - 09/27/2024, 7:59:22 PM   DEBUG [Microservices:MediaRepository] ffmpeg -n 10 /usr/bin/ffmpeg -i upload/library/admin/2024/2024-03-01/the-black.mkv -y -c:v av1 -c:a aac -movflags faststart -fps_mode passthrough -map 0:0 -strict unofficial -map 0:1 -v verbose -vf scale=-2:720,zscale=t=linear:npl=100,tonemap=hable:desat=0,zscale=p=bt709:t=bt709:m=bt709:range=pc,format=yuv420p -preset 4 -crf 35 upload/encoded-video/ee17565c-e17b-4c82-9607-07902d53c9cd/c3/ac/c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 7:59:29 PM   DEBUG [Microservices:MediaRepository] Transcoding 5.05% done, estimated 2m, 18s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 7:59:37 PM   DEBUG [Microservices:MediaRepository] Transcoding 10.15% done, estimated 2m, 13s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 7:59:44 PM   DEBUG [Microservices:MediaRepository] Transcoding 15.23% done, estimated 2m, 6s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 7:59:49 PM   DEBUG [Microservices:MediaRepository] Transcoding 20.38% done, estimated 1m, 53s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 7:59:57 PM   DEBUG [Microservices:MediaRepository] Transcoding 25.52% done, estimated 1m, 48s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:00:12 PM   DEBUG [Microservices:MediaRepository] Transcoding 30.55% done, estimated 1m, 59s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:00:26 PM   DEBUG [Microservices:MediaRepository] Transcoding 35.79% done, estimated 2m, 1s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:00:35 PM   DEBUG [Microservices:MediaRepository] Transcoding 40.83% done, estimated 1m, 52s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:00:45 PM   DEBUG [Microservices:MediaRepository] Transcoding 46.18% done, estimated 1m, 44s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:00:51 PM   DEBUG [Microservices:MediaRepository] Transcoding 51.54% done, estimated 1m, 29s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:01:01 PM   DEBUG [Microservices:MediaRepository] Transcoding 56.70% done, estimated 1m, 20s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:01:08 PM   DEBUG [Microservices:MediaRepository] Transcoding 61.73% done, estimated 1m, 9s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:01:15 PM   DEBUG [Microservices:MediaRepository] Transcoding 66.99% done, estimated 58s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:01:20 PM   DEBUG [Microservices:MediaRepository] Transcoding 72.37% done, estimated 47s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:01:26 PM   DEBUG [Microservices:MediaRepository] Transcoding 77.68% done, estimated 37s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:01:34 PM   DEBUG [Microservices:MediaRepository] Transcoding 82.78% done, estimated 29s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:01:42 PM   DEBUG [Microservices:MediaRepository] Transcoding 87.80% done, estimated 20s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:01:55 PM   DEBUG [Microservices:MediaRepository] Transcoding 92.88% done, estimated 12s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:02:07 PM   DEBUG [Microservices:MediaRepository] Transcoding 100.00% done for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:02:08 PM     LOG [Microservices:MediaService] Successfully encoded c3acdaf0-3a86-4e54-950f-a33a8241dcaf

NVENC:

immich_server            | [Nest] 7  - 09/27/2024, 8:04:41 PM     LOG [Microservices:MediaService] Encoding video c3acdaf0-3a86-4e54-950f-a33a8241dcaf with NVENC acceleration
immich_server            | [Nest] 7  - 09/27/2024, 8:04:41 PM   DEBUG [Microservices:MediaRepository] ffmpeg -n 10 /usr/bin/ffmpeg -hwaccel cuda -hwaccel_output_format cuda -noautorotate -threads 1 -i upload/library/admin/2024/2024-03-01/the-black.mkv -y -tune hq -qmin 0 -rc-lookahead 20 -i_qfactor 0.75 -c:v av1_nvenc -c:a aac -movflags faststart -fps_mode passthrough -map 0:0 -strict unofficial -map 0:1 -bf 3 -g 256 -b_ref_mode middle -b_qfactor 1.1 -temporal-aq 1 -v verbose -vf scale_cuda=-2:720,tonemap_cuda=desat=0:matrix=bt709:primaries=bt709:range=pc:tonemap=hable:transfer=bt709:format=nv12 -preset p7 -cq:v 35 upload/encoded-video/ee17565c-e17b-4c82-9607-07902d53c9cd/c3/ac/c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:04:48 PM   DEBUG [Microservices:MediaRepository] Transcoding 5.74% done, estimated 1m, 48s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:04:50 PM   DEBUG [Microservices:MediaRepository] Transcoding 11.14% done, estimated 1m, 8s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:04:52 PM   DEBUG [Microservices:MediaRepository] Transcoding 16.52% done, estimated 53s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:04:54 PM   DEBUG [Microservices:MediaRepository] Transcoding 22.41% done, estimated 43s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:04:56 PM   DEBUG [Microservices:MediaRepository] Transcoding 27.46% done, estimated 38s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:04:58 PM   DEBUG [Microservices:MediaRepository] Transcoding 32.54% done, estimated 34s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:04:58 PM   DEBUG [Microservices:MediaRepository] Transcoding 37.56% done, estimated 30s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:05:00 PM   DEBUG [Microservices:MediaRepository] Transcoding 42.62% done, estimated 27s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:05:02 PM   DEBUG [Microservices:MediaRepository] Transcoding 47.73% done, estimated 24s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:05:04 PM   DEBUG [Microservices:MediaRepository] Transcoding 53.24% done, estimated 21s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:05:06 PM   DEBUG [Microservices:MediaRepository] Transcoding 58.37% done, estimated 19s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:05:08 PM   DEBUG [Microservices:MediaRepository] Transcoding 63.65% done, estimated 16s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:05:10 PM   DEBUG [Microservices:MediaRepository] Transcoding 69.37% done, estimated 13s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:05:12 PM   DEBUG [Microservices:MediaRepository] Transcoding 75.54% done, estimated 10s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:05:14 PM   DEBUG [Microservices:MediaRepository] Transcoding 80.84% done, estimated 8s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:05:16 PM   DEBUG [Microservices:MediaRepository] Transcoding 85.85% done, estimated 6s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:05:19 PM   DEBUG [Microservices:MediaRepository] Transcoding 92.04% done, estimated 3s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:05:21 PM   DEBUG [Microservices:MediaRepository] Transcoding 97.09% done, estimated 1s remaining for output c3acdaf0-3a86-4e54-950f-a33a8241dcaf.mp4
immich_server            | [Nest] 7  - 09/27/2024, 8:05:22 PM     LOG [Microservices:MediaService] Successfully encoded c3acdaf0-3a86-4e54-950f-a33a8241dcaf

jrasm91
jrasm91 approved these changes Sep 27, 2024
View reviewed changes
server/src/repositories/media.repository.ts Outdated Show resolved Hide resolved
server/src/repositories/media.repository.ts Outdated Show resolved Hide resolved
@mertalev mertalev merged commit 4248594 into main Sep 27, 2024
35 checks passed
@mertalev mertalev deleted the feat/server-better-transcoding-logs branch September 27, 2024 22:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jrasm91 jrasm91 jrasm91 approved these changes

@danieldietzler danieldietzler Awaiting requested review from danieldietzler danieldietzler is a code owner

Assignees
No one assigned
Labels
changelog:enhancement 🗄️server
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@mertalev @jrasm91