Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preliminary VAAPI support #170

Closed
wants to merge 3 commits into from
Closed

Preliminary VAAPI support #170

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8d16e6f
Preliminary VAAPI support
AbAb1l Jan 24, 2020
26479eb
Changes as per review comments in PR#170
AbAb1l Feb 4, 2020
963ec07
ffmpeg: `av_hwdevice_find_type_by_name("vaapi")` would always return …
AbAb1l Feb 4, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,33 @@ To run the tests on a particular GPU, use the GPU_DEVICE environment variable:
GPU_DEVICE=3 go test -tag nvidia -run Nvidia
```

Hardware transcoding on Intel/AMD GPU is supported.

In order to enable VAAPI support for Intel GPU following packages needs to be installed

```
sudo apt install libva-dev intel-gpu-tools
```

In order to enable VAAPI support for AMD GPU following packages needs to be installed

```
sudo apt install libva-dev mesa-va-drivers radeontop
```

To execute the vaapi tests within the `ffmpeg` directory, run this command:

```
go test -tags=vaapi -run Vaapi
```

To run the tests on a particular GPU, use the GPU_DEVICE environment variable:

```
# Runs on GPU number /dev/dri/renderD128
GPU_DEVICE=/dev/dri/renderD128 go test -tag vaapi -run Vaapi
```

Aside from the tests themselves, there is a
[sample program](https://github.com/livepeer/lpms/blob/master/cmd/transcoding/transcoding.go)
that can be used as a reference to the LPMS GPU transcoding API. The sample
Expand All @@ -164,6 +191,10 @@ go run cmd/transcoding/transcoding.go transcoder/test.ts P144p30fps16x9,P240p30f

# nvidia processing, GPU number 2
go run cmd/transcoding/transcoding.go transcoder/test.ts P144p30fps16x9,P240p30fps16x9 nv 2

# vaapi processing, GPU /dev/dri/renderD128
go run cmd/transcoding/transcoding.go transcoder/test.ts P144p30fps16x9,P240p30fps16x9 va /dev/dri/renderD128

```

You can follow the development of LPMS and Livepeer @ our [forum](http://forum.livepeer.org)
7 changes: 5 additions & 2 deletions cmd/transcoding/transcoding.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,11 @@ func main() {
str2accel := func(inp string) (ffmpeg.Acceleration, string) {
if inp == "nv" {
return ffmpeg.Nvidia, "nv"
} else if inp == "va" {
return ffmpeg.VAAPI, "va"
} else {
return ffmpeg.Software, "sw"
AbAb1l marked this conversation as resolved.
Show resolved Hide resolved
}
return ffmpeg.Software, "sw"
}
str2profs := func(inp string) []ffmpeg.VideoProfile {
profs := []ffmpeg.VideoProfile{}
Expand Down Expand Up @@ -57,7 +60,7 @@ func main() {
options := profs2opts(profiles)

var dev string
if accel == ffmpeg.Nvidia {
if accel == ffmpeg.Nvidia || accel == ffmpeg.VAAPI {
if len(os.Args) <= 4 {
panic("Expected device number")
AbAb1l marked this conversation as resolved.
Show resolved Hide resolved
}
Expand Down
13 changes: 12 additions & 1 deletion ffmpeg/ffmpeg.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ type Acceleration int
const (
Software Acceleration = iota
Nvidia
Amd
VAAPI
)

type ComponentOptions struct {
Expand Down Expand Up @@ -143,6 +143,15 @@ func configAccel(inAcc, outAcc Acceleration, inDev, outDev string) (string, stri
}
return "h264_nvenc", "scale_cuda", nil
}
case VAAPI:
switch outAcc {
case VAAPI:
// If we encode on a different device from decode then need to transfer
if outDev != "" && outDev != inDev {
return "", "", ErrTranscoderInp // XXX not allowed
}
return "h264_vaapi", "scale_vaapi", nil
}
}
return "", "", ErrTranscoderHw
}
Expand All @@ -152,6 +161,8 @@ func accelDeviceType(accel Acceleration) (C.enum_AVHWDeviceType, error) {
return C.AV_HWDEVICE_TYPE_NONE, nil
case Nvidia:
return C.AV_HWDEVICE_TYPE_CUDA, nil
case VAAPI:
return C.AV_HWDEVICE_TYPE_VAAPI, nil

}
return C.AV_HWDEVICE_TYPE_NONE, ErrTranscoderHw
Expand Down
95 changes: 90 additions & 5 deletions ffmpeg/lpms_ffmpeg.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ struct input_ctx {
// Hardware decoding support
AVBufferRef *hw_device_ctx;
enum AVHWDeviceType hw_type;
enum AVPixelFormat hw_pix_fmt;
char *device;

int64_t next_pts_a, next_pts_v;
Expand Down Expand Up @@ -267,6 +268,36 @@ static enum AVPixelFormat hw2pixfmt(AVCodecContext *ctx)
return AV_PIX_FMT_NONE;
}

static int set_vaapi_hwframe_ctx(AVCodecContext *ctx, AVBufferRef *hw_device_ctx)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we reuse get_hw_pixfmt here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No we can't. In case of VAAPI, AVHWFramesContext initialization doesn't occur in time for the filters as compared to CUDA. Hence it needs initialization at earliest as possible.

{
AVBufferRef *hw_frames_ref;
AVHWFramesContext *frames_ctx = NULL;
int err = 0;

if (!(hw_frames_ref = av_hwframe_ctx_alloc(hw_device_ctx))) {
fprintf(stderr, "Failed to create VAAPI frame context.\n");
return -1;
}
frames_ctx = (AVHWFramesContext *)(hw_frames_ref->data);
frames_ctx->format = AV_PIX_FMT_VAAPI;
frames_ctx->sw_format = AV_PIX_FMT_NV12;
frames_ctx->width = ctx->width;
frames_ctx->height = ctx->height;
frames_ctx->initial_pool_size = 20;
if ((err = av_hwframe_ctx_init(hw_frames_ref)) < 0) {
fprintf(stderr, "Failed to initialize VAAPI frame context."
"Error code: %s\n",av_err2str(err));
av_buffer_unref(&hw_frames_ref);
return err;
}
ctx->hw_frames_ctx = av_buffer_ref(hw_frames_ref);
if (!ctx->hw_frames_ctx)
err = AVERROR(ENOMEM);

av_buffer_unref(&hw_frames_ref);
return err;
}

static enum AVPixelFormat get_hw_pixfmt(AVCodecContext *vc, const enum AVPixelFormat *pix_fmts)
{
AVHWFramesContext *frames;
Expand Down Expand Up @@ -312,6 +343,19 @@ fprintf(stderr,"possible format: %s\n", av_get_pix_fmt_name(*p));
return frames->format;
}

static enum AVPixelFormat get_vaapi_format(AVCodecContext *ctx, const enum AVPixelFormat *pix_fmts)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should be able to reuse hw2pixfmt instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No we cannot, signature difference.
required
enum AVPixelFormat (*get_format)(struct AVCodecContext *s, const enum AVPixelFormat * fmt);

where as hw2pixfmt has
static enum AVPixelFormat hw2pixfmt(AVCodecContext *ctx)

{
const enum AVPixelFormat *p;

for (p = pix_fmts; *p != AV_PIX_FMT_NONE; p++) {
if (*p == AV_PIX_FMT_VAAPI)
return *p;
}

fprintf(stderr, "Unable to decode this file using VA-API.\n");
return AV_PIX_FMT_NONE;
}

static int init_video_filters(struct input_ctx *ictx, struct output_ctx *octx)
{
#define filters_err(msg) { \
Expand All @@ -326,7 +370,7 @@ static int init_video_filters(struct input_ctx *ictx, struct output_ctx *octx)
AVFilterInOut *outputs = avfilter_inout_alloc();
AVFilterInOut *inputs = avfilter_inout_alloc();
AVRational time_base = ictx->ic->streams[ictx->vi]->time_base;
enum AVPixelFormat pix_fmts[] = { AV_PIX_FMT_YUV420P, AV_PIX_FMT_CUDA, AV_PIX_FMT_NONE }; // XXX ensure the encoder allows this
enum AVPixelFormat pix_fmts[] = { AV_PIX_FMT_YUV420P, AV_PIX_FMT_CUDA, AV_PIX_FMT_VAAPI, AV_PIX_FMT_NONE }; // XXX ensure the encoder allows this
struct filter_ctx *vf = &octx->vf;
char *filters_descr = octx->vfilters;
enum AVPixelFormat in_pix_fmt = ictx->vc->pix_fmt;
Expand Down Expand Up @@ -733,9 +777,9 @@ static void free_input(struct input_ctx *inctx)

static int open_video_decoder(input_params *params, struct input_ctx *ctx)
{
#define dd_err(msg) { \
#define dd_err(msg,...) { \
if (!ret) ret = -1; \
fprintf(stderr, msg); \
fprintf(stderr, msg, ##__VA_ARGS__); \
goto open_decoder_err; \
}
int ret = 0;
Expand All @@ -753,6 +797,37 @@ static int open_video_decoder(input_params *params, struct input_ctx *ctx)
AVCodec *c = avcodec_find_decoder_by_name("h264_cuvid");
if (c) codec = c;
else fprintf(stderr, "Cuvid decoder not found; defaulting to software\n");
} else if (AV_CODEC_ID_H264 == codec->id &&
AV_HWDEVICE_TYPE_VAAPI == params->hw_type) {
enum AVHWDeviceType type;
type = av_hwdevice_find_type_by_name("vaapi");
if (type == AV_HWDEVICE_TYPE_NONE) {
fprintf(stderr, "Device type vaapi is not supported.\n");
fprintf(stderr, "Available device types:");
while((type = av_hwdevice_iterate_types(type)) != AV_HWDEVICE_TYPE_NONE)
AbAb1l marked this conversation as resolved.
Show resolved Hide resolved
fprintf(stderr, " %s", av_hwdevice_get_type_name(type));
fprintf(stderr, "\n");
ret = -1;
goto open_decoder_err;
}

AVCodec *c = NULL;
int i;
ret = av_find_best_stream(ic, AVMEDIA_TYPE_VIDEO, -1, -1, &c, 0);
AbAb1l marked this conversation as resolved.
Show resolved Hide resolved
if (c) {
codec = c;
for (i = 0;; i++) {
const AVCodecHWConfig *config = avcodec_get_hw_config(codec, i);
AbAb1l marked this conversation as resolved.
Show resolved Hide resolved
if (!config) {
dd_err("Decoder %s does not support device type %s.\n", codec->name, av_hwdevice_get_type_name(type));
}
if (config->methods & AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX &&
config->device_type == type) {
ctx->hw_pix_fmt = config->pix_fmt;
break;
}
}
} else fprintf(stderr, "Vaapi decoder not found; defaulting to software\n");
AbAb1l marked this conversation as resolved.
Show resolved Hide resolved
}
AVCodecContext *vc = avcodec_alloc_context3(codec);
if (!vc) dd_err("Unable to alloc video codec\n");
Expand All @@ -767,7 +842,17 @@ static int open_video_decoder(input_params *params, struct input_ctx *ctx)
if (ret < 0) dd_err("Unable to open hardware context for decoding\n")
ctx->hw_type = params->hw_type;
vc->hw_device_ctx = av_buffer_ref(ctx->hw_device_ctx);
vc->get_format = get_hw_pixfmt;
if (params->hw_type == AV_HWDEVICE_TYPE_VAAPI) {
vc->pix_fmt = AV_PIX_FMT_VAAPI;
int err;
/* set hw_frames_ctx for encoder's AVCodecContext */
if ((err = set_vaapi_hwframe_ctx(vc, vc->hw_device_ctx)) < 0) {
dd_err("Failed to set hwframe context.\n");
}
vc->get_format = get_vaapi_format;
} else {
vc->get_format = get_hw_pixfmt;
}
}
vc->pkt_timebase = ic->streams[ctx->vi]->time_base;
ret = avcodec_open2(vc, codec, NULL);
Expand Down Expand Up @@ -1110,7 +1195,7 @@ int transcode(struct transcode_thread *h,
ret = avio_open(&ictx->ic->pb, inp->fname, AVIO_FLAG_READ);
if (ret < 0) main_err("Unable to reopen file");
// XXX check to see if we can also reuse decoder for sw decoding
if (AV_HWDEVICE_TYPE_CUDA != ictx->hw_type) {
if (AV_HWDEVICE_TYPE_CUDA != ictx->hw_type && AV_HWDEVICE_TYPE_VAAPI != ictx->hw_type) {
ret = open_video_decoder(inp, ictx);
if (ret < 0) main_err("Unable to reopen video decoder");
}
Expand Down
Loading