HWAccelIntro
QuickSync
“Intel Quick Sync Video” is the marketing name for a set of hardware features available inside many Intel GPUs.
Hardware Support
Platform Name | Graphics | Adds support for… |
---|---|---|
Ironlake | gen5 | MPEG-2, H.264 decode. |
Sandy Bridge | gen6 | VC-1 decode; H.264 encode. |
Ivy Bridge | gen7 | JPEG decode; MPEG-2 encode. |
Bay Trail | gen7 | – |
Haswell | gen7.5 | – |
Broadwell | gen8 | VP8 decode. |
Braswell | gen8 | H.265 decode; JPEG, VP8 encode. |
Skylake | gen9 | H.265 encode. |
Apollo Lake | gen9 | VP9, H.265 Main10 decode. |
Kaby Lake | gen9.5 | VP9 profile 2 decode; VP9, H.265 Main10 encode. |
Coffee Lake | gen9.5 | – |
Gemini Lake | gen9.5 | – |
Cannonlake | gen10 | – |
(Each new platform supports a superset of the capabilities of the previous platform.)
API Support
The hardware can be accessed through a number of different APIs:
DXVA2 / D3D11VA
These are standard Windows APIs, which are implemented by the Intel graphics driver to support video decode.
libmfx on Linux
This is a library from Intel which can be installed as part of the Intel Media SDK, and supports a subset of encode and decode cases.
libmfx on Windows
This is a library supplied with Intel’s graphics drivers which supports all encode and decode cases.
Media Foundation
Another Windows API which supports some encode and decode cases via the Intel graphics drivers. Not supported in ffmpeg.
VAAPI with i965 driver
This is a mostly-free (but see below) driver for the libva / VAAPI instructure. Most Linux distributions package it.
VAAPI with iHD driver
The back-end of libmfx on Linux uses a modified libva and VAAPI driver; this can also be used directly by the user.
Linux
A whole open source media stack is provided with much wider HW platforms and Linux distributions supported.
(You can also download the installation binary Media Server Studio from https://software.intel.com/en-us/intel-media-server-studio/.
However, only limited HW platforms and Linux distributions are supported by Media Server Studio.)
Intel open source media stack
Project Name | Supported Gen Graphics | Open Source Repo |
---|---|---|
MSDK | gen8+ | https://github.com/Intel-Media-SDK/MediaSDK |
Libva | gen5+ | https://github.com/intel/libva |
i965 driver | gen5 ~ gen9.5 | https://github.com/intel/intel-vaapi-driver |
iHD driver | gen8+ | https://github.com/intel/media-driver |
VAAPI VS libmfx
VAAPI / i965
- Packaged as standard in most Linux distributions.
- Runs on all usable hardware, including older and cheaper devices.
- Wider codec support.
- Common API for applications which may also use AMD / Nvidia hardware with Mesa.
- Interopable with standard APIs (EGL/OpenGL, OpenCL).
libmfx / iHD
- May give better encode quality in some cases (such as look_ahead).
- May give higher encode throughput in some cases (such as MFE, particularly on Iris graphics).
- Common API for applications which may also run on Windows.
- Interoperable with Intel OpenCL implementation.
Windows
(TODO)
Windows version MSDK can be got from https://software.intel.com/media-sdk
Running
VAAPI
See Hardware/VAAPI.
DXVA2 / D3D11VA
libmfx
The library has a large number of options to set, the possible valid values of are dependent on the version and hardware. libavcodec attempts to map common options sensibly to the libmfx options, but the mapping is crude and has holes, especially around rate control.
Installing the Media SDK on Linux
Install with open source MSDK stack
Install MSDK and iHD driver:
- Follow the guide of https://github.com/Intel-Media-SDK/MediaSDK to build from source code.
- Starting from Ubuntu 19.04, it is possible to install via apt (see: https://github.com/Intel-Media-SDK/MediaSDK/wiki/Intel-media-stack-on-Ubuntu):
sudo apt-get install libva-dev libmfx-dev
Install with MediaServerStudio?
Note that the kernel patches and modified system libraries are all required. It is recommended not to install this on any machine also used for other purposes, as it does not use normal distribution mechanisms and may break / be broken by other packages unexpectedly.
Build machine:
- Build and install the packaged dispatcher: <https://github.com/lu-zero/mfx_dispatch>. (It is also possible to extract the necessary files from the Media SDK installation as described in the install manual – this is not recommended, just use the package instead.)
Target machine:
- Ensure the target machine has a supported CPU. Current versions only support gen8/gen9 graphics on expensive CPUs (“Xeon”/”Core i” branding). The same graphics cores on cheaper CPUs (“Pentium”/”Celeron”/”Atom” branding) are explicitly disabled, presumably for commercial reasons.
- Get a clean version of the supported kernel version (currently 4.4: <https://www.kernel.org/pub/linux/kernel/v4.x/linux-4.4.tar.xz>) and apply the supplied patches. Build and install.
- Build and install the supplied libva and libdrm trees.
- Run the Media SDK install script to install the proprietary components.
- Reboot.
Build FFmpeg with MSDK
./configure --enable-libmfx && make -j8 && make install
Licence status of i965 VAAPI driver
All of the user source code is available, but it includes proprietary blobs of compiled GPU code. Since the complete human-readable source is not available, this certainly renders it GPL-incompatible and is likely to cause issues with other copyleft licences. Using the libva dynamic-loading shim mostly sidesteps this, and therefore is encouraged.
Full Examples
Decode-only
Check supported qsv decoder list
ffmpeg -decoders|grep qsv
H264 video decode and download as yuv420p raw file
ffmpeg -hwaccel qsv -c:v h264_qsv -i input.mp4 -vf hwdownload,format=nv12 -pix_fmt yuv420p output.yuv
H264 video decode and display with sdl
ffmpeg -hwaccel qsv -c:v h264_qsv -i input.mp4 -vf hwdownload,format=nv12 -pix_fmt yuv420p -f sdl -
H264 video decode without output (this can be used as a crude benchmark of the decoder)
ffmpeg -hwaccel qsv -c:v h264_qsv -i input.mp4 -f null -
HEVC 10bit video decode and download as p010le yuv file
ffmpeg -hwaccel qsv -c:v hevc_qsv -load_plugin hevc_hw -i input.mp4 -vf hwdownload,format=p010 -pix_fmt p010le output.yuv
Encode-only
Check supported qsv decoder list
ffmpeg -encoders|grep qsv
Check private option list of h264 encoder
ffmpeg -h encoder=h264_qsv
Encode a 1080p raw yuv input as H264 with 5Mbps using VBR mode
ffmpeg -init_hw_device qsv=hw -filter_hw_device hw -f rawvideo -pix_fmt yuv420p -s:v 1920x1080 -i input.yuv -vf hwupload=extra_hw_frames=64,format=qsv -c:v h264_qsv -b:v 5M output.mp4
Encode a 1080p p010le raw yuv input as HEVC main10 profile (supported since Kaby Lake platform)
ffmpeg -init_hw_device qsv=hw -filter_hw_device hw -v verbose -f rawvideo -video_size 1920x1080 -pix_fmt p010le -i input.yuv -an -vf 'hwupload=extra_hw_frames=64,format=qsv' -c:v hevc_qsv -profile:v main10 output.mp4
Transcode
Software decode + h264 qsv encode with 5Mbps using CBR mode
ffmpeg -init_hw_device qsv=hw -filter_hw_device hw -i input.mp4 -vf hwupload=extra_hw_frames=64,format=qsv -c:v h264_qsv -b:v 5M -maxrate 5M output.mp4
Software decode + h264 qsv encode with CQP mode
ffmpeg -init_hw_device qsv=hw -filter_hw_device hw -i input.mp4 -vf hwupload=extra_hw_frames=64,format=qsv -c:v h264_qsv -q 25 output.mp4
H264 qsv decode + h264 qsv encode with 5Mbps using VBR && Look_ahead mode (look_ahead may not be supported on some platforms)
ffmpeg -hwaccel qsv -c:v h264_qsv -i input.mp4 -c:v h264_qsv -b:v 5M -look_ahead 1 output.mp4
As previous, but use ICQ mode (which is similar to crf mode of x264)
ffmpeg -hwaccel qsv -c:v h264_qsv -i input.mp4 -c:v h264_qsv -global_quality 25 output.mp4
As previous, but use ICQ && Look_ahead mode
ffmpeg -hwaccel qsv -c:v h264_qsv -i input.mp4 -c:v h264_qsv -global_quality 25 -look_ahead 1 output.mp4
Hevc qsv decode + qsv scaling to 1080p@60fps + h264 qsv encode
ffmpeg -hwaccel qsv -c:v hevc_qsv -i input.mp4 -vf 'vpp_qsv=framerate=60,scale_qsv=w=1920:h=1080' -c:v h264_qsv output.mp4
Hevc qsv decode + qsv scaling to 1080p@60fps and color space convert from nv12 to rgb32 + output as sdl
ffmpeg -hwaccel qsv -c:v hevc_qsv -i input.mp4 -vf 'vpp_qsv=framerate=60,scale_qsv=w=1920:h=1080:format=rgb32,hwdownload,format=rgb32' -f sdl -
1:N transcoding: (MFE will be enabled by default if MSDK can support it)
ffmpeg -hwaccel qsv -c:v h264_qsv -i input.mp4 \
-filter_complex "split=2[s1][s2]; [s1]scale_qsv=1280:720[o1];[s2]scale_qsv=960:540[o2]" \
-map [o1] -c:v h264_qsv -b:v 3200k 3200a.mp4 \
-map [o2] -c:v h264_qsv -b:v 1750k 1750a.264
M:N transcoding
ffmpeg -hwaccel qsv -c:v h264_qsv -i input1.mp4 -hwaccel qsv -c:v h264_qsv -i input2.mp4 \
-filter_complex '[0:v]split=2[out1][out2],[1:v]split=2[out3][out4]' \
-map '[out1]' -c:v h264_qsv output1.mp4 -map '[out2]' -c:v h264_qsv output2.mp4 \
-map '[out3]' -c:v h264_qsv output3.mp4 -map '[out4]' -c:v h264_qsv output4.mp4
-qsv_device
is an qsv customized option can be used to specify a hardware device and avoid the default device initialization failure when multiple devices usable (eg: an Intel integrated GPU and an AMD/Nvidia discrete graphics card). One example on Linux (more details please see https://trac.ffmpeg.org/ticket/7649)
ffmpeg -hwaccel qsv -qsv_device /dev/dri/renderD128 -c:v h264_qsv -i input.mp4 -c:v h264_qsv output.mp4
Hybrid transcode
It is also possible to use “vaapi decode + vaapi scaling + qsv encode” (available on Linux platform)
ffmpeg -hwaccel vaapi -hwaccel_output_format vaapi -i input.mp4 -vf 'scale_vaapi=1280:720,hwmap=derive_device=qsv,format=qsv' -c:v h264_qsv output.mp4
Or use “dxva decode + qsv scaling + qsv encode” (available on Windows)
ffmpeg -hwaccel dxva2 -hwaccel_output_format dxva2_vld -i input.mp4 -vf 'hwmap=derive_device=qsv,format=qsv,scale_qsv=w=1280:h=720' -c:v h264_qsv output.mp4