Support multiple FFmpeg versions (#3464)

Summary: Pull Request resolved: #3464

Differential Revision: D47300223

Pulled By: mthrok

fbshipit-source-id: 0fef04461478402a457a537ce9bd6f68c54b02ab

CMakeLists.txt

@@ -171,7 +171,12 @@ if (BUILD_SOX)
   add_subdirectory(torchaudio/csrc/sox)
 endif()
 if (USE_FFMPEG)
-  add_subdirectory(third_party/ffmpeg)
+  if (DEFINED ENV{FFMPEG_ROOT})
+    add_subdirectory(third_party/ffmpeg/single)
+  else()
+    message(STATUS "Building FFmpeg integration with multi version support")
+    add_subdirectory(third_party/ffmpeg/multi)
+  endif()
   add_subdirectory(torchaudio/csrc/ffmpeg)
 endif()
 if (BUILD_CUDA_CTC_DECODER)

docs/source/build.jetson.rst

@@ -137,7 +137,7 @@ Verify the installation by checking the version and CUDA device accessibility.
 
    git clone https://github.com/pytorch/audio
    cd audio
-   USE_CUDA=1 USE_FFMPEG=1 pip install -v -e . --no-use-pep517
+   USE_CUDA=1 pip install -v -e . --no-use-pep517
 
 4. Check the installation
 ~~~~~~~~~~~~~~~~~~~~~~~~~

docs/source/build.linux.rst

@@ -24,30 +24,37 @@ Here, we install nightly build.
 
    conda install cmake ninja pkg-config
 
-4. Install external dependencies
---------------------------------
-
-.. code-block::
-
-   conda install -c conda-forge ffmpeg
-
-5. Clone the torchaudio repository
+4. Clone the torchaudio repository
 ----------------------------------
 
 .. code-block::
 
    git clone https://github.com/pytorch/audio
    cd audio
 
-6. Build
+5. Build
 --------
 
 .. code-block::
 
-   USE_FFMPEG=1 python setup.py develop
+   python setup.py develop
 
 .. note::
    Due to the complexity of build process, TorchAudio only supports in-place build.
    To use ``pip``, please use ``--no-use-pep517`` option.
 
-   ``USE_FFMPEG=1 pip install -v -e . --no-use-pep517``
+   ``pip install -v -e . --no-use-pep517``
+
+[Optional] Build TorchAudio with a custom built FFmpeg
+------------------------------------------------------
+
+By default, torchaudio tries to build FFmpeg extension with support for multiple FFmpeg versions. This process uses pre-built FFmpeg libraries compiled for specific CPU architectures like ``x86_64`` and ``aarch64`` (``arm64``).
+
+If your CPU is not one of those, then the build process can fail. To workaround, one can disable FFmpeg integration (by setting the environment variable ``USE_FFMPEG=0``) or switch to the single version FFmpeg extension.
+
+To build single version FFmpeg extension, FFmpeg binaries must be provided by user and available in the build environment. To do so, install FFmpeg and set ``FFMPEG_ROOT`` environment variable to specify the location of FFmpeg.
+
+.. code-block::
+
+   conda install -c conda-forge ffmpeg
+   FFMPEG_ROOT=${CONDA_PREFIX} python setup.py develop

docs/source/build.windows.rst

@@ -118,10 +118,6 @@ When using conda, the directories are ``${CONDA_PREFIX}/bin``, ``${CONDA_PREFIX}
 
    conda install cmake ninja
 
-.. code-block::
-
-   conda install -c conda-forge ffmpeg
-
 6. Build TorchAudio
 -------------------
 
@@ -136,20 +132,34 @@ Now that we have everything ready, we can build TorchAudio.
 .. code-block::
 
    # In Command Prompt
-   set USE_FFMPEG=1
    python setup.py develop
 
 .. code-block::
 
    # In Bash
-   USE_FFMPEG=1 python setup.py develop
+   python setup.py develop
 
 .. note::
    Due to the complexity of build process, TorchAudio only supports in-place build.
    To use ``pip``, please use ``--no-use-pep517`` option.
 
-   ``USE_FFMPEG=1 pip install -v -e . --no-use-pep517``
+   ``pip install -v -e . --no-use-pep517``
+
+[Optional] Build TorchAudio with a custom FFmpeg
+------------------------------------------------
+
+By default, torchaudio tries to build FFmpeg extension with support for multiple FFmpeg versions. This process uses pre-built FFmpeg libraries compiled for specific CPU architectures like ``x86_64``.
+
+If your CPU is different, then the build process can fail. To workaround, one can disable FFmpeg integration (by setting the environment variable ``USE_FFMPEG=0``) or switch to the single version FFmpeg extension.
+
+To build single version FFmpeg extension, FFmpeg binaries must be provided by user and available in the build environment. To do so, install FFmpeg and set ``FFMPEG_ROOT`` environment variable to specify the location of FFmpeg.
+
+.. code-block::
+
+   conda install -c conda-forge ffmpeg
+   FFMPEG_ROOT=${CONDA_PREFIX}/Library python setup.py develop
 
+   
 [Optional] Building FFmpeg from source
 --------------------------------------
 
@@ -170,6 +180,10 @@ FFmpeg's official documentation touches this https://trac.ffmpeg.org/wiki/Compil
 
 Please follow the instruction at https://www.msys2.org/ to install MSYS2.
 
+.. note::
+
+   In CI environment, often `Chocolatery <https://chocolatey.org/>`_ can be used to install MSYS2.
+
 2. Launch MSYS2
 ~~~~~~~~~~~~~~~
 
@@ -229,22 +243,3 @@ If the build succeeds, ``ffmpeg.exe`` should be found in the same directory. Mak
 Check that the resulting FFmpeg binary is accessible from Conda env
 
 Now launch a new command prompt and enable the TorchAudio development environment. Make sure that you can run the ``ffmpeg.exe`` command generated in the previous step.
-
-6. Build TorchAudio with the custom FFmpeg
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-To use this FFmpeg libraries for building torchaudio, do the following;
-1. Uninstall ``ffmpeg`` package installed by conda. ``conda uninstall ffmpeg``.
-2. When building set ``FFMPEG_ROOT`` environment variable to the directory where the libraries like ``libavcodec`` are found.
-
-.. code-block::
-
-   # In Command Prompt
-   set USE_FFMPEG=1
-   set FFMPEG_ROOT=<FFMPEG_BUILD_DIR>
-   python setup.py clean develop
-
-.. code-block::
-
-   # In Bash
-   USE_FFMPEG=1 FFMPEG_ROOT=<FFMPEG_BUILD_DIR> python setup.py clean develop

docs/source/installation.rst

@@ -16,7 +16,7 @@ Please refer to https://pytorch.org/get-started/locally/ for the details.
    each of which requires a corresponding PyTorch distribution.
 
 .. note::
-   This software was compiled against an unmodified copy of FFmpeg (licensed under `the LGPLv2.1 <https://github.com/FFmpeg/FFmpeg/blob/a5d2008e2a2360d351798e9abe883d603e231442/COPYING.LGPLv2.1>`_), with the specific rpath removed so as to enable the use of system libraries. The LGPL source can be downloaded `here <https://github.com/FFmpeg/FFmpeg/releases/tag/n4.1.8>`_.
+   This software was compiled against an unmodified copies of FFmpeg, with the specific rpath removed so as to enable the use of system libraries. The LGPL source can be downloaded from the following locations: `n4.1.8 <https://github.com/FFmpeg/FFmpeg/releases/tag/n4.1.8>`_ (`license <https://github.com/FFmpeg/FFmpeg/blob/n4.1.8/COPYING.LGPLv2.1>`_), `n5.0.3 <https://github.com/FFmpeg/FFmpeg/releases/tag/n5.0.3>`_ (`license <https://github.com/FFmpeg/FFmpeg/blob/n5.0.3/COPYING.LGPLv2.1>`_) and `n6.0 <https://github.com/FFmpeg/FFmpeg/releases/tag/n6.0>`_ (`license <https://github.com/FFmpeg/FFmpeg/blob/n6.0/COPYING.LGPLv2.1>`_).
 
 Dependencies
 ------------

packaging/ffmpeg/build.bat

@@ -1,10 +1,9 @@
 @echo off
 
-if exist "third_party\ffmpeg\" goto end
 set PROJ_FOLDER=%cd%
 
 choco install -y --no-progress msys2 --package-parameters "/NoUpdate"
 C:\tools\msys64\usr\bin\env MSYSTEM=MINGW64 /bin/bash -l -c "pacman -S --noconfirm --needed base-devel mingw-w64-x86_64-toolchain diffutils"
-C:\tools\msys64\usr\bin\env MSYSTEM=MINGW64 /bin/bash -l -c "cd ${PROJ_FOLDER} && export FFMPEG_ROOT=${PWD}/third_party/ffmpeg && packaging/vc_env_helper.bat bash ./packaging/ffmpeg/build.sh"
+C:\tools\msys64\usr\bin\env MSYSTEM=MINGW64 /bin/bash -l -c "cd ${PROJ_FOLDER} && packaging/vc_env_helper.bat bash ./packaging/ffmpeg/build.sh"
 
 :end

test/torchaudio_unittest/common_utils/case_utils.py

@@ -112,7 +112,7 @@ class TorchaudioTestCase(TestBaseMixin, PytorchTestCase):
 
 
 def is_ffmpeg_available():
-    return torchaudio._extension._FFMPEG_INITIALIZED
+    return torchaudio._extension._FFMPEG_EXT is not None
 
 
 _IS_CTC_DECODER_AVAILABLE = None