Use pre-built binaries for ffmpeg extension #3460
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/audio/3460
Note: Links to docs will display an error until the docs builds have been completed. ❌ 4 New Failures, 1 Unrelated FailureAs of commit 505780b: NEW FAILURES - The following jobs have failed:
BROKEN TRUNK - The following job failed but were present on the merge base d9f51ce:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
mthrok
force-pushed
the
prebuilt-ffmpeg
branch
3 times, most recently
from
cfa76f8
to
d835de6
Compare
|
@mthrok has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
mthrok
force-pushed
the
prebuilt-ffmpeg
branch
5 times, most recently
from
083f072
to
a369cc4
Compare
|
@mthrok has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
Hey @mthrok. Some guidance:Use 'module: ops' for operations under 'torchaudio/{transforms, functional}', and ML-related components under 'torchaudio/csrc' (e.g. RNN-T loss). Things in "examples" directory:
Regarding examples in code documentations, please also use 'module: docs'. Please use 'other' tag only when you’re sure the changes are not much relevant to users, or when all other tags are not applicable. Try not to use it often, in order to minimize efforts required when we prepare release notes. When preparing release notes, please make sure 'documentation' and 'tutorials' occur as the last sub-categories under each primary category like 'new feature', 'improvements' or 'prototype'. Things related to build are by default excluded from the release note, except when it impacts users. For example: |
pytorch/audio#3460 changed the way torchaudio is built, and there is no longer need to supply FFmepg externally. test-infra no longer needs to build FFmpeg and use pre/post-scripts for torchaudio. The only issue remaining for now is smoke test. Currently torchaudio's smoke test script by default expects that FFmpeg libraries are available. This can be disabled with `--no-ffmpeg`, but the way YML files are written in test-infra repo does not allow to pass a flag to smoke test. We can switch the behavior, or use other smoke test.
pytorch/audio#3460 changed the way torchaudio is built, and there is no longer need to supply FFmepg externally. test-infra no longer needs to build FFmpeg and use pre/post-scripts for torchaudio. The only issue remaining for now is smoke test. Currently torchaudio's smoke test script by default expects that FFmpeg libraries are available. This can be disabled with `--no-ffmpeg`, but the way YML files are written in test-infra repo does not allow to pass a flag to smoke test. We can switch the behavior, or use other smoke test.
This commit removes all the code related to FFmpeg, which are specific to torchaudio. pytorch/audio#3460 changed the way torchaudio is built. FFmpeg integration is handled automatically and enabled by default. This eliminates the need to supply FFmepg externally. test-infra no longer needs to build FFmpeg and use pre/post-scripts for torchaudio. Note that smoke test was updated in pytorch/audio#3465 to not use FFmpeg for wheel. We will revisit this later with multiple FFmpeg version support.
Summary: In #3460, we switched the build process for FFmpeg extension. Since it is complicated to install FFmpeg in some environments, at build time, pre-built binaries and its headers are downloaded and used as a scaffolding for torchaudio build. Now even though we did not change any code or FFmpeg version, it turned out that this causes segmentation fault on Ubuntu when using system Python and FFmpeg 4.4 installed via aptitude. While investigating the issue, I swapped the said pre-built FFmpeg scaffolding with FFmpeg 4.4 from aptitude, and the segmentation fault did not happen. This indicates that it is binary compatibility issue. Before #3460, each binary build job was building FFmpeg 4.1.8 using the same compiler used to build torchaudio, but after #3460 the environments to build FFmpeg 4.1.8 and torchaudio are different. My hypothesis is that this difference is causing some ABI incompatibility when linking against FFmpeg 4.4. (Also, I don't remember well, but I read somewhere that 4.4 has a different ABI) Through experiments, it turned out upgrading the pre-built FFmpeg scaffolding to 4.4 resolves this. So this commit upgrade the pre-built FFmpeg 4 to 4.4. The potential (yet unconfirmed) downside is that torchaudio will no longer work with 4.1, 4.2, and 4.3. Since FFmpeg 4.4 is what Ubuntu 20.04 and 22.04 support by default, and Google Colab is also on 20.04, I think it is more important to support 4.4. Therefore we drop the support for 4.1-4.3 from normal build (and official distributions). Those who wish to use 4.1-4.3 can build torchaudio from source by linking to specific FFmpeg. Pull Request resolved: #3561 Reviewed By: hwangjeff Differential Revision: D48340201 Pulled By: mthrok fbshipit-source-id: 7ece82910f290c7cf83f58311c4cf6a384e8795e
This commit changes the way FFmpeg extension is built.
Originally, the build process expected the FFmpeg binaries to be somehow available in build env.
This makes the build process unpredictable and prevents default enabling FFmpeg extension.
The proposed change uses pre-built FFmpeg binaries as build-time only scaffold, which are built in our CI job https://github.com/pytorch/audio/actions/workflows/ffmpeg.yml.
This makes the build process more predictable and removes the necessity to build FFmpeg in our CI.
Currently, it supports macOS (arm64, x86_64), unix (x86_64, aarch64) and windows (amd64).
The downside is that it no longer works with the architecture not listed above.
We can potentially workaround by searching the FFmpeg binaries available in system (the old way) for
these system, but since they are not supported by PyTorch, the priority is low.