Upgrade to 7.x

[ackava]

Place holder issue to upgrade to 7.x, this library is great but there are various bugs in 5.x which were resolved later and can work with latest high def files.

Questions to address.

1. Video Conversion is a heavy process, for most file uploads, if conversion can be done in browser, it will help reducing server resource costs. And any such URL endpoint can easily accommodate SharedArrayBuffer with required headers. Time required to upload video and convert it on server is less than converting with single thread. There seems no benefit of single threaded ffmpeg. So is it alright to drop single threaded core because the performance is very poor?

2. Are there any blocking issues from moving core-mt to ffmpeg 7?

[ghnp5]

I was just wondering this.
I've just realized that a "serious" issue/bug was only happening on the browser's ffmpeg (wasm), while when I test on my computer (ffmpeg 7), it worked perfectly.

Unfortunately, this bug can only be "resolved" if I re-encode the video (rather than a simple -c copy) which will require users to wait several minutes until their videos are processed.

Or, alternatively, we'd use ffmpeg 7.

But, I've absolutely no idea how easy it is to upgrade this project to ffmpeg 7,
and the repo author may no longer be as active (based on README asking for maintainers).

Does anyone know what would be necessary to do, to upgrade to ffmpeg 7?

---

It may be as simple as updating the Dockerfile:

# We cannot upgrade to n6.0 as ffmpeg bin only supports multithread at the moment.
ENV FFMPEG_VERSION=n5.1.4

And then we just have to use multithread mode?

I can give it a try tomorrow! Hopefully I'll not run into many issues.

[ghnp5]

Ok... I've just realized there are a lot of other repos that are dependencies, for each of the codecs, e.g. https://github.com/ffmpegwasm/x265.git

They probably need their own updates too, to support ffmpeg v7.

This looks like a battle I simply cannot win.

For example, x265 build is crashing with a lot of:

clang++: error: unsupported option '-march=' for target 'wasm32-unknown-emscripten'

[ghnp5]

I made all these updates to the repos parts (as I realized they're clones),
but still getting errors.

I'll have to continue later, as I'm exhausted now :) been a long day.

# Build x264
FROM emsdk-base AS x264-builder
#ENV X264_BRANCH=4-cores
#ADD https://github.com/ffmpegwasm/x264.git#$X264_BRANCH /src
ENV X264_BRANCH=stable
ADD https://code.videolan.org/videolan/x264.git#$X264_BRANCH /src
COPY build/x264.sh /src/build.sh
RUN bash -x /src/build.sh

# Build x265
FROM emsdk-base AS x265-builder
#ENV X265_BRANCH=3.4
#ADD https://github.com/ffmpegwasm/x265.git#$X265_BRANCH /src
ENV X265_BRANCH=stable
ADD https://bitbucket.org/multicoreware/x265_git.git#$X265_BRANCH /src
COPY build/x265.sh /src/build.sh
RUN bash -x /src/build.sh

# Build libvpx
FROM emsdk-base AS libvpx-builder
#ENV LIBVPX_BRANCH=v1.13.1
#ADD https://github.com/ffmpegwasm/libvpx.git#$LIBVPX_BRANCH /src
ENV LIBVPX_BRANCH=v1.14.1
ADD https://chromium.googlesource.com/webm/libvpx#$LIBVPX_BRANCH /src
COPY build/libvpx.sh /src/build.sh
RUN bash -x /src/build.sh

# Build lame
FROM emsdk-base AS lame-builder
ENV LAME_BRANCH=master
ADD https://github.com/ffmpegwasm/lame.git#$LAME_BRANCH /src
# https://sourceforge.net/projects/lame/postdownload
COPY build/lame.sh /src/build.sh
RUN bash -x /src/build.sh

# Build ogg
FROM emsdk-base AS ogg-builder
#ENV OGG_BRANCH=v1.3.4
#ADD https://github.com/ffmpegwasm/Ogg.git#$OGG_BRANCH /src
ENV OGG_BRANCH=v1.3.5
ADD https://gitlab.xiph.org/xiph/ogg#$OGG_BRANCH /src
COPY build/ogg.sh /src/build.sh
RUN bash -x /src/build.sh

# Build theora
FROM emsdk-base AS theora-builder
COPY --from=ogg-builder $INSTALL_DIR $INSTALL_DIR
ENV THEORA_BRANCH=v1.1.1
ADD https://github.com/ffmpegwasm/theora.git#$THEORA_BRANCH /src
#ENV THEORA_BRANCH=v1.1.1  (same version, at the moment)
#ADD https://gitlab.xiph.org/xiph/theora#$THEORA_BRANCH /src
COPY build/theora.sh /src/build.sh
RUN bash -x /src/build.sh

# Build opus
FROM emsdk-base AS opus-builder
#ENV OPUS_BRANCH=v1.3.1
#ADD https://github.com/ffmpegwasm/opus.git#$OPUS_BRANCH /src
ENV OPUS_BRANCH=v1.5.2
ADD https://github.com/xiph/opus#$OPUS_BRANCH /src
COPY build/opus.sh /src/build.sh
RUN bash -x /src/build.sh

# Build vorbis
FROM emsdk-base AS vorbis-builder
COPY --from=ogg-builder $INSTALL_DIR $INSTALL_DIR
#ENV VORBIS_BRANCH=v1.3.3
#ADD https://github.com/ffmpegwasm/vorbis.git#$VORBIS_BRANCH /src
ENV VORBIS_BRANCH=v1.3.7
ADD https://gitlab.xiph.org/xiph/vorbis#$VORBIS_BRANCH /src
COPY build/vorbis.sh /src/build.sh
RUN bash -x /src/build.sh

# Build zlib
FROM emsdk-base AS zlib-builder
#ENV ZLIB_BRANCH=v1.2.11
#ADD https://github.com/ffmpegwasm/zlib.git#$ZLIB_BRANCH /src
ENV ZLIB_BRANCH=v1.3.1
ADD https://github.com/madler/zlib#$ZLIB_BRANCH /src
COPY build/zlib.sh /src/build.sh
RUN bash -x /src/build.sh

# Build libwebp
FROM emsdk-base AS libwebp-builder
COPY --from=zlib-builder $INSTALL_DIR $INSTALL_DIR
#ENV LIBWEBP_BRANCH=v1.3.2
#ADD https://github.com/ffmpegwasm/libwebp.git#$LIBWEBP_BRANCH /src
ENV LIBWEBP_BRANCH=v1.4.0
ADD https://github.com/webmproject/libwebp#$LIBWEBP_BRANCH /src
COPY build/libwebp.sh /src/build.sh
RUN bash -x /src/build.sh

# Build freetype2
FROM emsdk-base AS freetype2-builder
#ENV FREETYPE2_BRANCH=VER-2-10-4
#ADD https://github.com/ffmpegwasm/freetype2.git#$FREETYPE2_BRANCH /src
ENV FREETYPE2_BRANCH=VER-2-13-3
ADD https://git.savannah.gnu.org/git/freetype/freetype2.git#$FREETYPE2_BRANCH /src
COPY build/freetype2.sh /src/build.sh
RUN bash -x /src/build.sh

# Build fribidi
FROM emsdk-base AS fribidi-builder
ENV FRIBIDI_BRANCH=v1.0.16
ADD https://github.com/fribidi/fribidi.git#$FRIBIDI_BRANCH /src
COPY build/fribidi.sh /src/build.sh
RUN bash -x /src/build.sh

# Build harfbuzz
FROM emsdk-base AS harfbuzz-builder
ENV HARFBUZZ_BRANCH=10.0.1
ADD https://github.com/harfbuzz/harfbuzz.git#$HARFBUZZ_BRANCH /src
COPY build/harfbuzz.sh /src/build.sh
RUN bash -x /src/build.sh

# Build libass
FROM emsdk-base AS libass-builder
COPY --from=freetype2-builder $INSTALL_DIR $INSTALL_DIR
COPY --from=fribidi-builder $INSTALL_DIR $INSTALL_DIR
COPY --from=harfbuzz-builder $INSTALL_DIR $INSTALL_DIR
ENV LIBASS_BRANCH=0.17.3
ADD https://github.com/libass/libass.git#$LIBASS_BRANCH /src
COPY build/libass.sh /src/build.sh
RUN bash -x /src/build.sh

# Build zimg
FROM emsdk-base AS zimg-builder
ENV ZIMG_BRANCH=release-3.0.5
RUN apt-get update && apt-get install -y git
RUN git clone --recursive -b $ZIMG_BRANCH https://github.com/sekrit-twc/zimg.git /src
COPY build/zimg.sh /src/build.sh
RUN bash -x /src/build.sh

[ghnp5]

I finally got Success with this. I got a ".wasm" file at the end, with ffmpeg 7.0.2.

I had to disable x265 and Vorbis, as I couldn't get those building, after many hours spent trying to fix the errors. I'll give it a try later. (I'm exhausted again, from all day on this...)

Took me a while to realize that there are actual changes on the fftools code, as it's not just a Snapshot of ffmpeg C++ code. It has changes to bridge the ffmpeg code with JS/WASM. So I had to replicate those changes, for 7.0.2. (a lot has changed since 5.1.2, so I had to redo them carefully)

Another thing that took me a lot of time was that I was trying to upgrade to 7.1 (instead of 7.0.2), but that version requires stdbit.h, which isn't available in the GCC packages in Debian, yet, as it's a recent thing. Eventually I realized I should target 7.0.2 instead, as it's "the widely stable release" currently.

I tried using this ".wasm" file, but I got an error, which seems to be conflicting with the rest of the JS files. That's another thing I have to look at.

Tomorrow is another day...

[ghnp5]

Been trying to figure out (trying MANY things, many parameters, many flags, etc...) why v7 is significantly slower than v5, processing the same video...

(and ironically, v7 is running on MT, while v5 is running on ST...)

v5 (current ffmpeg wasm, ST):

22:12:33.719 frame= 0 fps=0.0 q=-1.0 size= 0kB time=00:00:00.00 bitrate=N/A speed=N/A
22:12:33.832 frame= 1850 fps=0.0 q=-1.0 Lsize= 9427kB time=00:00:59.96 bitrate=1287.9kbits/s speed= 526x

v7 (my upgraded wasm, MT):

22:33:13.425 size= 768KiB time=00:00:04.51 bitrate=1394.1kbits/s speed=8.96x
22:33:13.940 size= 2048KiB time=00:00:11.84 bitrate=1416.2kbits/s speed=11.6x
22:33:14.450 size= 3072KiB time=00:00:17.84 bitrate=1410.1kbits/s speed=11.7x
22:33:14.965 size= 4096KiB time=00:00:24.21 bitrate=1385.8kbits/s speed=11.9x
22:33:15.475 size= 4864KiB time=00:00:30.34 bitrate=1313.0kbits/s speed=11.9x
22:33:15.990 size= 5888KiB time=00:00:36.38 bitrate=1325.9kbits/s speed=11.9x
22:33:16.502 size= 6912KiB time=00:00:43.01 bitrate=1316.4kbits/s speed= 12x
22:33:17.004 size= 7936KiB time=00:00:49.01 bitrate=1326.4kbits/s speed= 12x
22:33:17.501 size= 9216KiB time=00:00:55.64 bitrate=1356.7kbits/s speed=12.2x
22:33:17.550 size= 9427KiB time=00:00:59.88 bitrate=1289.7kbits/s speed=12.9x

---

Note: one of the things I tried was to add -sPROXY_TO_PTHREAD, but this requires main() to exist. So, I created a main() function that just calls ffmpeg() (the "renamed" main()), and it compiled. But then, the video processing hangs completely.

[ghnp5]

-sPROXY_TO_PTHREAD is actually what fixed the issue above!!

Had to make several changes, and do a setInterval that listens to Module.ret, and then executes a callback passed to exec.

Offloading/proxying to pthread makes it run instantly again. My theory here is that MT causes many threads to be created just to run this simple command, and then having to manage them and "merge" everything back, could be what was causing this huge delay...

I have a functional ffmpeg v7 now (but still 2 codecs to fix, and it's very "fitting my purpose", so I'm not in a position to do a pull request).