Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

(realtime-experimental) Add MediaMTX Docker image #209

Open
wants to merge 5 commits into
base: realtime-ai-experimental
Choose a base branch
from
Open

(realtime-experimental) Add MediaMTX Docker image #209

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
33ecbe3
readme: update with note
emranemran Sep 18, 2024
51e4f2f
add mediamtx base config
j0sh Sep 20, 2024
5128b95
updates to mediamtx from base config
j0sh Sep 20, 2024
276e2d6
Add Dockerfile.mediamtx, README
j0sh Sep 20, 2024
eaa1dbc
Update runner/docker/README.md
j0sh Sep 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
# ai-worker

> [!NOTE]
> This branch will explore adding AI inference to realtime video streams. Majority of the commits here will likely change significantly and not intended for production purposes.

> [!WARNING]
> The AI network is in it's **Beta** phase and although it is ready for production it is still under development. Please report any issues you encounter to the [Livepeer Discord](https://discord.gg/7nbPbTK).

Expand Down
7 changes: 7 additions & 0 deletions runner/docker/Dockerfile.mediamtx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
FROM bluenviron/mediamtx:1.9.0-ffmpeg
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, we plan to run media server as a separate Docker container and then send video over the HTTP into ai-runner container. Is that correct?

Asking to understand it better. I think it's fine. It'll both run locally, so AFAIU the only additional time would be to segment the video and send it over HTTP POST Form Data.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, we plan to run media server as a separate Docker container and then send video over the HTTP into ai-runner container. Is that correct?

Yes, for the post-MVP phase. This PR is supposed to be the throwaway MVP setup where we ingest directly into the worker.

"Not for prod" - famous last words

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I guess this PR is for MVP-phase and it's already running mediamtx as a separate Docker container. So, do I understand correctly that even for MVP we plan to run mediamtx and ai-runner in two separate containers?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We had some offline discussions around this and summary is one ai-runner like docker container that will do the following:

mediamtx (ingress) -> decode video -> encode image -> inference -> encode video -> mediamtx (egress)

Post MVP, we might move to HTTP streaming workflow which would require swapping out the mediamtx bits but for now, it serves it's purpose to allow ingress/egress capabilities to the ai-runner container. Having separate containers creates more moving parts along with more integration code across boundaries - let's keep it simple for now and evolve it as we build.

So we'll jam in everything in this new container that Josh started.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up comment: we might have to move to a two container approach post-MVP. One thing we're seeing is the requirements/dependencies of the model and the requirements/dependencies of the rest of the pipeline (video-decode -> image-encode etc) might conflict with one another.

On the flip side, there's also the idea that if we can keep everything within GPU memory (zero-copy), then that will be better suited to a single container. But then we violate the docker "one concern per container" which is not a hard and fast rule and is generally a recommendation -- we can easily use a supervisor daemon process to manage lifetime of the individual components of the realtime pipeline.

For now, it's easiest to keep it in one container and continue building for the MVP. Once we have it in place, we should have more info on which path to take.

Copy link

@leszko leszko Sep 25, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It all makes sense. The thing I'm confused about is that this PR implements the two-container solution (mediamtx is one container, ai-runnner is a separate container). While you mentioned about that the MVP solution is one-container 🤔


RUN apk add --no-cache curl

COPY mediamtx.yml .

ENTRYPOINT ["/mediamtx"]
21 changes: 21 additions & 0 deletions runner/docker/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,24 @@ To build a pipeline-specific container, you need to build the base container fir
```

This command builds the `segment-anything-2` pipeline-specific container using the Dockerfile located at [docker/Dockerfile.segment_anything_2](docker/Dockerfile.segment_anything_2) and tags it as `livepeer/ai-runner:segment-anything-2`.

4. **Set Up the `mediamtx` Container for Ingest**:

```bash
docker build -f Dockerfile.mediamtx -t livepeer/ai-runner:mediamtx .
```

This command builds and configures [MediaMTX](https://github.com/bluenviron/mediamtx) for ingest. Until we have docker-compose or similar orchestrator, it will need to be run manually in order to listen for incoming connections

```bash
docker run --rm -it -p 8189:8189/udp -p 8889:8889 -p 1935:1935 -p 9998:9998 livepeer/ai-runner:mediamtx
```

This will forward the following ports from the host container:

Port | Purpose
---|---
8189 | WebRTC UDP
8889 | WHIP HTTP
1935 | RTMP
9998 | Prometheus /metrics
Loading
Loading