(realtime-experimental) Add MediaMTX Docker image

[j0sh]

Hits an endpoint via curl when newly published stream is ready to be read (via runOnReady mediaMTX config).

WHIP can be published via http://<host>:8889/<streamname>/publish , assuming correct port-forwarding has been configured in the running Docker container (see README)

WHEP playback via http://<host>:8889/<streamname>
RTMP playback via http://<host>/<streamname>

Stream name can be anything for now.

Assumes we have a listener at http://realtime-ai-worker:3080/ready - hardcoded for now. This can be updated to match whatever we end up doing. The exact curl command is this:

curl http://realtime-ai-worker:3080/ready -F path=$MTX_PATH -F query=$MTX_QUERY -F source_type=$MTX_SOURCE_TYPE -F source_id=$MTX_SOURCE_ID

Diff from mediamtx base config is here: 5128b95

Presumably we will be running this manually for now so have some quick instructions in the ai-worker/runner/docker README

For reference, here is a basic Python endpoint that could listen for runOnReady events

from flask import Flask, request

app = Flask(__name__)

@app.route('/ready', methods=['POST'])
def submit_form():
    # Parsing form data
    form_data = request.form  # For 'application/x-www-form-urlencoded'

    # Printing form data
    print("Form Data:")
    for key, value in form_data.items():
        print(f"{key}: {value}")

    # Stream name in `path`
    # Access RTMP via rtmp://localhost/<streamname>
    print(f"rtmp://localhost/{form_data['path']}")

    return 'OK'

if __name__ == '__main__':
    app.run(port=3080, debug=True)

[leszko]

So, we plan to run media server as a separate Docker container and then send video over the HTTP into ai-runner container. Is that correct?

Asking to understand it better. I think it's fine. It'll both run locally, so AFAIU the only additional time would be to segment the video and send it over HTTP POST Form Data.

[j0sh]

So, we plan to run media server as a separate Docker container and then send video over the HTTP into ai-runner container. Is that correct?

Yes, for the post-MVP phase. This PR is supposed to be the throwaway MVP setup where we ingest directly into the worker.

"Not for prod" - famous last words

[leszko]

But I guess this PR is for MVP-phase and it's already running mediamtx as a separate Docker container. So, do I understand correctly that even for MVP we plan to run mediamtx and ai-runner in two separate containers?

[emranemran]

We had some offline discussions around this and summary is one ai-runner like docker container that will do the following:

mediamtx (ingress) -> decode video -> encode image -> inference -> encode video -> mediamtx (egress)

Post MVP, we might move to HTTP streaming workflow which would require swapping out the mediamtx bits but for now, it serves it's purpose to allow ingress/egress capabilities to the ai-runner container. Having separate containers creates more moving parts along with more integration code across boundaries - let's keep it simple for now and evolve it as we build.

So we'll jam in everything in this new container that Josh started.

[emranemran]

Follow-up comment: we might have to move to a two container approach post-MVP. One thing we're seeing is the requirements/dependencies of the model and the requirements/dependencies of the rest of the pipeline (video-decode -> image-encode etc) might conflict with one another.

On the flip side, there's also the idea that if we can keep everything within GPU memory (zero-copy), then that will be better suited to a single container. But then we violate the docker "one concern per container" which is not a hard and fast rule and is generally a recommendation -- we can easily use a supervisor daemon process to manage lifetime of the individual components of the realtime pipeline.

For now, it's easiest to keep it in one container and continue building for the MVP. Once we have it in place, we should have more info on which path to take.

[leszko]

It all makes sense. The thing I'm confused about is that this PR implements the two-container solution (mediamtx is one container, ai-runnner is a separate container). While you mentioned about that the MVP solution is one-container 🤔

[emranemran]

We had some offline discussions around this and summary is one ai-runner like docker container that will do the following:

mediamtx (ingress) -> decode video -> encode image -> inference -> encode video -> mediamtx (egress)

Post MVP, we might move to HTTP streaming workflow which would require swapping out the mediamtx bits but for now, it serves it's purpose to allow ingress/egress capabilities to the ai-runner container. Having separate containers creates more moving parts along with more integration code across boundaries - let's keep it simple for now and evolve it as we build.

So we'll jam in everything in this new container that Josh started.