Add evaluation, multimodal, native tests to run-readme-pr-mps.yml

[mikekgfb]

Add evaluation, multimodal, native tests to run-readme-pr-mps.yml

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/1410

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Unrelated Failure

As of commit 0de9067 with merge base 68b8087 ():

NEW FAILURE - The following job has failed:

• Run the aoti runner with CUDA using stories / test-runner-aot-cuda / linux-job (gh)
  RuntimeError: Command docker exec -t fe5a7ee696bca8649c516d8eecb2b7f2fcf8371bdb63b41574c2ee079467588e /exec failed with exit code 1

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / runner-aoti (16-core-ubuntu) (gh) (trunk failure)
  Process completed with exit code 1.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[Jack-Khuu]

Just realized that none of the test in this file are being ran:

Invalid workflow file: .github/workflows/run-readme-pr-mps.yml#L13
The workflow is not valid. .github/workflows/run-readme-pr-mps.yml (Line: 13, Col: 24): Invalid input, timeout-minutes is not defined in the referenced workflow.

[mikekgfb]

Just realized that none of the test in this file are being ran:

Invalid workflow file: .github/workflows/run-readme-pr-mps.yml#L13
The workflow is not valid. .github/workflows/run-readme-pr-mps.yml (Line: 13, Col: 24): Invalid input, timeout-minutes is not defined in the referenced workflow.

Should be fixed now. Please rerun tests?

[mikekgfb]

Just realized that none of the test in this file are being ran:

Invalid workflow file: .github/workflows/run-readme-pr-mps.yml#L13
The workflow is not valid. .github/workflows/run-readme-pr-mps.yml (Line: 13, Col: 24): Invalid input, timeout-minutes is not defined in the referenced workflow.

Should be fixed now. Please rerun tests?

Still timed out @ 50 minutes, increased timeout further.

Beyond the "let's extend timeout" bandaid, I'm wondering whether this may be due to us trying to update llvm+openmp with the command conda create -y -n test-readme-mps-macos python=3.10.11 llvm-openmp which may be adding significant runtime to the test? This might benefit from somebody looking at it who is familiar with pytorch ci/cd setup on github? (i.e., the pt releng team.

Also, MPS is the only set of doc tests using conda, which on the one hand is good because we're testing setup with conda as well, but otoh makes MPS the odd man platform out at the moment.

[malfet]

Nice typo fix, should we land it as separate PR?