Create doc and tests for distributed inference #1358
Assignees
Labels
actionable
Items in the backlog waiting for an appropriate impl/fix
Distributed
Issues related to all things distributed
documentation
Improvements or additions to documentation
triaged
This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Comments
Jack-Khuu
added
documentation
|
Definitely something we plan to add! |
Jack-Khuu
added
the
triaged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
🚀 The feature, motivation and pitch
Once distributed inference integration into torchchat is functional, let's add a docs/distributed.md with an example, and plumb that example into
.ci/scripts/run-docs distributed. (updown.py extracts all commands between triple backticks into a test script.)torchchat has the same runners as pytorch/pytorch, so at least a minimal 2 or 4 GPU setup on a single node would be great. Not sure whether we can run multi-node testing, you can suppress commands from tests with
[skip default]: beginand[skip default]: endaround those commands.cc: @mreso @lessw2020 @kwen2501
Alternatives
None
Additional context
No response
RFC (Optional)
No response
The text was updated successfully, but these errors were encountered: