Skip to content

Commit

Permalink
Fix parsing of pp split points from toml file
Browse files Browse the repository at this point in the history
Previously, string_list formatting was only applied to pp split points
arg when it came from the cmd line.  The string_list formatting needs to
be applied when loading via toml too.

ghstack-source-id: fb41e66314e7fd79c02509ba68e3dc590d5f6976
Pull Request resolved: #450
  • Loading branch information
wconstab committed Jul 10, 2024
1 parent 3e3a913 commit 4be6bbb
Showing 1 changed file with 9 additions and 0 deletions.
9 changes: 9 additions & 0 deletions torchtitan/config_manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -514,6 +514,15 @@ def parse_args(self, args_list: list = sys.argv[1:]):
logger.exception(f"Error details: {str(e)}")
raise e

if (
"experimental" in args_dict
and "pipeline_parallel_split_points" in args_dict["experimental"]
):
exp = args_dict["experimental"]
exp["pipeline_parallel_split_points"] = string_list(
exp["pipeline_parallel_split_points"]
)

# override args dict with cmd_args
cmd_args_dict = self._args_to_two_level_dict(cmd_args)
for section, section_args in cmd_args_dict.items():
Expand Down

0 comments on commit 4be6bbb

Please sign in to comment.