-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
support parameter sync with unbalanced pipe stages #106
base: main
Are you sure you want to change the base?
Conversation
layer_re = re.compile(r'layers\.([0-9]+)') | ||
layer_set = set() | ||
for name in self.named_parameters: | ||
layer_num = re.findall(layer_re, name) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这里是只匹配layers中的参数?对embedding之类的能否兼容?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
embedding 得到的 layer_num 为空,不会加到layer_set里
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
embedding的后面是怎么索引到?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这个 function是计算layer数,embedding 现在没有算一个单独的layer
No description provided.