Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable dynamo export: remove unnecessary model attribute assignment on 'freqs_cis' #349

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Enable dynamo export: remove unnecessary model attribute assignment on 'freqs_cis' #349

wants to merge 1 commit into from

Conversation

BowenBao
Copy link

Otherwise a graph break is triggered.

        Mutating module attribute freqs_cis during export.
        
        from user code:
           File "/bert_ort/bowbao/repos/bench/torchbenchmark/torchbenchmark/models/llama/model.py", line 227, in forward
            self.freqs_cis = self.freqs_cis.to(h.device)
        
        Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information
        
        
        You can suppress this exception and fall back to eager by setting:
            import torch._dynamo
            torch._dynamo.config.suppress_errors = True

Unblocks #pytorch/benchmark#1766 to fix pytorch/benchmark#1767.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 14, 2023
@BowenBao BowenBao mentioned this pull request Jul 14, 2023
Closed
facebook-github-bot pushed a commit to pytorch/benchmark that referenced this pull request Jul 19, 2023
@BowenBao @facebook-github-bot
Remove unnecessary model attribute assignment on 'freqs_cis' (#1766)
e8c1cf0 
Summary:
Fixes #1767, more details there.

Upstream PR link meta-llama/llama#349

Pull Request resolved: #1766

Reviewed By: msaroufim

Differential Revision: D47556245

Pulled By: xuzhao9

fbshipit-source-id: 5ad541ff3e38a36b6515b1b7135b3da92109c9f5
@msaroufim msaroufim self-requested a review September 1, 2023 20:11
@msaroufim msaroufim added the performance Runtime / memory / accuracy performance issues label Sep 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@msaroufim msaroufim msaroufim approved these changes

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. performance Runtime / memory / accuracy performance issues
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[export][llama] AssertionError: Mutating module attribute freqs_cis during export
3 participants
@BowenBao @msaroufim @facebook-github-bot