Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BE] Migration of Builder Repository to pytorch/pytorch and pytorch/test-infra repository #2054

Open
10 of 14 tasks
atalman opened this issue Dec 9, 2024 · 0 comments
Open
10 of 14 tasks

[BE] Migration of Builder Repository to pytorch/pytorch and pytorch/test-infra repository #2054

atalman opened this issue Dec 9, 2024 · 0 comments

Comments

@atalman
Copy link
Contributor

atalman commented Dec 9, 2024 •
edited
Loading

Our Nightly and Release pipeline spread over 3 repositories: pytorch/pytorch, pytorch/builder and pytorch/test-infra .This creates maintenance overhead and following issues:

  • Builder Repo Changes are not getting tested by PyTorch CI on PR. To test these changes developer need to open special PR against Pytorch/Pytorch poining to his builder fork. This makes it extremely difficult to test changes to builder repo
  • Changes to Docker builds can't be tested.
  • During Pytorch Release process we perform branch cut and cherry-picking to builder repo, not only pytorch/pytorch

Hence to solve these issues we are working on Moving all Build and Test Scripts for pytorch binary to pytorch/pytorch
All Validation and analytics scripts to pytorch/test-infra repository.

Following Work Has been Completed:

Please note: If you intend to add or modify PyTorch build or test scripts please do it directly in pytorch/pytorch repository
@atalman atalman pinned this issue Dec 9, 2024
@atalman atalman mentioned this issue Dec 10, 2024
Closed
pytorchmergebot pushed a commit to pytorch/pytorch that referenced this issue Dec 10, 2024
@atalman @pytorchmergebot
Migrate smoke tests to pytorch/pytorch (#142482)
f57606a 
Related to pytorch/builder#2054
This should fix nightly xpu failure: https://github.com/pytorch/pytorch/actions/runs/12251477588/job/34180135207 and rocm failure: https://github.com/pytorch/pytorch/actions/runs/12251477588/job/34182185374 due to missing : `` /builder/check_binary.sh``

Builder Scripts revision: pytorch/builder@3468139
Pull Request resolved: #142482
Approved by: https://github.com/chuanqi129, https://github.com/kit1980, https://github.com/malfet, https://github.com/jeffdaily, https://github.com/huydhn
mori360 pushed a commit to mori360/pytorch that referenced this issue Dec 11, 2024
@atalman @mori360
Migrate smoke tests to pytorch/pytorch (pytorch#142482)
868ba10 
Related to pytorch/builder#2054
This should fix nightly xpu failure: https://github.com/pytorch/pytorch/actions/runs/12251477588/job/34180135207 and rocm failure: https://github.com/pytorch/pytorch/actions/runs/12251477588/job/34182185374 due to missing : `` /builder/check_binary.sh``

Builder Scripts revision: pytorch/builder@3468139
Pull Request resolved: pytorch#142482
Approved by: https://github.com/chuanqi129, https://github.com/kit1980, https://github.com/malfet, https://github.com/jeffdaily, https://github.com/huydhn
@atalman atalman mentioned this issue Dec 11, 2024
Merged
atalman added a commit to pytorch/test-infra that referenced this issue Dec 11, 2024
@atalman
Move Validation Framework workflows to test-infra (#6029)
550a01b 
Moves all the worklfows from builder Validation framework except:

https://github.com/pytorch/builder/blob/main/.github/workflows/validate-repackaged-binary-sizes.yml

Part of pytorch/builder#2054

It needs to be rewritten, followup PR required.
@atalman atalman mentioned this issue Dec 11, 2024
Closed
pytorchmergebot pushed a commit to pytorch/pytorch that referenced this issue Dec 12, 2024
@atalman @pytorchmergebot
Use validate-docker-images workflow from test-infra (#143081)
bd7d81d 
After PR: pytorch/test-infra#6029 use validate-docker-images.yml from test-infra.
Related to: pytorch/builder#2054

Pull Request resolved: #143081
Approved by: https://github.com/huydhn
pytorchbot pushed a commit to pytorch/pytorch that referenced this issue Dec 12, 2024
@atalman @pytorchbot
Use validate-docker-images workflow from test-infra (#143081)
5d44d1f 
After PR: pytorch/test-infra#6029 use validate-docker-images.yml from test-infra.
Related to: pytorch/builder#2054

Pull Request resolved: #143081
Approved by: https://github.com/huydhn

(cherry picked from commit bd7d81d)
@pytorchbot pytorchbot mentioned this issue Dec 12, 2024
Merged
kit1980 pushed a commit to pytorch/pytorch that referenced this issue Dec 12, 2024
@pytorchbot @atalman
Use validate-docker-images workflow from test-infra (#143083)
c69eae3 
Use validate-docker-images workflow from test-infra (#143081)

After PR: pytorch/test-infra#6029 use validate-docker-images.yml from test-infra.
Related to: pytorch/builder#2054

Pull Request resolved: #143081
Approved by: https://github.com/huydhn

(cherry picked from commit bd7d81d)

Co-authored-by: atalman <[email protected]>
This was referenced Dec 12, 2024
Merged
Closed
pytorchmergebot pushed a commit to pytorch/pytorch that referenced this issue Dec 13, 2024
@atalman @pytorchmergebot
Add check_binary workflow to pytorch/pytorch (#143201)
fe9365f 
Migrated from pytorch/builder
Related to: pytorch/builder#2054

Copying from : pytorch/builder@3468139
Pull Request resolved: #143201
Approved by: https://github.com/seemethere, https://github.com/malfet
bluenote10 pushed a commit to bluenote10/pytorch that referenced this issue Dec 14, 2024
@atalman @bluenote10
Migrate smoke tests to pytorch/pytorch (pytorch#142482)
8225497 
Related to pytorch/builder#2054
This should fix nightly xpu failure: https://github.com/pytorch/pytorch/actions/runs/12251477588/job/34180135207 and rocm failure: https://github.com/pytorch/pytorch/actions/runs/12251477588/job/34182185374 due to missing : `` /builder/check_binary.sh``

Builder Scripts revision: pytorch/builder@3468139
Pull Request resolved: pytorch#142482
Approved by: https://github.com/chuanqi129, https://github.com/kit1980, https://github.com/malfet, https://github.com/jeffdaily, https://github.com/huydhn
bluenote10 pushed a commit to bluenote10/pytorch that referenced this issue Dec 14, 2024
@atalman @bluenote10
Use validate-docker-images workflow from test-infra (pytorch#143081)
7e0a0f2 
After PR: pytorch/test-infra#6029 use validate-docker-images.yml from test-infra.
Related to: pytorch/builder#2054

Pull Request resolved: pytorch#143081
Approved by: https://github.com/huydhn
bluenote10 pushed a commit to bluenote10/pytorch that referenced this issue Dec 14, 2024
@atalman @bluenote10
Add check_binary workflow to pytorch/pytorch (pytorch#143201)
662723d 
Migrated from pytorch/builder
Related to: pytorch/builder#2054

Copying from : pytorch/builder@3468139
Pull Request resolved: pytorch#143201
Approved by: https://github.com/seemethere, https://github.com/malfet
aditew01 pushed a commit to aditew01/pytorch that referenced this issue Dec 18, 2024
@atalman @aditew01
Add check_binary workflow to pytorch/pytorch (pytorch#143201)
61d312b 
Migrated from pytorch/builder
Related to: pytorch/builder#2054

Copying from : pytorch/builder@3468139
Pull Request resolved: pytorch#143201
Approved by: https://github.com/seemethere, https://github.com/malfet
This was referenced Dec 23, 2024
Merged
Merged
Closed
pytorchmergebot pushed a commit to pytorch/pytorch that referenced this issue Dec 24, 2024
@atalman @pytorchmergebot
Remove builder repo from workflows and scripts (#143776)
6c32ef4 
Part of pytorch/builder#2054
Builder is repo is no longer used. Hence remove any references to builder repo

Pull Request resolved: #143776
Approved by: https://github.com/huydhn
@atalman atalman mentioned this issue Dec 24, 2024
Merged
atalman added a commit to pytorch/test-infra that referenced this issue Dec 24, 2024
@atalman
Move analytics scripts from builder (#6111)
9cef6c2 
Part of pytorch/builder#2054
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@atalman