Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[dr ci] Flaky log classifier classifies non-infra failure as flaky #5621

Open
malfet opened this issue Sep 3, 2024 · 0 comments
Open

[dr ci] Flaky log classifier classifies non-infra failure as flaky #5621

malfet opened this issue Sep 3, 2024 · 0 comments
Assignees

Comments

@malfet
Copy link
Contributor

malfet commented Sep 3, 2024

For PR pytorch/pytorch#99272
image
Thinks this failure https://github.com/pytorch/pytorch/actions/runs/10691260585/job/29638325386 is flaky/infra related, when it's clearly not:

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
=================================== FAILURES ===================================
__________________ TestAutocastMPS.test_cast_cache_is_global ___________________
Traceback (most recent call last):
  File "/Users/ec2-user/runner/_work/pytorch/pytorch/test/test_autocast.py", line 393, in test_cast_cache_is_global
    self.assertEqual(weight_dtype_cast_counter, 2)
  File "/Users/ec2-user/runner/_work/_temp/conda_environment_10691260585/lib/python3.9/site-packages/torch/testing/_internal/common_utils.py", line 3885, in assertEqual
    raise error_metas.pop()[0].to_error(
AssertionError: Scalars are not equal!

Expected 2 but got 0.
Absolute difference: 2
Relative difference: 1.0

To execute this test, run the following from the base repo dir:
    python test/test_autocast.py TestAutocastMPS.test_cast_cache_is_global

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
- generated xml file: /Users/ec2-user/runner/_work/pytorch/pytorch/test/test-reports/python-pytest/test_autocast/test_autocast-bcb9258c59903eb8.xml -
=========================== short test summary info ============================
FAILED [0.0022s] test_autocast.py::TestAutocastMPS::test_cast_cache_is_global - AssertionError: Scalars are not equal!

Expected 2 but got 0.
Absolute difference: 2
Relative difference: 1.0

To execute this test, run the following from the base repo dir:
    python test/test_autocast.py TestAutocastMPS.test_cast_cache_is_global

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!
================== 1 failed, 16 deselected, 2 rerun in 0.23s ===================
Got exit code 1
Stopping at first consistent failure
The following tests failed consistently: ['test/test_autocast.py::TestAutocastMPS::test_cast_cache_is_global']
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Cold Storage
Development

No branches or pull requests

2 participants