Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ARC E2E] Torchbench accuracy failed #912

Open
mengfei25 opened this issue Sep 13, 2024 · 1 comment
Open

[ARC E2E] Torchbench accuracy failed #912

mengfei25 opened this issue Sep 13, 2024 · 1 comment

Comments

@mengfei25
Copy link
Contributor

🐛 Describe the bug

Category Model Accuracy
torchbench_amp_bf16_training basic_gnn_edgecnn eager_two_runs_differ
torchbench_amp_bf16_training densenet121 fail_accuracy
torchbench_bfloat16_training densenet121 fail_accuracy
torchbench_float16_training fastNLP_Bert fail_accuracy
torchbench_bfloat16_training functorch_maml_omniglot fail_accuracy
torchbench_amp_bf16_training hf_Reformer fail_accuracy
torchbench_amp_fp16_training hf_Reformer fail_accuracy
torchbench_bfloat16_training hf_Reformer eager_two_runs_differ
torchbench_float16_training hf_Reformer fail_accuracy
torchbench_amp_bf16_training hf_Roberta_base eager_2nd_run_fail
torchbench_amp_fp16_training hf_Roberta_base eager_2nd_run_fail
torchbench_bfloat16_training hf_Roberta_base fail_accuracy
torchbench_float16_training hf_Roberta_base fail_accuracy
torchbench_float32_training hf_Roberta_base eager_2nd_run_fail
torchbench_amp_bf16_training hf_Whisper fail_accuracy
torchbench_amp_fp16_training hf_Whisper fail_accuracy
torchbench_float32_training maml_omniglot fail_accuracy
torchbench_bfloat16_inference pyhpc_equation_of_state fail_accuracy
torchbench_float16_inference pyhpc_equation_of_state fail_accuracy
torchbench_amp_fp16_training pytorch_CycleGAN_and_pix2pix fail_accuracy
torchbench_float16_training pytorch_CycleGAN_and_pix2pix fail_accuracy
torchbench_float16_inference sam fail_accuracy
torchbench_amp_bf16_training timm_regnet fail_accuracy
torchbench_float32_training timm_regnet eager_two_runs_differ

Versions

torch-xpu-ops: 7e3d00a
pytorch: 0d1d69fd25fdc096763bfe85f4d379e27ea1c9f8
device: ARC 24.04
driver: 24.31.30508.7

@mengfei25
Copy link
Contributor Author

Compared with 22.04

Category Model Ubuntu 24.04 Ubuntu 22.04
torchbench_amp_bf16_training densenet121 fail_accuracy pass
torchbench_bfloat16_training densenet121 fail_accuracy pass
torchbench_float32_inference hf_T5 pass eager_fail_to_run
torchbench_float32_inference vision_maskrcnn fail_accuracy pass
torchbench_float32_training basic_gnn_edgecnn pass eager_two_runs_differ

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant