Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add optional flag_gems support #2491

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Add optional flag_gems support #2491

wants to merge 2 commits into from

Conversation

xuzhao9
Copy link
Contributor

@xuzhao9 xuzhao9 commented Oct 6, 2024

Import optional Triton kernels FlagGems: https://github.com/FlagOpen/FlagGems.
Support softmax and addmm operators.

Test plan:

$ python run_benchmark.py triton --op addmm --only flaggems,triton_addmm --num-inputs 2 --metrics latency,gbps,tflops
         (M, N, K)    flaggems-gbps    flaggems-latency    flaggems-tflops    triton_addmm-gbps    triton_addmm-latency    triton_addmm-tflops
------------------  ---------------  ------------------  -----------------  -------------------  ----------------------  ---------------------
(20120, 512, 1536)          220.794            0.473686            66.808               234.791                0.445449                71.043
(34579, 512, 1536)          224.696            0.79493             68.4186              231.003                0.773224                70.3393

@xuzhao9 xuzhao9 changed the title Install flag_gems and add an example backend to addmm. Add optional flag_gems support Oct 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants