Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

📖 [Story] Optimize the launch overhead of TRT engine and pytorch kernels #3274

Open
2 tasks done
keehyuna opened this issue Nov 4, 2024 · 0 comments
Open
2 tasks done

📖 [Story] Optimize the launch overhead of TRT engine and pytorch kernels #3274

keehyuna opened this issue Nov 4, 2024 · 0 comments
Assignees
@keehyuna
Labels
Story Issues proposing a new Story

Comments

@keehyuna
Copy link
Collaborator

keehyuna commented Nov 4, 2024 •
edited
Loading

TL;DR

Runtime optimization in torch-tensorrt is crucial for maximizing model performance in real-world applications.
This story tracks the effort to improve runtime performance.

Goal(s)

  • Understand the overhead in cpp/python runtime module and improve the inference performance
  • Ensure no or minimized impact on accuracy and resource with optimization

Tasks

Tasks
Preview Give feedback
  1. feat: Runtime output buffer optimization #3276
    cla signed component: api [Python] component: core component: dynamo component: runtime component: tests documentation
    keehyuna
  2. ✨[Feature] Performance optimization of PyTorch + TRT subgraphs #3277
    feature request
    keehyuna

Additional context

Tasks
Preview Give feedback
No tasks being tracked yet.
@keehyuna keehyuna added the Story Issues proposing a new Story label Nov 4, 2024
@keehyuna keehyuna self-assigned this Nov 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@keehyuna keehyuna

Labels
Story Issues proposing a new Story
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@keehyuna