Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement quantized model inference for generate_v2 #1814

Open
Tracked by #1827
SalmanMohammadi opened this issue Oct 11, 2024 · 0 comments · May be fixed by #1866
Open
Tracked by #1827

Implement quantized model inference for generate_v2 #1814

SalmanMohammadi opened this issue Oct 11, 2024 · 0 comments · May be fixed by #1866
Assignees
@joecummings
Labels
enhancement New feature or request high-priority inference Anything related to our inference capabilities

Comments

@SalmanMohammadi
Copy link
Collaborator

SalmanMohammadi commented Oct 11, 2024 •
edited
Loading

We'll probably also need #1782.
@SalmanMohammadi SalmanMohammadi added enhancement New feature or request inference Anything related to our inference capabilities high-priority labels Oct 11, 2024
@SalmanMohammadi SalmanMohammadi mentioned this issue Oct 14, 2024
Open
@joecummings joecummings mentioned this issue Oct 15, 2024
Open
33 tasks
@joecummings joecummings self-assigned this Oct 15, 2024
@joecummings joecummings linked a pull request Oct 18, 2024 that will close this issue
[WIP] PTQ for generate_v2 #1866
Draft
13 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@joecummings joecummings

Labels
enhancement New feature or request high-priority inference Anything related to our inference capabilities
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[WIP] PTQ for generate_v2 joecummings/torchtune
2 participants
@joecummings @SalmanMohammadi