You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Recent mistral models inlcuding mistral 7b v0.3 instruct have consolidated.safetensors which have different weights key names compared to what LoRAx expects. Also there are keys like lm_head, embed_tokens, layernorm and postattention_layernorm that vllm finds difficult to deal with.
Are you able to implement an update where a user who has generated a lora safetensors file from mistral-finetune can simply load this directly as a lora adapter into LoRAx and it just works instead of having to first try to map the weights to another weights key name convention as well as figuring out how to deal with unfamiliar keys such as layernorm and postattention_layernorm.
Motivation
Mistral-Finetune will become widely used so users who have generated lora safetensors from this should be able to simply plug and play their lora adapters into LoRAx.
Your contribution
I am happy to provide a lora safetensors if needed to help you understand the problem better.
The text was updated successfully, but these errors were encountered:
Feature request
Recent mistral models inlcuding mistral 7b v0.3 instruct have consolidated.safetensors which have different weights key names compared to what LoRAx expects. Also there are keys like lm_head, embed_tokens, layernorm and postattention_layernorm that vllm finds difficult to deal with.
Are you able to implement an update where a user who has generated a lora safetensors file from mistral-finetune can simply load this directly as a lora adapter into LoRAx and it just works instead of having to first try to map the weights to another weights key name convention as well as figuring out how to deal with unfamiliar keys such as layernorm and postattention_layernorm.
Motivation
Mistral-Finetune will become widely used so users who have generated lora safetensors from this should be able to simply plug and play their lora adapters into LoRAx.
Your contribution
I am happy to provide a lora safetensors if needed to help you understand the problem better.
The text was updated successfully, but these errors were encountered: