Skip to content

Commit 249eda4

Browse files
committed
add temperature_length
1 parent 08e0a4e commit 249eda4

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

convert_hf_to_gguf.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9988,6 +9988,8 @@ def set_vocab(self):
99889988
def set_gguf_parameters(self):
99899989
super().set_gguf_parameters()
99909990
MistralModel.set_mistral_config(self.gguf_writer, self.hparams)
9991+
yarn_params = self.hparams["yarn"]
9992+
self.gguf_writer.add_attn_temperature_length(yarn_params["original_max_position_embeddings"])
99919993

99929994
# TODO @ngxson : this should be in tensor_mapping, but I don't have time for now
99939995
# copied from https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/models/mistral_large_3.py

0 commit comments

Comments
 (0)