Skip to content

Commit 25e46d8

Browse files
committed
Dynamic gen: Fix return_logits=True when using CFG
1 parent 1179b8a commit 25e46d8

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

exllamav2/generator/dynamic.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1932,7 +1932,7 @@ def emit(
19321932
self.held_k_tokens.append(next_k_tokens)
19331933
self.held_k_probs.append(next_k_probs)
19341934
if self.return_logits:
1935-
self.held_logits.append(logits)
1935+
self.held_logits.append(logits[:1, :, :])
19361936

19371937
# Stop if we reach max_new_tokens
19381938

0 commit comments

Comments
 (0)