Commit 2fa51c1
authored
model-conversion : add token ids to prompt token output [no ci] (ggml-org#17863)
This commit adds the token ids to the printed prompt outputs.
The motivation for this is that is can be useful to see the actual token
ids alongside the token strings for debugging.1 parent 951520d commit 2fa51c1
1 file changed
+1
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
144 | 144 | | |
145 | 145 | | |
146 | 146 | | |
147 | | - | |
| 147 | + | |
148 | 148 | | |
149 | 149 | | |
150 | 150 | | |
| |||
0 commit comments