Commit af12e80
committed
Fixes softmax LSE calculation by removing scale factor
Removes the multiplication by softmax_scale from the log-sum-exp calculation
when sum is valid, keeping only the row maximum and log sum components.
This corrects the mathematical formula to properly compute the LSE value
without the unnecessary scaling factor that was affecting numerical accuracy.1 parent 9a408be commit af12e80
1 file changed
+3
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
209 | 209 | | |
210 | 210 | | |
211 | 211 | | |
212 | | - | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
213 | 215 | | |
214 | 216 | | |
215 | 217 | | |
| |||
0 commit comments