Commit 5ee5acf
committed
Adds mask and bias tensor support to backward kernel
Introduces gMask, gBias, and gdBias tensor declarations to enable
attention masking and bias functionality in the backward pass.
Extends the kernel to handle masked attention computations and
bias gradient calculations for more flexible attention mechanisms.1 parent be9c3ac commit 5ee5acf
1 file changed
+15
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
143 | 143 | | |
144 | 144 | | |
145 | 145 | | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
146 | 161 | | |
147 | 162 | | |
148 | 163 | | |
| |||
0 commit comments