Commit be9c3ac
committed
Adds mask and bias offset calculations to backward kernel
Introduces row offset computations for mask, bias, and bias gradient tensors in the backward pass computation function.
Enables proper memory addressing for attention mask and bias operations during gradient computation by calculating the appropriate stride-based offsets for batch, head, and spatial dimensions.1 parent 6d6ab5d commit be9c3ac
1 file changed
+6
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
107 | 107 | | |
108 | 108 | | |
109 | 109 | | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
110 | 116 | | |
111 | 117 | | |
112 | 118 | | |
| |||
0 commit comments