Commit edf840c
committed
Adds mask and bias memory copy operations
Introduces dedicated global-to-shared memory copy operations for mask and bias tensors in the backward kernel computation function.
Enables proper handling of attention masks and bias terms during gradient computation by creating separate thread slices for these operations.1 parent 18db4a1 commit edf840c
1 file changed
+4
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
274 | 274 | | |
275 | 275 | | |
276 | 276 | | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
277 | 281 | | |
278 | 282 | | |
279 | 283 | | |
| |||
0 commit comments