Skip to content

Commit d5c1206

Browse files
committed
Increase ropeWithCacheWorker local work group size to 512 in FP16 FFN layers to optimize worker grid configuration.
1 parent 577b6b1 commit d5c1206

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/main/java/org/beehive/gpullama3/tornadovm/layers/type/fp16/LlamaFP16FFNLayers.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ public GridScheduler updateGridScheduler(GridScheduler tornadoForwardScheduler)
4444
int fusedQKVRows = config.dim() + 2 * config.kvDim();
4545
int fusedQKVGlobal = fusedQKVRows * LOCAL_WORK_GROUP_SIZE_ALLOC;
4646
WorkerGrid fusedQKVWorker = WorkerGridFactory.genericWorker(fusedQKVGlobal, LOCAL_WORK_GROUP_SIZE_ALLOC);
47-
WorkerGrid ropeWithCacheWorker = WorkerGridFactory.genericWorker(config.dim() / 2, 128);
47+
WorkerGrid ropeWithCacheWorker = WorkerGridFactory.genericWorker(config.dim() / 2, 512);
4848

4949
// Map workers to tasks
5050
for (int i = 0; i < config.numberOfLayers(); i++) {

0 commit comments

Comments
 (0)