Skip to content

Commit 5c8dbda

Browse files
committed
Remove compilation config 3 for multi-node launches as they seem to break the launch
1 parent 34ef534 commit 5c8dbda

File tree

1 file changed

+0
-7
lines changed

1 file changed

+0
-7
lines changed

vec_inf/config/models.yaml

Lines changed: 0 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,6 @@ models:
1414
--tensor-parallel-size: 4
1515
--max-model-len: 8192
1616
--max-num-seqs: 256
17-
--compilation-config: 3
1817
c4ai-command-r-plus-08-2024:
1918
model_family: c4ai-command-r
2019
model_variant: plus-08-2024
@@ -30,7 +29,6 @@ models:
3029
--tensor-parallel-size: 4
3130
--max-model-len: 65536
3231
--max-num-seqs: 256
33-
--compilation-config: 3
3432
c4ai-command-r-08-2024:
3533
model_family: c4ai-command-r
3634
model_variant: 08-2024
@@ -494,7 +492,6 @@ models:
494492
--tensor-parallel-size: 4
495493
--max-model-len: 16384
496494
--max-num-seqs: 256
497-
--compilation-config: 3
498495
Mistral-7B-Instruct-v0.1:
499496
model_family: Mistral
500497
model_variant: 7B-Instruct-v0.1
@@ -566,7 +563,6 @@ models:
566563
--tensor-parallel-size: 4
567564
--max-model-len: 32768
568565
--max-num-seqs: 256
569-
--compilation-config: 3
570566
Mistral-Large-Instruct-2411:
571567
model_family: Mistral
572568
model_variant: Large-Instruct-2411
@@ -582,7 +578,6 @@ models:
582578
--tensor-parallel-size: 4
583579
--max-model-len: 32768
584580
--max-num-seqs: 256
585-
--compilation-config: 3
586581
Mixtral-8x7B-Instruct-v0.1:
587582
model_family: Mixtral
588583
model_variant: 8x7B-Instruct-v0.1
@@ -613,7 +608,6 @@ models:
613608
--tensor-parallel-size: 4
614609
--max-model-len: 65536
615610
--max-num-seqs: 256
616-
--compilation-config: 3
617611
Mixtral-8x22B-Instruct-v0.1:
618612
model_family: Mixtral
619613
model_variant: 8x22B-Instruct-v0.1
@@ -629,7 +623,6 @@ models:
629623
--tensor-parallel-size: 4
630624
--max-model-len: 65536
631625
--max-num-seqs: 256
632-
--compilation-config: 3
633626
Phi-3-medium-128k-instruct:
634627
model_family: Phi-3
635628
model_variant: medium-128k-instruct

0 commit comments

Comments
 (0)