feat: PyTorch PT2 Model Generation #8547
Conversation
qa/common/gen_qa_models.py
Outdated
| # Use a different model name for the non-batching variant | ||
| model_name = tu.get_model_name( | ||
| "libtorch_nobatch" if max_batch == 0 else "libtorch", | ||
| "libtorch_nobatch" if max_batch == 0 else "libtorch2" if torch2 else "libtorch", |
There was a problem hiding this comment.
Please refactor and make the statement clear
There was a problem hiding this comment.
done. also, apologies for the force push, hadn't realized that anyone had reviewed the changes already. won't happen again. thanks.
| with open(f"{config_path}/config.pbtxt", "w") as f: | ||
| f.write( | ||
| """ | ||
| backend: "pytorch" |
There was a problem hiding this comment.
trying to keep them as the same backend.
There was a problem hiding this comment.
I see. They differs by "platform"?
There was a problem hiding this comment.
Correct. That's the plan. Ideally we don't have competing "pytorch" and "pytorch_inductor" backends.
There was a problem hiding this comment.
renamed the platform to torch_aoti to better reflect what it is and does.
whoisj
force-pushed
the
jwyman/pt2/gen-models
branch
from
55ebafa to
558c361
Compare
This change adds the generation of PT2 format serialized PyTorch model files to the QA model generation scripts.
whoisj
force-pushed
the
jwyman/pt2/gen-models
branch
from
558c361 to
78f755f
Compare
| python3 $TRITON_MDLS_SRC_DIR/gen_qa_models.py --torch-aoti --variable --models_dir=$TRITON_MDLS_QA_VARIABLE_MODEL | ||
| chmod -R 777 $TRITON_MDLS_QA_VARIABLE_MODEL | ||
| python3 $TRITON_MDLS_SRC_DIR/gen_qa_identity_models.py --libtorch --models_dir=$TRITON_MDLS_QA_IDENTITY_MODEL | ||
| python3 $TRITON_MDLS_SRC_DIR/gen_qa_identity_models.py --libtorch --models_dir=$VOLUME_IDENTITYDESTDIR |
There was a problem hiding this comment.
Given variable is not declared, may I suggest suggest to preserve the existing one, otherwise following issue will happen.
+ chmod -R 777 /mnt/20251208_1446/25.11/qa_variable_model_repository
+ python3 /mnt/20251208_1446/gen_srcdir/gen_qa_identity_models.py --libtorch --models_dir=
+ chmod -R 777 /mnt/20251208_1446/25.11/qa_identity_model_repository
+ python3 /mnt/20251208_1446/gen_srcdir/gen_qa_reshape_models.py --libtorch --variable --models_dir=/mnt/20251208_1446/25.11/qa_reshape_model_repository
This change adds the generation of PT2 format serialized PyTorch model files to the QA model generation scripts.
TRI-172