Skip to content

Commit 56e05cc

Browse files
metascroyfacebook-github-bot
authored andcommitted
Expose static llama in OSS (#16184)
Summary: This exposes a static llama model for CoreML. We want to unify development behind one static model so that we do not need to apply fixes in multiple places (e.g., iOS 26 fixes). Reviewed By: billmguo Differential Revision: D88875745
1 parent c3a53f3 commit 56e05cc

File tree

4 files changed

+868
-0
lines changed

4 files changed

+868
-0
lines changed

.ci/scripts/test_ane_static_llama.sh

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,11 @@ pushd $EXECUTORCH_ROOT/examples/apple/coreml/llama
2828
# Download stories llama110m artifacts
2929
download_stories_model_artifacts
3030

31+
# Test static ANE llama model
32+
python export_static_llm_coreml.py --checkpoint stories110M.pt --params params.json --output model.pte
33+
python run_static_llm.py --model static_llm_coreml_model.pte --params $HOME/models/llama1b/params.json --tokenizer tokenizer.model --prompt "Once upon a time," --lookahead
34+
35+
# Test export of deprecated model
3136
python export.py -n model.pte -p params.json -c stories110M.pt --seq_length 32 --max_seq_length 64 --dtype fp16 --coreml-quantize c4w --embedding-quantize 4,32
3237

3338
popd

0 commit comments

Comments
 (0)