Skip to content

Commit d1f7131

Browse files
windsonwindson
committed
clean up
1 parent 07867ab commit d1f7131

8 files changed

+15
-25
lines changed

‎llama-2-hf-tgi/llama-2-13b-chat-hf/1-deploy-llama-2-13b-chat-hf-tgi-sagemaker.ipynb‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@
3030
"\n",
3131
"### Hugging Face Account\n",
3232
"\n",
33-
"You need to have Hugging Face account. Sing Up here https://huggingface.co/join with your email if you do not already have account.\n",
33+
"You need to have Hugging Face account. Sign Up here https://huggingface.co/join with your email if you do not already have account.\n",
3434
"\n",
3535
"- For seamless access of the models avaialble on Hugging Face especially gated models such as Llama, for fine-tuning and inferencing purposes, you need to have Hugging Face Account to obtain read Access Token.\n",
3636
"- After signup, login to visit https://huggingface.co/settings/tokens to create read Access token.\n",
@@ -118,7 +118,7 @@
118118
"id": "74e72556-eb2f-4e61-a2d6-339d49c5892c",
119119
"metadata": {},
120120
"source": [
121-
"Obtain the latest Hugging Face LLM DLC is powered by Text Generation Inference (TGI) available on SageMaker. We will use this image to deploy `meta-llama/Llama-2-13b-chat-hf` model on SageMaker, "
121+
"Obtain the latest Hugging Face LLM DLC powered by Text Generation Inference (TGI) available on SageMaker. We will use this image to deploy `meta-llama/Llama-2-13b-chat-hf` model on SageMaker, "
122122
]
123123
},
124124
{
@@ -165,6 +165,8 @@
165165
"| Llama-2-70b-hf | meta-llama/Llama-2-70b-hf | ml.g5.48xlarge |\n",
166166
"| Llama-2-70b-chat | meta-llama/Llama-2-70b-chat-hf | ml.g5.48xlarge |\n",
167167
"\n",
168+
"Reference: [Llama 2 foundation models from Meta are now available in Amazon SageMaker JumpStart](https://aws.amazon.com/blogs/machine-learning/llama-2-foundation-models-from-meta-are-now-available-in-amazon-sagemaker-jumpstart/)\n",
169+
"\n",
168170
"We will proceed with deploying `meta-llama/Llama-2-13b-chat-hf` model on `ml.g5.12xlarge`. Also notice that the config for `SM_NUM_GPUS` is 4 for `meta-llama/Llama-2-13b-chat-hf` model."
169171
]
170172
},

‎llama-2-hf-tgi/llama-2-70b-chat-hf/1-deploy-llama-2-70b-chat-hf-tgi-sagemaker.ipynb‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@
3030
"\n",
3131
"### Hugging Face Account\n",
3232
"\n",
33-
"You need to have Hugging Face account. Sing Up here https://huggingface.co/join with your email if you do not already have account.\n",
33+
"You need to have Hugging Face account. Sign Up here https://huggingface.co/join with your email if you do not already have account.\n",
3434
"\n",
3535
"- For seamless access of the models avaialble on Hugging Face especially gated models such as Llama, for fine-tuning and inferencing purposes, you need to have Hugging Face Account to obtain read Access Token.\n",
3636
"- After signup, login to visit https://huggingface.co/settings/tokens to create read Access token.\n",
@@ -118,7 +118,7 @@
118118
"id": "74e72556-eb2f-4e61-a2d6-339d49c5892c",
119119
"metadata": {},
120120
"source": [
121-
"Obtain the latest Hugging Face LLM DLC is powered by Text Generation Inference (TGI) available on SageMaker. We will use this image to deploy `meta-llama/Llama-2-70b-chat-hf` model on SageMaker, "
121+
"Obtain the latest Hugging Face LLM DLC powered by Text Generation Inference (TGI) available on SageMaker. We will use this image to deploy `meta-llama/Llama-2-70b-chat-hf` model on SageMaker, "
122122
]
123123
},
124124
{
@@ -165,6 +165,8 @@
165165
"| Llama-2-70b-hf | meta-llama/Llama-2-70b-hf | ml.g5.48xlarge |\n",
166166
"| Llama-2-70b-chat | meta-llama/Llama-2-70b-chat-hf | ml.g5.48xlarge |\n",
167167
"\n",
168+
"Reference: [Llama 2 foundation models from Meta are now available in Amazon SageMaker JumpStart](https://aws.amazon.com/blogs/machine-learning/llama-2-foundation-models-from-meta-are-now-available-in-amazon-sagemaker-jumpstart/)\n",
169+
"\n",
168170
"We will proceed with deploying `meta-llama/Llama-2-70b-chat-hf` model on `ml.g5.48xlarge`. Also notice that the config for `SM_NUM_GPUS` is 8 for `meta-llama/Llama-2-70b-chat-hf` model.\n",
169171
"\n",
170172
"https://huggingface.co/docs/text-generation-inference/main/en/basic_tutorials/launcher#maxbatchtotaltokens"

‎llama-2-hf-tgi/llama-2-7b-chat-hf/1-deploy-llama-2-7b-chat-hf-tgi-sagemaker.ipynb‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@
3030
"\n",
3131
"### Hugging Face Account\n",
3232
"\n",
33-
"You need to have Hugging Face account. Sing Up here https://huggingface.co/join with your email if you do not already have account.\n",
33+
"You need to have Hugging Face account. Sign Up here https://huggingface.co/join with your email if you do not already have account.\n",
3434
"\n",
3535
"- For seamless access of the models avaialble on Hugging Face especially gated models such as Llama, for fine-tuning and inferencing purposes, you need to have Hugging Face Account to obtain read Access Token.\n",
3636
"- After signup, login to visit https://huggingface.co/settings/tokens to create read Access token.\n",
@@ -118,7 +118,7 @@
118118
"id": "74e72556-eb2f-4e61-a2d6-339d49c5892c",
119119
"metadata": {},
120120
"source": [
121-
"Obtain the latest Hugging Face LLM DLC is powered by Text Generation Inference (TGI) available on SageMaker. We will use this image to deploy `meta-llama/Llama-2-7b-chat-hf` model on SageMaker, "
121+
"Obtain the latest Hugging Face LLM DLC powered by Text Generation Inference (TGI) available on SageMaker. We will use this image to deploy `meta-llama/Llama-2-7b-chat-hf` model on SageMaker, "
122122
]
123123
},
124124
{
@@ -165,6 +165,8 @@
165165
"| Llama-2-70b-hf | meta-llama/Llama-2-70b-hf | ml.g5.48xlarge |\n",
166166
"| Llama-2-70b-chat | meta-llama/Llama-2-70b-chat-hf | ml.g5.48xlarge |\n",
167167
"\n",
168+
"Reference: [Llama 2 foundation models from Meta are now available in Amazon SageMaker JumpStart](https://aws.amazon.com/blogs/machine-learning/llama-2-foundation-models-from-meta-are-now-available-in-amazon-sagemaker-jumpstart/)\n",
169+
"\n",
168170
"We will proceed with deploying `meta-llama/Llama-2-7b-chat-hf` model on `ml.g5.2xlarge`"
169171
]
170172
},

‎llama-2-lmi/llama-2-13b-chat/1-deploy-llama-2-13b-chat-lmi-response-streaming.ipynb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@
3636
"## Prerequisite\n",
3737
"### Hugging Face Account\n",
3838
"\n",
39-
"You need to have Hugging Face account. Sing Up here https://huggingface.co/join with your email if you do not already have account.\n",
39+
"You need to have Hugging Face account. Sign Up here https://huggingface.co/join with your email if you do not already have account.\n",
4040
"\n",
4141
"- For seamless access of the models avaialble on Hugging Face especially gated models such as Llama, for fine-tuning and inferencing purposes, you need to have Hugging Face Account to obtain read Access Token.\n",
4242
"- After signup, [login](https://huggingface.co/login) to visit https://huggingface.co/settings/tokens to create read Access token.\n",

‎llama-2-lmi/llama-2-70b-chat/1-deploy-llama-2-70b-chat-lmi-response-streaming.ipynb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@
3636
"## Prerequisite\n",
3737
"### Hugging Face Account\n",
3838
"\n",
39-
"You need to have Hugging Face account. Sing Up here https://huggingface.co/join with your email if you do not already have account.\n",
39+
"You need to have Hugging Face account. Sign Up here https://huggingface.co/join with your email if you do not already have account.\n",
4040
"\n",
4141
"- For seamless access of the models avaialble on Hugging Face especially gated models such as Llama, for fine-tuning and inferencing purposes, you need to have Hugging Face Account to obtain read Access Token.\n",
4242
"- After signup, [login](https://huggingface.co/login) to visit https://huggingface.co/settings/tokens to create read Access token.\n",

‎llama-2-lmi/llama-2-70b-chat/2-inference-llama-2-70b-chat-lmi-response-streaming.ipynb‎

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -281,14 +281,6 @@
281281
"}"
282282
]
283283
},
284-
{
285-
"cell_type": "markdown",
286-
"id": "85464065-b85e-452a-83e8-4546b0115219",
287-
"metadata": {},
288-
"source": [
289-
"As we are interested in streaming response, the request payload must provide a key value pair with **\"stream\": True**"
290-
]
291-
},
292284
{
293285
"cell_type": "code",
294286
"execution_count": null,

‎llama-2-lmi/llama-2-7b-chat/1-deploy-llama-2-7b-lmi-response-streaming.ipynb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@
3636
"## Prerequisite\n",
3737
"### Hugging Face Account\n",
3838
"\n",
39-
"You need to have Hugging Face account. Sing Up here https://huggingface.co/join with your email if you do not already have account.\n",
39+
"You need to have Hugging Face account. Sign Up here https://huggingface.co/join with your email if you do not already have account.\n",
4040
"\n",
4141
"- For seamless access of the models avaialble on Hugging Face especially gated models such as Llama, for fine-tuning and inferencing purposes, you need to have Hugging Face Account to obtain read Access Token.\n",
4242
"- After signup, [login](https://huggingface.co/login) to visit https://huggingface.co/settings/tokens to create read Access token.\n",

‎llama-2-lmi/llama-2-7b-chat/2-inference-llama-2-7b-lmi-response-streaming.ipynb‎

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -292,14 +292,6 @@
292292
"}"
293293
]
294294
},
295-
{
296-
"cell_type": "markdown",
297-
"id": "85464065-b85e-452a-83e8-4546b0115219",
298-
"metadata": {},
299-
"source": [
300-
"As we are interested in streaming response, the request payload must provide a key value pair with **\"stream\": True**"
301-
]
302-
},
303295
{
304296
"cell_type": "code",
305297
"execution_count": null,

0 commit comments

Comments
 (0)