You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
LLMs (Large Language Models) are increasingly used to power chatbots or other knowledge assistants. While these models are pre-trained on vast swaths of information, they are not trained on your own private data or knowledge base. To overcome this, you need to provide this data to the LLM (a process called context augmentation). This tutorial showcases a particular method of context augmentation called Retrieval-Augmented Generation (RAG), which indexes your data and attaches relevant data as context when users sends the LLM queries.
16
17
17
-
Follow this tutorial to deploy a RAG pipeline on Akamai’s LKE service using our latest GPU instances. Once deployed, you will have a web chatbot that can respond to queries using data from your own custom data source.
18
+
Follow this tutorial to deploy a RAG pipeline on Akamai’s LKE service using our latest GPU instances. Once deployed, you will have a web chatbot that can respond to queries using data from your own custom data source like shown in the screenshot below.
19
+
20
+
18
21
19
22
## Diagram
20
23
@@ -31,8 +34,8 @@ Follow this tutorial to deploy a RAG pipeline on Akamai’s LKE service using ou
31
34
32
35
-**Kubeflow:** This open-source software platform includes a suite of applications that are used for machine learning tasks. It is designed to be run on Kubernetes. While each application can be installed individually, this tutorial installs all default applications and makes specific use of the following:
33
36
-**KServe:** Serves machine learning models. This tutorial installs the Llama 3 LLM to KServe, which then serves it to other applications, such as the chatbot UI.
34
-
-**Kubeflow Pipeline:** Used to deploy pipelines, reusable machine learning workflows built using the Kubeflow Pipelines SDK. In this tutorial, a pipeline is used to run LlamaIndex to train the LLM with additional data.
35
-
-**Meta’s Llama 3 LLM:**We use Llama 3 as the LLM, along with the LlamaIndex tool to capture data from an external source and send embeddings to the Milvus database.
37
+
-**Kubeflow Pipeline:** Used to deploy pipelines, reusable machine learning workflows built using the Kubeflow Pipelines SDK. In this tutorial, a pipeline is used to run LlamaIndex to process the dataset and store embeddings.
38
+
-**Meta’s Llama 3 LLM:**The [meta-llama/Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) model is used as the LLM. You should review and agree to the licensing agreement before deploying.
36
39
-**Milvus:** Milvus is an open-source vector database and is used for generative AI workloads. This tutorial uses Milvus to store embeddings generated by LlamaIndex and make them available to queries sent to the Llama 3 LLM.
37
40
-**Open WebUI:** This is an self-hosted AI chatbot application that’s compatible with LLMs like Llama 3 and includes a built-in inference engine for RAG solutions. Users interact with this interface to query the LLM. This can be configured to send queries straight to Llama 3 or to first load data from Milvus and send that context along with the query.
38
41
@@ -54,11 +57,11 @@ The configuration instructions in this document are expected to not expose any s
54
57
It’s not part of the scope of this document to cover the setup required to secure this configuration for a production deployment.
55
58
{{< /note >}}
56
59
57
-
# Set up infrastructure
60
+
##Set up infrastructure
58
61
59
62
The first step is to provision the infrastructure needed for this tutorial and configure it with kubectl, so that you can manage it locally and install software through helm. As part of this process, we’ll also need to install the NVIDIA GPU operator at this step so that the NVIDIA cards within the GPU worker nodes can be used on Kubernetes.
60
63
61
-
1.**Provision an LKE cluster.** We recommend using at least two**RTX4000 Ada x2 Medium** GPU plans (plan ID: `g2-gpu-rtx4000a2-m`), though you can adjust this as needed. For reference, Kubeflow recommends 32 GB of RAM and 16 CPU cores. This tutorial has been tested using Kubernetes v1.31, though other versions should also work. To learn more about provisioning a cluster, see the [Create a cluster](https://techdocs.akamai.com/cloud-computing/docs/create-a-cluster) guide.
64
+
1.**Provision an LKE cluster.** We recommend using at least 3**RTX4000 Ada x1 Medium** GPU plans (plan ID: `g2-gpu-rtx4000a1-m`), though you can adjust this as needed. For reference, Kubeflow recommends 32 GB of RAM and 16 CPU cores for just their own application. This tutorial has been tested using Kubernetes v1.31, though other versions should also work. To learn more about provisioning a cluster, see the [Create a cluster](https://techdocs.akamai.com/cloud-computing/docs/create-a-cluster) guide.
62
65
63
66
{{< note noTitle=true >}}
64
67
GPU plans are available in a limited number of data centers. Review the [GPU product documentation](https://techdocs.akamai.com/cloud-computing/docs/gpu-compute-instances#availability) to learn more about availability.
@@ -77,7 +80,7 @@ The first step is to provision the infrastructure needed for this tutorial and c
77
80
You can confirm that the operator has been installed on your cluster by running reviewing your pods. You should see a number of pods in the `gpu-operator` namespace.
78
81
79
82
```command
80
-
kubectl get pods -A
83
+
kubectl get pods -n gpu-operator
81
84
```
82
85
83
86
### Deploy Kubeflow
@@ -114,6 +117,8 @@ Next, let’s deploy Kubeflow on the LKE cluster. These instructions deploy all
114
117
kubectl get pods -A
115
118
```
116
119
120
+
You may notice a status of `CrashLoopBackOff` on one or more pods. This can be caused to a temporary issue with a persistent volume attaching to a worker node and should be resolved within a minute or so.
121
+
117
122
### Install Llama3 LLM on KServe
118
123
119
124
After Kubeflow has been installed, we can now deploy the Llama 3 LLM to KServe. This tutorial uses HuggingFace (a platform that provides pre-trained AI models) to deploy Llama 3 to the LKE cluster. Specifically, these instructions use the [meta-llama/Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) model.
@@ -152,7 +157,7 @@ After Kubeflow has been installed, we can now deploy the Llama 3 LLM to KServe.
@@ -218,7 +223,7 @@ Kubeflow Pipeline pulls together the entire workflow for ingesting data from our
218
223
219
224
1. Download a zip archive from the specified URL.
220
225
1. Uses LlamaIndex to read the Markdown files within the archive.
221
-
1. Generate embeddings from the content of those files using the [BAAI/bge-large-en-v1.5](https://huggingface.co/BAAI/bge-large-en-v1.5) model.
226
+
1. Generate embeddings from the content of those files.
222
227
1. Store the embeddings within the Milvus database collection.
223
228
224
229
Keep this workflow in mind when going through the Kubeflow Pipeline set up steps in this section. If you require a different pipeline workflow, you will need to adjust the python file and Kubeflow Pipeline configuration discussed in this section.
@@ -240,7 +245,7 @@ This tutorial employs a Python script to create the YAML file used within Kubefl
240
245
pip install kfp
241
246
```
242
247
243
-
1. Use the following python script to generate a YAML file to use forthe Kubeflow Pipeline. This script configures the pipeline to download the Markdown data you wish to ingest, read the content using LlamaIndex, generate embeddings of the content using BAAI general embedding model, and store the embeddingsin the Milvus database. Replace values as needed before proceeding.
248
+
1. Use the following python script to generate a YAML file to use forthe Kubeflow Pipeline. This script configures the pipeline to download the Markdown data you wish to ingest, read the content using LlamaIndex, generate embeddings of the content, and store the embeddingsin the Milvus database. Replace values as needed before proceeding.
@@ -350,7 +355,7 @@ Despite the naming, these RAG pipeline files are not related to the Kubeflow pip
350
355
351
356
1. Create a new directory on your local machine and navigate to that directory.
352
357
353
-
1. Create a pipeline-requirements.txt file with the following contents:
358
+
1. Create a `pipeline-requirements.txt` file with the following contents:
354
359
355
360
```file {title="pipeline-requirements.txt"}
356
361
requests
@@ -361,7 +366,7 @@ Despite the naming, these RAG pipeline files are not related to the Kubeflow pip
361
366
llama-index-llms-openai-like
362
367
```
363
368
364
-
1. Create a rag-pipeline.py file with the following contents:
369
+
1. Create a file called `rag_pipeline.py`with the following contents. The filenames of both the `pipeline-requirements.txt` and `rag_pipeline.py` files should not be changed as they are referenced within the Open WebUI Pipeline configuration file.
365
370
366
371
```file {title="rag-pipeline.py"}
367
372
"""
@@ -388,7 +393,7 @@ Despite the naming, these RAG pipeline files are not related to the Kubeflow pip
@@ -581,8 +586,12 @@ Now that the chatbot has been configured, the final step is to access the chatbo
581
586
582
587
1. The first time you access this interface you are prompted to create an admin account. Do this now and thencontinue once you are successfully logged in using that account.
583
588
584
-
1. You are now presented with the chatbot interface. Within the dropdown menu, you should be able to selectfrom several models. Select one and ask it a question.
589
+
1. You should now be presented with the chatbot interface. Within the dropdown menu, you should be able to selectfrom several models. Select one and ask it a question.
590
+
591
+
- The **llama3** model uses information that was trained by other data sources (not your own custom data). If you ask this model a question, the data from your own dataset is not used.
592
+
593
+
585
594
586
-
- The **llama3** model will just use information that was trained by other data sources (not your own custom data). If you ask this model a question, the data from your own dataset will not be used.
595
+
- The **RAG Pipeline** model that you defined in a previous section does use data from your custom dataset. Ask it a question relevant to your data and the chatbot should respond with an answer that is informed by the custom dataset you configured.
587
596
588
-
- The **RAG Pipeline** model that you defined in a previous section does indeed use data from your custom dataset. Ask it a question relevant to your data and the chatbot should respond with an answer that is informed by the custom dataset you configured.
597
+
{{< note type="warning" title="This app is no longer available for deployment" >}}
20
-
ARK: Survival Evolved has been removed from the App Marketplace and can no longer be deployed. This guide has been retained for reference only. For information on how to deploy and set up ARK: Survival Evolved manually on a Compute Instance, see our [Creating a Dedicated ARK Server on Ubuntu](/docs/guides/create-an-ark-server-on-ubuntu) guide.
20
+
ARK: Survival Evolved has been removed from the App Marketplace and can no longer be deployed. This guide is retained for reference only. For information on how to deploy and set up ARK: Survival Evolved manually on a Compute Instance, see our [Creating a Dedicated ARK Server on Ubuntu](/docs/guides/create-an-ark-server-on-ubuntu) guide.
21
21
{{< /note >}}
22
22
23
23
[ARK: Survival Evolved](http://playark.com/ark-survival-evolved/) is a multiplayer action-survival game released in 2017. The game places you on a series of fictional islands inhabited by dinosaurs and other prehistoric animals. In ARK, the main objective is to survive. ARK is an ongoing battle where animals and other players have the ability to destroy you. To survive, you must build structures, farm resources, breed dinosaurs, and even set up trading hubs with neighboring tribes.
{{< note type="warning" title="This app is no longer available for deployment" >}}
13
+
Budibase has been removed from the App Marketplace and can no longer be deployed. This guide is retained for reference only.
14
+
{{< /note >}}
12
15
13
16
[Budibase](https://github.com/Budibase/budibase) is an open-source, low-code platform for building modern business applications. Build, design, and automate different types of applications, including admin panels, forms, internal tools, and client portals. Using Budibase helps developers avoid spending weeks building simple CRUD applications and, instead, allows them to complete many projects in significantly less time.
0 commit comments