You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The **{lcs-name} ({lcs-short})** and Llama Stack deploy together as sidecar containers to augment {rhdh-short} functionality.
6
+
The {lcs-name} ({lcs-short}) and Llama Stack deploy together as sidecar containers to augment {rhdh-short} functionality.
7
7
8
-
{lcs-short} acts as an intermediary service layer for interfacing with Large Language Model (LLM) providers. {lcs-short} handles LLM provider setup, authentication, and includes key functionalities such as question validation, user feedback collection, and Retrieval Augmented Generation (RAG).
8
+
{lcs-short} serves as the Llama Stack service intermediary, managing configurations for key components. These components include the Large Language Model (LLM), inference providers, tool runtime providers, safety providers, and Retrieval Augmented Generation (RAG) settings.
9
9
10
-
Llama Stack provides the model server functionality that {lcs-short} uses to process requests. This service requires a Kubernetes Secret to securely store environment variables for the chosen LLM provider.
10
+
* **{lcs-short}** manages authentication, user feedback collection, MCP server configuration, and caching.
11
11
12
-
The {ls-brand-name} plugin within {rhdh-short} communicates with the {lcs-short} sidecar to send prompts and receives responses from the configured LLM service. The {lcs-short} sidecar centralizes the LLM interaction logic and configuration alongside your {rhdh-short} instance.
12
+
* Llama Stack provides the inference functionality that {lcs-short} uses to process requests. The service requires a **Kubernetes Secret** to securely store environment variables for the chosen LLM provider.
13
+
14
+
* The {ls-brand-name} plugin in {rhdh-short} sends prompts and receives LLM responses through the {lcs-short} sidecar. {lcs-short} then uses the Llama Stack sidecar service to perform inference and MCP tool calling.
13
15
14
16
[NOTE]
15
17
====
16
-
{ls-brand-name} is a Developer Preview release. You must manually deploy the {lcs-name} and Llama Stack sidecar containers, and then install the {ls-brand-name} plugin on your {rhdh-short} instance.
18
+
{ls-brand-name} is a Developer Preview release. You must manually deploy the {lcs-name} and Llama Stack sidecar containers, and install the {ls-brand-name} plugin on your {rhdh-short} instance.
Copy file name to clipboardExpand all lines: modules/developer-lightspeed/con-rag-embeddings.adoc
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,6 +3,6 @@
3
3
[id="con-rag-embeddings_{context}"]
4
4
= Retrieval Augmented Generation embeddings
5
5
6
-
The {product} documentation set has been added as a Retrieval-Augmented Generation (RAG) data source.
6
+
The {product} documentation set serves as the Retrieval-Augmented Generation (RAG) data source.
7
7
8
-
The {lcs-name} ({lcs-short}) sidecar handles the RAG process, using specialized Llama Stack components to manage the documentation data and generate vector embeddings.
8
+
RAG initialization occurs through an init container, which copies the RAG data to a shared volume. The Llama Stack sidecar then mounts this volume to access the data. The Llama Stack service uses the resulting RAG embeddings in the vector database as a tool. This tool allows the service to provide references to production documentation during the inference process.
Copy file name to clipboardExpand all lines: modules/developer-lightspeed/con-supported-architecture.adoc
+2-7Lines changed: 2 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,14 +3,9 @@
3
3
[id="con-supported-architecture_{context}"]
4
4
= Supported architecture for {ls-brand-name}
5
5
6
-
{ls-short} is available as a plugin on all platforms that host {product-very-short}. It requires the use of {lcs-name}, which runs as a sidecar container. {lcs-short} acts as the intermediary layer for interfacing with Large Language Model (LLM) providers.
6
+
{ls-short} is available as a plugin on all platforms that host {product-very-short}. It requires two sidecar containers: the {lcs-name} and the Llama Stack service.
7
7
8
-
{ls-short} handles functionalities like question validation, feedback, and Retrieval-Augmented Generation (RAG) by leveraging components often built with Llama Stack.
9
-
10
-
[NOTE]
11
-
====
12
-
Currently, the provided {lcs-short} image is built for x86 platforms. To use other platforms (for example, arm64), you must ensure that you enable emulation.
13
-
====
8
+
The {lcs-short} container acts as the intermediary layer, which interfaces with and manages the Llama Stack service.
14
9
15
10
.Additional resources
16
11
* link:https://access.redhat.com/support/policy/updates/developerhub[{product} Life Cycle and supported platforms]
Copy file name to clipboardExpand all lines: modules/developer-lightspeed/proc-configuring-mcp-components.adoc
+18-9Lines changed: 18 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,11 +3,12 @@
3
3
[id="proc-configure-mcp-components_{context}"]
4
4
= Configuring the Model Context Protocol (MCP)
5
5
6
-
You can configure the Model Context Protocol (MCP) to leverage {ls-short}. With MCP configuration, the deployed {lcs-short} and Llama Stack server utilize external tools and retrieve real-time data. This allows the virtual assistant to execute complex actions and incorporate current operational context into its responses. You must synchronize settings across three components: the Llama Stack tool definition, the {lcs-short} server endpoint definition, and the {ls-brand-name} Plugin authorization token.
6
+
To leverage Model Context Protocol (MCP) servers, you must configure them within the Llama Stack service. Configuring the MCP in {lcs-short} enables the deployed {lcs-short} and Llama Stack to use external tools and retrieve real-time data. This integration allows the virtual assistant to execute complex actions and incorporate current operational context into its responses.
7
+
8
+
Configuration requires synchronizing settings across three components: the Llama Stack tool definition, the {lcs-short} MCP server endpoint definition, and the {ls-brand-name} plugin MCP authorization token.
7
9
8
10
.Procedure
9
-
. Configure the {llama-name} tool definition.
10
-
Define the MCP provider in the `tool_runtime` section of the Llama Stack configuration file (`run.yaml`).
11
+
. Configure the Llama Stack tool definition by defining the MCP provider in the `tool_runtime` section of the Llama Stack configuration file (`run.yaml`), as shown in the following code:
11
12
+
12
13
[source,yaml]
13
14
----
@@ -18,24 +19,32 @@ providers:
18
19
config: {}
19
20
----
20
21
21
-
. Configure the {lcs-short} server endpoint.
22
-
In the {lcs-short} configuration file (`lightspeed-stack.yaml`), define the MCP server endpoints. The `provider_id` must** match the Llama Stack definition (`model-context-protocol`).
23
-
22
+
. Configure the {lcs-short} MCP server endpoint by defining the MCP server endpoints in the {lcs-short} configuration file (`lightspeed-stack.yaml`), as shown in the following code:
`provider_id` must match the Llama Stack definition (`model-context-protocol`).
35
+
====
31
36
32
-
. Configure the {ls-brand-name} plugin authorization token.
33
-
In the {ls-brand-name} plugin configuration file (`lightspeed-app-config.yaml`), specify the MCP servers and provide the token for authorization. This token is used when the plugin makes requests to the {lcs-short} `/v1/streaming_query` endpoint.
37
+
. Configure the {ls-brand-name} plugin authorization token by specifying the MCP servers and providing the token for authentication in the {ls-brand-name} plugin configuration file (`lightspeed-app-config.yaml`), as shown in the following code:
34
38
+
35
39
[source,yaml]
36
40
----
37
41
lightspeed:
38
42
mcpServers:
39
43
- name: mcp::backstage
40
44
token: ${MCP_TOKEN}
41
-
----
45
+
----
46
+
+
47
+
[IMPORTANT]
48
+
====
49
+
`name` must match the {lcs-name} definition (`mcp::backstage`).
Copy file name to clipboardExpand all lines: modules/developer-lightspeed/proc-gathering-feedback.adoc
+10-4Lines changed: 10 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,11 +3,15 @@
3
3
[id="proc-gathering-feedback_{context}"]
4
4
= Gathering feedback in {ls-short}
5
5
6
-
Feedback collection is an optional feature configured on the {lcs-short}. This feature gathers user feedback by providing thumbs-up/down ratings and text comments directly from the chat window. {lcs-short} gathers the feedback, along with the user's query and the response of the model, and stores it as a JSON file within the local file system of the Pod for later collection and analysis by the platform administrator. This can be useful for assessing model performance and improving your users' experience. The collected feedback is stored in the cluster where {product-very-short} and {lcs-short} are deployed, and as such, is only accessible by the platform administrators for that cluster. For users that intend to have their data removed, they must request their respective platform administrator to perform that action as {company-name} does not collect (or have access to) any of this data.
6
+
Feedback collection is an optional feature configured on the {lcs-short}. This feature gathers user feedback by providing thumbs-up/down ratings and text comments directly from the chat window.
7
+
8
+
{lcs-short} collects the feedback, the user's query, and the response of the model, storing the data as a JSON file on the local file system of the Pod. A platform administrator must later collect and analyze this data to assess model performance and improve the user experience.
9
+
10
+
The collected data resides in the cluster where {product-very-short} and {lcs-short} are deployed, making it accessible only to platform administrators for that cluster. For data removal, users must request this action from their platform administrator, as {company-name} neither collects nor accesses this data.
7
11
8
12
.Procedure
9
13
10
-
* To enable or disable feedback, in your {lcs-short} configuration file, add the following settings:
14
+
. To enable or disable feedback collection, in the {lcs-short} configuration file (`lightspeed-stack.yaml`), add the following settings:
0 commit comments