Skip to content

Commit fb4e91b

Browse files
author
Dinesh Mane
committed
Initial Commit
1 parent 9d143cc commit fb4e91b

File tree

14 files changed

+574
-6
lines changed

14 files changed

+574
-6
lines changed

‎README.md‎

Lines changed: 149 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,160 @@
1-
## My Project
1+
# AWS Bedrock Langchain Python CDK Stack
22

3-
TODO: Fill this README out!
3+
## Overview
4+
Optimize AWS Lambda functions with Boto3 by adding the latest packages and creating Lambda layers using `aws-cdk.aws-lambda-python-alpha`. Avoid common errors, like the numpy module issue, by following the guide. Use provided code and insights to enhance performance across various development environments.
45

5-
Be sure to:
66

7-
* Change the title in this README
8-
* Edit your repository description on GitHub
7+
This AWS Cloud Development Kit (CDK) stack, named `AwsBedrockLangchainPythonCdkStack`, is designed to deploy AWS Lambda functions along with the necessary AWS Identity and Access Management (IAM) roles and policies. The stack includes the following components:
98

9+
1. **IAM Roles and Policies:**
10+
- A Lambda execution role (`LambdaRole`) is created with permissions to access S3 buckets and the Bedrock service. It is associated with the `AWSLambdaBasicExecutionRole` managed policy and a custom policy (`bedrock_policy`) allowing specific Bedrock actions.
11+
12+
2. **Lambda Layers:**
13+
- Two Lambda layers, `boto3_lambda_layer` and `langchain_lambda_layer`, are defined with specific configurations for architecture and runtime. These layers contain Python dependencies for the Lambda functions.
14+
15+
3. **Lambda Functions:**
16+
- The stack deploys two Lambda functions, `boto3_bedrock_example_lambda` and `langchain_bedrock_example_lambda`, each with its own code, runtime settings, and associated layers.
17+
18+
4. **CDK NAG Suppressions:**
19+
- CDK NAG (Not a Good Idea) suppressions are applied to address specific warnings related to IAM policies. These suppressions ensure that the Lambda execution policy and associated resources comply with the intended access patterns.
20+
21+
---
22+
23+
### Installing the Latest Boto3 Package
24+
25+
To ensure you have the latest version of the Boto3 package, follow these steps:
26+
27+
1. **Create a Lambda Layer:**
28+
29+
Use the `aws-cdk.aws-lambda-python-alpha` package to define a Lambda layer containing the latest Boto3 package.
30+
31+
```python
32+
boto3_lambda_layer = _alambda.PythonLayerVersion(self,
33+
'boto3-lambda-layer',
34+
entry = './aws_bedrock_langchain_python_cdk/lambda/layer/boto3_latest/',
35+
compatible_architectures=[_lambda.Architecture.ARM_64],
36+
compatible_runtimes=[_lambda.Runtime.PYTHON_3_11],
37+
)
38+
```
39+
40+
2. **Attach the Layer to Your Lambda Function:**
41+
42+
Once the layer is created, attach it to your Lambda function to ensure access to the latest Boto3 package and its bedrock API calls.
43+
44+
```python
45+
boto3_bedrock_example_lambda = _lambda.Function(
46+
self,
47+
"boto3_bedrock_example_lambda",
48+
handler="index.lambda_handler",
49+
code=_lambda.Code.from_asset("./aws_bedrock_langchain_python_cdk/lambda/code/boto3_example/"),
50+
runtime=_lambda.Runtime.PYTHON_3_11,
51+
architecture=_lambda.Architecture.ARM_64,
52+
role=lambda_role,
53+
layers=[
54+
boto3_lambda_layer
55+
],
56+
timeout=Duration.seconds(300),
57+
memory_size=1024,
58+
)
59+
```
60+
61+
### Langchain Package Integration
62+
63+
Similarly, optimize your Lambda functions by installing the Langchain package. Follow the same process of creating a Lambda layer as with Boto3, replacing "boto3" in the `requirements.txt` file with "langchain," and then attach it to your function to leverage Langchain's capabilities.
64+
65+
1. **Create a Lambda Layer:**
66+
67+
```python
68+
langchain_lambda_layer = _alambda.PythonLayerVersion(self,
69+
'langchain-lambda-layer',
70+
entry = './aws_bedrock_langchain_python_cdk/lambda/layer/langchain_latest/',
71+
compatible_architectures=[_lambda.Architecture.ARM_64],
72+
compatible_runtimes=[_lambda.Runtime.PYTHON_3_11],
73+
)
74+
```
75+
76+
2. **Attach the Layer to Your Lambda Function:**
77+
78+
```python
79+
langchain_bedrock_example_lambda = _lambda.Function(
80+
self,
81+
"langchain-bedrock-example-lambda",
82+
handler="index.lambda_handler",
83+
code=_lambda.Code.from_asset("./aws_bedrock_langchain_python_cdk/lambda/code/langchain_example/"),
84+
runtime=_lambda.Runtime.PYTHON_3_11,
85+
architecture=_lambda.Architecture.ARM_64,
86+
role=lambda_role,
87+
layers=[
88+
boto3_lambda_layer,
89+
langchain_lambda_layer
90+
],
91+
timeout=Duration.seconds(300),
92+
memory_size=1024,
93+
)
94+
```
95+
96+
### Avoiding Common Pitfalls
97+
98+
When creating Lambda layers, be mindful of specifying the compatible architecture, either ARM_64 or X86_64. Failure to define this may result in container image errors, such as the numpy module error. This error can manifest as:
99+
100+
```json
101+
{
102+
"errorMessage": "Unable to import module 'index': Error importing numpy: you should not try to import numpy from its source directory; please exit the numpy source tree, and relaunch your python interpreter from there.",
103+
"errorType": "Runtime.ImportModuleError",
104+
"requestId": "977004a7-f205-481a-8b53-7a4b5bf48d2b",
105+
"stackTrace": []
106+
}
107+
```
108+
109+
To prevent this issue, **explicitly mention** the compatible architectures during the layer creation process.
110+
111+
112+
## Useful commands
113+
114+
To manually create a virtualenv on MacOS and Linux:
115+
116+
```
117+
$ python3 -m venv .venv
118+
```
119+
120+
After the init process completes and the virtualenv is created, you can use the following
121+
step to activate your virtualenv.
122+
123+
```
124+
$ source .venv/bin/activate
125+
```
126+
127+
If you are a Windows platform, you would activate the virtualenv like this:
128+
129+
```
130+
% .venv\Scripts\activate.bat
131+
```
132+
133+
Once the virtualenv is activated, you can install the required dependencies.
134+
135+
```
136+
$ pip install -r requirements.txt
137+
```
138+
139+
At this point you can now synthesize the CloudFormation template for this code.
140+
141+
```
142+
$ cdk synth
143+
```
144+
145+
To add additional dependencies, for example other CDK libraries, just add
146+
them to your `setup.py` file and rerun the `pip install -r requirements.txt`
147+
command.
148+
149+
* `cdk ls` list all stacks in the app
150+
* `cdk synth` emits the synthesized CloudFormation template
151+
* `cdk deploy` deploy this stack to your default AWS account/region
152+
* `cdk diff` compare deployed stack with current state
153+
* `cdk docs` open CDK documentation
10154
## Security
11155

12156
See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.
13157

14158
## License
15159

16160
This library is licensed under the MIT-0 License. See the LICENSE file.
17-

‎app.py‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
#!/usr/bin/env python3
2+
import cdk_nag
3+
from aws_cdk import Aspects
4+
import aws_cdk as cdk
5+
from aws_bedrock_langchain_python_cdk.aws_bedrock_langchain_python_cdk_stack import AwsBedrockLangchainPythonCdkStack
6+
7+
app = cdk.App()
8+
aws_bedrock_langchain_stack = AwsBedrockLangchainPythonCdkStack(app, "AwsBedrockLangchainPythonCdkStack",)
9+
10+
Aspects.of(app).add(cdk_nag.AwsSolutionsChecks(reports=True, verbose=True))
11+
12+
app.synth()

‎aws_bedrock_langchain_python_cdk/aws_bedrock_langchain_python_cdk_stack.py‎

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
from aws_cdk import (
2+
Stack,
3+
Duration,
4+
aws_iam as iam,
5+
aws_lambda as _lambda,
6+
aws_lambda_python_alpha as _alambda,
7+
aws_iam as iam,
8+
Stack,
9+
Duration
10+
)
11+
from constructs import Construct
12+
from cdk_nag import NagSuppressions
13+
14+
class AwsBedrockLangchainPythonCdkStack(Stack):
15+
16+
def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
17+
super().__init__(scope, construct_id, **kwargs)
18+
19+
bedrock_policy = iam.PolicyStatement(
20+
effect= iam.Effect.ALLOW,
21+
actions= [
22+
"bedrock:*",
23+
],
24+
resources= ["*"]
25+
)
26+
27+
lambda_role = iam.Role(
28+
self,
29+
"LambdaRole",
30+
assumed_by=iam.ServicePrincipal('lambda.amazonaws.com'),
31+
description="Role to access s3 buckets and Bedrock service by lambda",
32+
managed_policies=[iam.ManagedPolicy.from_aws_managed_policy_name(
33+
'service-role/AWSLambdaBasicExecutionRole'),
34+
]
35+
)
36+
lambda_role.add_to_principal_policy(bedrock_policy)
37+
38+
boto3_lambda_layer = _alambda.PythonLayerVersion(self,
39+
'boto3-lambda-layer',
40+
entry = './aws_bedrock_langchain_python_cdk/lambda/layer/boto3_latest/',
41+
compatible_architectures=[_lambda.Architecture.ARM_64],
42+
compatible_runtimes=[_lambda.Runtime.PYTHON_3_11],
43+
)
44+
45+
langchain_lambda_layer = _alambda.PythonLayerVersion(self,
46+
'langchain-lambda-layer',
47+
entry = './aws_bedrock_langchain_python_cdk/lambda/layer/langchain_latest/',
48+
compatible_architectures=[_lambda.Architecture.ARM_64],
49+
compatible_runtimes=[_lambda.Runtime.PYTHON_3_11 ],
50+
)
51+
52+
boto3_bedrock_example_lambda = _lambda.Function(
53+
self,
54+
"boto3_bedrock_example_lambda",
55+
handler="index.lambda_handler",
56+
code=_lambda.Code.from_asset("./aws_bedrock_langchain_python_cdk/lambda/code/boto3_example/"),
57+
runtime=_lambda.Runtime.PYTHON_3_11,
58+
architecture=_lambda.Architecture.ARM_64,
59+
role=lambda_role,
60+
layers=[
61+
boto3_lambda_layer
62+
],
63+
timeout=Duration.seconds(300),
64+
memory_size=1024,
65+
)
66+
67+
langchain_bedrock_example_lambda = _lambda.Function(
68+
self,
69+
"langchain-bedrock-example-lambda",
70+
handler="index.lambda_handler",
71+
code=_lambda.Code.from_asset("./aws_bedrock_langchain_python_cdk/lambda/code/langchain_example/"),
72+
runtime=_lambda.Runtime.PYTHON_3_11,
73+
architecture=_lambda.Architecture.ARM_64,
74+
role=lambda_role,
75+
layers=[
76+
boto3_lambda_layer,
77+
langchain_lambda_layer
78+
],
79+
timeout=Duration.seconds(300),
80+
memory_size=1024,
81+
)
82+
83+
# CDK NAG suppression
84+
NagSuppressions.add_stack_suppressions(self, [
85+
{
86+
"id": 'AwsSolutions-IAM4',
87+
"reason": 'Lambda execution policy for custom resources created by higher level CDK constructs',
88+
"appliesTo": [
89+
'Policy::arn:<AWS::Partition>:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole',
90+
],
91+
}])
92+
93+
NagSuppressions.add_resource_suppressions(lambda_role,
94+
suppressions=[{
95+
"id": "AwsSolutions-IAM5",
96+
"reason": "Lambda needs * access to all objects inside S3 bucket. And Bedrock all actions are allowed",
97+
}
98+
],
99+
apply_to_children=True)

‎aws_bedrock_langchain_python_cdk/lambda/code/boto3_example/helper.py‎

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
import json
2+
import boto3
3+
4+
'''
5+
Parameters:
6+
model_name - From list below or "list_foundation_models"
7+
prompt - Input to the model, string accepted, no default
8+
max_tokens - Configures the maximum number of tokens in the generated response
9+
temperature - 0-1, highest probability (least creative) to lowest probability (most creative)
10+
top_p - defines a cut off based on the sum of probabilities of the potential choices.
11+
top_k - Top K defines the cut off where the model no longer selects the words
12+
13+
Models currently supported:
14+
amazon.titan-tg1-large
15+
ai21.j2-grande-instruct
16+
ai21.j2-jumbo-instruct
17+
anthropic.claude-instant-v1
18+
anthropic.claude-v1
19+
anthropic.claude-v1-100k
20+
anthropic.claude-v2
21+
anthropic.claude-v2-100k
22+
23+
Notes:
24+
I needed to add a bedrock VPC endpoint to avoid timeouts.
25+
Some endpoints take 10-15 seconds to respond, set lambda timeout accordingly.
26+
Expects an environment variable called "secret_name". This is a Secrets Manager
27+
secret that contains AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_DEFAULT_REGION
28+
If your setup calls bedrock locally, make sure your Lambda permissions include bedrock access
29+
and remove the cross-account settings in the bedrock client calls.
30+
Youll need a layer for requests and one for the private version of boto3 that includes bedrock.
31+
https://towardsdatascience.com/building-custom-layers-on-aws-lambda-35d17bd9abbb
32+
33+
Working On:
34+
Better error handling, logging, and status responses
35+
Cohere (when its available)
36+
'''
37+
38+
bedrock_client = boto3.client('bedrock')
39+
bedrock_runtime_client = boto3.client('bedrock-runtime')
40+
41+
def call_list_models():
42+
43+
model_list = bedrock_client.list_foundation_models()
44+
response = []
45+
for model in model_list['modelSummaries']:
46+
print(model['modelId'])
47+
response.append(model['modelId'])
48+
49+
return response
50+
51+
52+
def call_bedrock_titan(prompt_text, model_name="amazon.titan-text-lite-v1", max_token_count=512, temperature=1, top_p=1, stop_sequences=[]):
53+
54+
body = json.dumps({
55+
"inputText": prompt_text
56+
})
57+
58+
response = bedrock_runtime_client.invoke_model(
59+
body=body,
60+
modelId=model_name,
61+
accept="application/json",
62+
contentType="application/json"
63+
)
64+
65+
response_body = json.loads(response.get("body").read())
66+
return response_body.get("results")[0].get("outputText")
67+
68+
69+
def call_bedrock_anthropic(prompt_text, model_name = "anthropic.claude-instant-v1", max_tokens=300, temperature=0.5, top_k=250, top_p=1):
70+
71+
body = json.dumps({
72+
"prompt": f"Human: {prompt_text} \n Assistant:",
73+
"max_tokens_to_sample": int(max_tokens),
74+
"temperature": float(temperature),
75+
"top_k": int(top_k),
76+
"top_p": float(top_p)
77+
})
78+
79+
response = bedrock_runtime_client.invoke_model(
80+
body=body,
81+
modelId=model_name,
82+
accept="application/json",
83+
contentType="application/json"
84+
)
85+
86+
response_body = json.loads(response.get("body").read())
87+
88+
return response_body.get("completion")
89+
90+
91+
def call_bedrock_jurassic(prompt_text, model_name="ai21.j2-ultra-v1", max_tokens=200, temperature=0.5, top_p=0.5):
92+
93+
body = json.dumps({
94+
"prompt": prompt_text,
95+
"maxTokens": int(max_tokens)
96+
})
97+
98+
response = bedrock_runtime_client.invoke_model(
99+
body=body,
100+
modelId=model_name,
101+
accept="application/json",
102+
contentType="application/json"
103+
)
104+
105+
response_body = json.loads(response.get("body").read())
106+
107+
return response_body.get("completions")[0].get("data").get("text")

0 commit comments

Comments
 (0)