Skip to content

Commit bc89959

Browse files
michaelnchinmichaelnchin
authored
Add Neptune Workbench CFN template (#442)
Co-authored-by: Michael Chin <chnmch@amazon.com>
1 parent 4794003 commit bc89959

File tree

6 files changed

+200
-5
lines changed

6 files changed

+200
-5
lines changed

‎additional-databases/sagemaker/README.md‎

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,7 @@
1-
## Launching graph-notebook using Amazon SageMaker
2-
You can easily launch instances of graph-notebook on Amazon SageMaker by using a lifecycle configuration. To learn more about lifecycle configurations and how to create one, see [documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/notebook-lifecycle-config.html).
1+
## Using graph-notebook on Amazon Sagemaker
32

4-
Use the sample lifecycle configuration in this folder, [`install-graph-notebook-lc.sh`](install-graph-notebook-lc.sh) ([`install-graph-notebook-lc-cn.sh`](install-graph-notebook-lc-cn.sh) if using `cn-north-1` or `cn-northwest-1` region) or create your own shell script.
3+
Amazon Sagemaker Notebooks provide an easy and effective solution for hosting, configuring, and running `graph-notebook` against a graph database. These notebooks also serve as the base platform for [Neptune Workbench](https://docs.aws.amazon.com/neptune/latest/userguide/graph-notebooks.html).
54

6-
After you create a lifecycle configuration on SageMaker, you can create new notebook instances by specifying a saved lifecycle configuration:
5+
If you would like to manually deploy a Neptune Workbench instance via AWS CloudFormation, please see the instructions in the [`neptune-notebook-cloudformation`](https://github.com/aws/graph-notebook/blob/main/additional-databases/sagemaker/neptune-notebook-cloudformation) folder.
76

8-
![create-a-notebook](/././images/Create-Notebook-Instance.png)
7+
For non-Neptune use cases, you can follow the instructions in the [`sagemaker-notebook-lifecycle`](https://github.com/aws/graph-notebook/blob/main/additional-databases/sagemaker/sagemaker-notebook-lifecycle) folder.

‎additional-databases/sagemaker/neptune-notebook-cloudformation/README.md‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
## Launching graph-notebook as Amazon Neptune Workbench via AWS CloudFormation
2+
3+
The AWS CloudFormation template in this folder, [`neptune-workbench-stack.yaml`](neptune-workbench-stack.yaml), deploys Amazon Neptune workbench notebooks as resources, and includes the base 'Getting Started' notebooks. The workbench lets you work with your Amazon Neptune cluster using Jupyter notebooks hosted by Amazon SageMaker. You are billed for workbench resources through Amazon SageMaker, separately from your Neptune billing.
4+
5+
### Parameter details
6+
#### Minimum permissions for the SageMakerNotebookRole
7+
This is the ARN for the AWS IAM role that the notebook instance will assume. Make sure that this role has at least the following minimum permissions within its service role policy:
8+
9+
```json
10+
{
11+
"Version": "2012-10-17",
12+
"Statement": [
13+
{
14+
"Effect": "Allow",
15+
"Action": [
16+
"s3:GetObject",
17+
"s3:ListBucket"
18+
],
19+
"Resource": [
20+
"arn:aws:s3:::aws-neptune-notebook",
21+
"arn:aws:s3:::aws-neptune-notebook/*"
22+
]
23+
},
24+
{
25+
"Effect": "Allow",
26+
"Action": "neptune-db:connect",
27+
"Resource": [
28+
"your-cluster-arn/*"
29+
]
30+
}
31+
]
32+
}
33+
```
34+
35+
The role should also establish the following trust relationship:
36+
37+
```json
38+
{
39+
"Version": "2012-10-17",
40+
"Statement": [
41+
{
42+
"Effect": "Allow",
43+
"Principal": {
44+
"Service": "sagemaker.amazonaws.com"
45+
},
46+
"Action": "sts:AssumeRole"
47+
}
48+
]
49+
}
50+
```
51+
52+
#### How to populate the 'Cluster' value within the AWS Console for Amazon Neptune Notebooks
53+
Add the following tags manually to the notebook instance.
54+
55+
| Key | Value |
56+
| ------------- |-------------|
57+
| **aws-neptune-cluster-id** | Amazon Neptune database cluster ID (found under *DB cluster id* under *Configuration* of the selected cluster in the AWS console) |
58+
| **aws-neptune-resource-id** | Amazon Neptune cluster resource ID (found under *Resource id* under *Configuration* of the selected cluster in the AWS console) |

‎additional-databases/sagemaker/neptune-notebook-cloudformation/neptune-workbench-stack.yaml‎

Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
AWSTemplateFormatVersion: '2010-09-09'
2+
3+
Description: A template to deploy Neptune Notebooks using CloudFormation resources.
4+
5+
Parameters:
6+
NotebookInstanceType:
7+
Description: The notebook instance type.
8+
Type: String
9+
Default: ml.t2.medium
10+
AllowedValues:
11+
- ml.t2.medium
12+
- ml.t2.large
13+
- ml.t2.xlarge
14+
- ml.t2.2xlarge
15+
- ml.t3.2xlarge
16+
- ml.t3.large
17+
- ml.t3.medium
18+
- ml.t3.xlarge
19+
- ml.m4.xlarge
20+
- ml.m4.2xlarge
21+
- ml.m4.4xlarge
22+
- ml.m4.10xlarge
23+
- ml.m4.16xlarge
24+
- ml.m5.12xlarge
25+
- ml.m5.24xlarge
26+
- ml.m5.2xlarge
27+
- ml.m5.4xlarge
28+
- ml.m5.xlarge
29+
- ml.p2.16xlarge
30+
- ml.p2.8xlarge
31+
- ml.p2.xlarge
32+
- ml.p3.16xlarge
33+
- ml.p3.2xlarge
34+
- ml.p3.8xlarge
35+
- ml.c4.2xlarge
36+
- ml.c4.4xlarge
37+
- ml.c4.8xlarge
38+
- ml.c4.xlarge
39+
- ml.c5.18xlarge
40+
- ml.c5.2xlarge
41+
- ml.c5.4xlarge
42+
- ml.c5.9xlarge
43+
- ml.c5.xlarge
44+
- ml.c5d.18xlarge
45+
- ml.c5d.2xlarge
46+
- ml.c5d.4xlarge
47+
- ml.c5d.9xlarge
48+
- ml.c5d.xlarge
49+
ConstraintDescription: Must be a valid SageMaker instance type.
50+
51+
NeptuneClusterEndpoint:
52+
Description: The cluster endpoint of an existing Neptune cluster.
53+
Type: String
54+
55+
NeptuneClusterPort:
56+
Description: 'OPTIONAL: The Port of an existing Neptune cluster (default 8182).'
57+
Type: String
58+
Default: '8182'
59+
60+
NeptuneClusterSecurityGroups:
61+
Description: The VPC security group IDs. The security groups must be for the same VPC as specified in the subnet.
62+
Type: List<AWS::EC2::SecurityGroup::Id>
63+
64+
NeptuneClusterSubnetId:
65+
Description: The ID of the subnet in a VPC to which you would like to have a connectivity from your ML compute instance.
66+
Type: AWS::EC2::Subnet::Id
67+
68+
SageMakerNotebookRole:
69+
Description: The ARN for the IAM role that the notebook instance will assume.
70+
Type: String
71+
AllowedPattern: ^arn:aws[a-z\-]*:iam::\d{12}:role/?[a-zA-Z_0-9+=,.@\-_/]+$
72+
73+
SageMakerNotebookName:
74+
Description: The name of the Neptune notebook.
75+
Type: String
76+
77+
Resources:
78+
NeptuneNotebookInstance:
79+
Type: AWS::SageMaker::NotebookInstance
80+
Properties:
81+
NotebookInstanceName: !Join
82+
- ''
83+
- - 'aws-neptune-'
84+
- !Ref SageMakerNotebookName
85+
InstanceType:
86+
Ref: NotebookInstanceType
87+
SubnetId:
88+
Ref: NeptuneClusterSubnetId
89+
SecurityGroupIds:
90+
Ref: NeptuneClusterSecurityGroups
91+
RoleArn:
92+
Ref: SageMakerNotebookRole
93+
LifecycleConfigName:
94+
Fn::GetAtt:
95+
- NeptuneNotebookInstanceLifecycleConfig
96+
- NotebookInstanceLifecycleConfigName
97+
98+
NeptuneNotebookInstanceLifecycleConfig:
99+
Type: AWS::SageMaker::NotebookInstanceLifecycleConfig
100+
Properties:
101+
OnStart:
102+
- Content:
103+
Fn::Base64:
104+
Fn::Join:
105+
- ''
106+
- - "#!/bin/bash\n"
107+
- sudo -u ec2-user -i << 'EOF'
108+
- "\n"
109+
- echo 'export GRAPH_NOTEBOOK_AUTH_MODE=
110+
- "DEFAULT' >> ~/.bashrc\n"
111+
- echo 'export GRAPH_NOTEBOOK_HOST=
112+
- !Ref NeptuneClusterEndpoint
113+
- "' >> ~/.bashrc\n"
114+
- echo 'export GRAPH_NOTEBOOK_PORT=
115+
- !Ref NeptuneClusterPort
116+
- "' >> ~/.bashrc\n"
117+
- echo 'export NEPTUNE_LOAD_FROM_S3_ROLE_ARN=
118+
- "' >> ~/.bashrc\n"
119+
- echo 'export AWS_REGION=
120+
- !Ref AWS::Region
121+
- "' >> ~/.bashrc\n"
122+
- aws s3 cp s3://aws-neptune-notebook/graph_notebook.tar.gz /tmp/graph_notebook.tar.gz
123+
- "\n"
124+
- rm -rf /tmp/graph_notebook
125+
- "\n"
126+
- tar -zxvf /tmp/graph_notebook.tar.gz -C /tmp
127+
- "\n"
128+
- /tmp/graph_notebook/install.sh
129+
- "\n"
130+
- EOF

‎additional-databases/sagemaker/sagemaker-notebook-lifecycle/README.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
## Launching graph-notebook on Amazon SageMaker using a lifecycle
2+
You can easily configure graph-notebook to run on an Amazon SageMaker Notebook instance by using a lifecycle configuration. To learn more about lifecycle configurations and how to create one, see [documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/notebook-lifecycle-config.html).
3+
4+
Use the sample lifecycle configuration in this folder, [`install-graph-notebook-lc.sh`](install-graph-notebook-lc.sh) ([`install-graph-notebook-lc-cn.sh`](install-graph-notebook-lc-cn.sh) if using `cn-north-1` or `cn-northwest-1` region) or create your own shell script.
5+
6+
After you create a lifecycle configuration on SageMaker, you can create new notebook instances by specifying a saved lifecycle configuration:
7+
8+
![create-a-notebook](/images/Create-Notebook-Instance.png)

‎additional-databases/sagemaker/install-graph-notebook-lc-cn.sh‎ renamed to ‎additional-databases/sagemaker/sagemaker-notebook-lifecycle/install-graph-notebook-lc-cn.sh‎

File renamed without changes.

‎additional-databases/sagemaker/install-graph-notebook-lc.sh‎ renamed to ‎additional-databases/sagemaker/sagemaker-notebook-lifecycle/install-graph-notebook-lc.sh‎

File renamed without changes.

0 commit comments

Comments
 (0)