Skip to content

Commit 3d90db6

Browse files
docs: update AWS Batch service (#183)
Co-authored-by: Nikos <nikos.michas@localstack.cloud>
1 parent 88c34bc commit 3d90db6

File tree

1 file changed

+94
-71
lines changed

1 file changed

+94
-71
lines changed

src/content/docs/aws/services/batch.mdx

Lines changed: 94 additions & 71 deletions
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,12 @@ import FeatureCoverage from "../../../../components/feature-coverage/FeatureCove
88

99
## Introduction
1010

11-
Batch is a cloud-based service provided by Amazon Web Services (AWS) that simplifies the process of running batch computing workloads on the AWS cloud infrastructure.
12-
Batch allows you to efficiently process large volumes of data and run batch jobs without the need to manage and provision underlying compute resources.
11+
Batch is a cloud-based service provided by Amazon Web Services (AWS) that simplifies the process of running batch computing workloads on the AWS cloud infrastructure. Batch allows you to efficiently process large volumes of data and run batch jobs without the need to manage and provision underlying compute resources.
12+
13+
Under the hood, the local Docker engine is used to run the containers that simulate your Batch jobs.
14+
15+
LocalStack allows you to use the Batch APIs to automate and scale computational tasks in your local environment while handling batch workloads. Batch jobs are executed using the ECS runtime, allowing for support of managed compute environments and improved service compatibility.
1316

14-
LocalStack allows you to use the Batch APIs to automate and scale computational tasks in your local environment while handling batch workloads.
1517
The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Batch integration with LocalStack.
1618

1719
## Getting started
@@ -30,57 +32,61 @@ We will demonstrate how you create and run a Batch job by following these steps:
3032
### Create a service role
3133

3234
You can create a role using the [`CreateRole`](https://docs.aws.amazon.com/cli/latest/reference/iam/create-role.html) API.
33-
For LocalStack, the service role simply needs to exist.
34-
However, when [enforcing IAM policies](/aws/capabilities/security-testing/iam-policy-enforcement), it is necessary that the policy is valid.
35+
36+
LocalStack requires the role to exist with a valid trust policy. When [enforcing IAM policies](/aws/capabilities/security-testing/iam-policy-enforcement), ensure that the policy is valid and the role is properly attached.
3537

36-
Run the following command to create a role with an empty policy document:
38+
Run the following command to create a role for ECS task execution:
3739

3840
```bash
3941
awslocal iam create-role \
40-
--role-name myrole \
41-
--assume-role-policy-document "{}"
42+
--role-name myrole \
43+
--assume-role-policy-document '{
44+
"Version": "2025-10-17",
45+
"Statement": [
46+
{
47+
"Effect": "Allow",
48+
"Principal": {
49+
"Service": "ecs-tasks.amazonaws.com"
50+
},
51+
"Action": "sts:AssumeRole"
52+
}
53+
]
54+
}'
4255
```
4356

44-
```bash title="Output"
45-
{
46-
"Role": {
47-
"Path": "/",
48-
"RoleName": "myrole",
49-
"RoleId": "AROAQAAAAAAAMKIDGTHVC",
50-
"Arn": "arn:aws:iam::000000000000:role/myrole",
51-
"CreateDate": "2023-08-10T20:52:06.196000Z",
52-
"AssumeRolePolicyDocument": {}
53-
}
54-
}
57+
Then attach the ECS task execution policy:
58+
59+
```bash
60+
awslocal iam attach-role-policy \
61+
--role-name myrole \
62+
--policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
5563
```
5664

65+
5766
### Create the compute environment
5867

5968
You can use the [`CreateComputeEnvironment`](https://docs.aws.amazon.com/cli/latest/reference/batch/create-compute-environment.html) API to create a compute environment.
60-
Run the following command using the role ARN above (`arn:aws:iam::000000000000:role/myrole`), to create the compute environment:
69+
70+
Run the following command using the role ARN above (arn:aws:iam::000000000000:role/myrole) to create a managed compute environment with FARGATE:
6171

6272
```bash
6373
awslocal batch create-compute-environment \
6474
--compute-environment-name myenv \
65-
--type UNMANAGED \
66-
--service-role <role-arn>
67-
```
68-
69-
```bash title="Output"
70-
{
71-
"computeEnvironmentName": "myenv",
72-
"computeEnvironmentArn": "arn:aws:batch:us-east-1:000000000000:compute-environment/myenv"
73-
}
75+
--type MANAGED \
76+
--state ENABLED \
77+
--compute-resources type=FARGATE,maxvCpus=128,subnets=subnet-12345678,securityGroupIds=sg-12345678 \
78+
--service-role arn:aws:iam::000000000000:role/myrole
7479
```
7580

7681
:::note
77-
While an unmanaged compute environment has been specified, there is no need to provision any compute resources for this setup to function.
78-
Your tasks will run independently in new Docker containers, alongside the LocalStack container.
82+
While networking resources such as subnets and security groups are required as input, LocalStack does not create real cloud infrastructure. These values must still be present for the compute environment to be created.
7983
:::
8084

85+
8186
### Create a job queue
8287

8388
You can fetch the ARN using the [`DescribeComputeEnvironments`](https://docs.aws.amazon.com/cli/latest/reference/batch/describe-compute-environments.html) API.
89+
8490
Run the following command to fetch the ARN of the compute environment:
8591

8692
```bash
@@ -89,21 +95,22 @@ awslocal batch describe-compute-environments --compute-environments myenv
8995

9096
```bash title="Output"
9197
{
92-
"computeEnvironments": [
93-
{
94-
"computeEnvironmentName": "myenv",
95-
"computeEnvironmentArn": "arn:aws:batch:us-east-1:000000000000:compute-environment/myenv",
96-
"ecsClusterArn": "arn:aws:ecs:us-east-1:000000000000:cluster/OnDemand_Batch_f2faa82c-8c31-466d-ab22-579925d810ac",
97-
"type": "UNMANAGED",
98-
"status": "VALID",
99-
"statusReason": "Compute environment is available",
100-
"serviceRole": "arn:aws:iam::000000000000:role/myrole"
101-
}
102-
]
98+
"computeEnvironments": [
99+
{
100+
"computeEnvironmentName": "myenv",
101+
"computeEnvironmentArn": "arn:aws:batch:us-east-1:000000000000:compute-environment/myenv",
102+
"ecsClusterArn": "arn:aws:ecs:us-east-1:000000000000:cluster/OnDemand_Batch_abc123",
103+
"type": "MANAGED",
104+
"status": "VALID",
105+
"statusReason": "Compute environment is available",
106+
"serviceRole": "arn:aws:iam::000000000000:role/myrole"
107+
}
108+
]
103109
}
104110
```
105111

106112
You can use the ARN to create the job queue using [`CreateJobQueue`](https://docs.aws.amazon.com/cli/latest/reference/batch/create-job-queue.html) API.
113+
107114
Run the following command to create the job queue:
108115

109116
```bash
@@ -114,45 +121,54 @@ awslocal batch create-job-queue \
114121
--state ENABLED
115122
```
116123

117-
```bash title="Output"
118-
{
119-
"jobQueueName": "myqueue",
120-
"jobQueueArn": "arn:aws:batch:us-east-1:000000000000:job-queue/myqueue"
121-
}
122-
```
123-
124124
### Create a job definition
125125

126-
Now, you can define what occurs during a job run, or at least what transpires by default.
127-
In this example, you can execute the `busybox` container from DockerHub and initiate the command: `sleep 30` within it.
128-
It's important to note that you can override this command when submitting the job.
126+
Now, you can define what occurs during a job run. In this example, you can execute the 'busybox' container from DockerHub and initiate the command: 'sleep 30'. It's important to note you can override this command when submitting the job.
129127

130128
Run the following command to create the job definition using the [`RegisterJobDefinition`](https://docs.aws.amazon.com/cli/latest/reference/batch/register-job-definition.html) API:
131129

130+
132131
```bash
133132
awslocal batch register-job-definition \
134133
--job-definition-name myjobdefn \
135134
--type container \
136-
--container-properties '{"image":"busybox","vcpus":1,"memory":128,"command":["sleep","30"]}'
137-
```
138-
139-
```bash title="Output"
140-
{
141-
"jobDefinitionName": "myjobdefn",
142-
"jobDefinitionArn": "arn:aws:batch:us-east-1:000000000000:job-definition/myjobdefn:1",
143-
"revision": 1
144-
}
135+
--platform-capabilities FARGATE \
136+
--container-properties '{
137+
"image": "busybox",
138+
"resourceRequirements": [
139+
{"type": "VCPU", "value": "0.25"},
140+
{"type": "MEMORY", "value": "512"}
141+
],
142+
"command": ["sleep", "30"],
143+
"networkConfiguration": {
144+
"assignPublicIp": "ENABLED"
145+
},
146+
"executionRoleArn": "arn:aws:iam::000000000000:role/myrole"
147+
}'
145148
```
146149

147150
If you want to pass arguments to the command as [parameters](https://docs.aws.amazon.com/batch/latest/userguide/job_definition_parameters.html#parameters), you can use the `Ref::` declaration to set placeholders for parameter substitution.
151+
148152
This allows the dynamic passing of values at runtime for specific job definitions.
149153

150154
```bash
151155
awslocal batch register-job-definition \
152156
--job-definition-name myjobdefn \
153157
--type container \
154158
--parameters '{"time":"10"}' \
155-
--container-properties '{"image":"busybox","vcpus":1,"memory":128,"command":["sleep","Ref::time"]}'
159+
--platform-capabilities FARGATE \
160+
--container-properties '{
161+
"image": "busybox",
162+
"resourceRequirements": [
163+
{"type": "VCPU", "value": "0.25"},
164+
{"type": "MEMORY", "value": "512"}
165+
],
166+
"command": ["sleep", "Ref::time"],
167+
"networkConfiguration": {
168+
"assignPublicIp": "ENABLED"
169+
},
170+
"executionRoleArn": "arn:aws:iam::000000000000:role/myrole"
171+
}'
156172
```
157173

158174
### Submit a job to the job queue
@@ -171,17 +187,24 @@ awslocal batch submit-job \
171187
--container-overrides '{"command":["sh", "-c", "sleep 5; pwd"]}'
172188
```
173189

174-
```bash title="Output"
175-
{
176-
"jobName": "myjob",
177-
"jobId": "23027eb6-cce0-4365-a412-36917a2dfd03"
178-
}
179-
```
180-
181190
## Current Limitations
182191

183-
As mentioned in the example above, the creation of a compute environment does not entail the provisioning of EC2 or Fargate instances.
184-
Rather, it executes Batch jobs on the local Docker daemon, operating alongside LocalStack.
192+
LocalStack simulates the execution of ECS-based AWS Batch jobs using the local ECS runtime. No real infrastructure is created or managed.
193+
194+
Array jobs are supported in sequential mode only.
195+
196+
A subset of environment variables is supported, including:
197+
- `AWS_BATCH_CE_NAME`
198+
- `AWS_BATCH_JOB_ARRAY_INDEX`
199+
- `AWS_BATCH_JOB_ARRAY_SIZE`
200+
- `AWS_BATCH_JOB_ATTEMPT`
201+
- `AWS_BATCH_JOB_ID`
202+
- `AWS_BATCH_JQ_NAME`
203+
204+
The configuration variable `ECS_DOCKER_FLAGS` can be used to pass additional Docker flags to the container runtime.
205+
206+
Setting `ECS_TASK_EXECUTOR=kubernetes` is supported as an alternative backend, though Kubernetes execution is experimental and may not support all features.
207+
185208

186209
## API Coverage
187210

0 commit comments

Comments
 (0)