AWS Batch Compute Environment
This page shows how to write Terraform and CloudFormation for AWS Batch Compute Environment and write them securely.
aws_batch_compute_environment (Terraform)
The Compute Environment in AWS Batch can be configured in Terraform with the resource name aws_batch_compute_environment
. The following sections describe 1 example of how to use the resource and its parameters.
Example Usage from GitHub
resource "aws_batch_compute_environment" "nf_spot" {
compute_environment_name = "nextflow-spot-compute"
tags = var.default_tags
compute_resources {
instance_role = aws_iam_instance_profile.nf_ecs_instance_role.arn
instance_type = [
Parameters
-
arn
optional computed - string -
compute_environment_name
optional computed - string -
compute_environment_name_prefix
optional - string -
ecs_cluster_arn
optional computed - string -
id
optional computed - string -
service_role
required - string -
state
optional - string -
status
optional computed - string -
status_reason
optional computed - string -
tags
optional - map from string to string -
type
required - string -
compute_resources
list block-
allocation_strategy
optional - string -
bid_percentage
optional - number -
desired_vcpus
optional computed - number -
ec2_key_pair
optional - string -
image_id
optional - string -
instance_role
required - string -
instance_type
required - set of string -
max_vcpus
required - number -
min_vcpus
required - number -
security_group_ids
required - set of string -
spot_iam_fleet_role
optional - string -
subnets
required - set of string -
tags
optional - map from string to string -
type
required - string -
launch_template
list block-
launch_template_id
optional - string -
launch_template_name
optional - string -
version
optional - string
-
-
Explanation in Terraform Registry
Creates a AWS Batch compute environment. Compute environments contain the Amazon ECS container instances that are used to run containerized batch jobs. For information about AWS Batch, see [What is AWS Batch?][1] . For information about compute environment, see [Compute Environments][2] .
Note: To prevent a race condition during environment deletion, make sure to set
depends_on
to the relatedaws_iam_role_policy_attachment
; otherwise, the policy may be destroyed too soon and the compute environment will then get stuck in theDELETING
state, see [Troubleshooting AWS Batch][3] .
AWS::Batch::ComputeEnvironment (CloudFormation)
The ComputeEnvironment in Batch can be configured in CloudFormation with the resource name AWS::Batch::ComputeEnvironment
. The following sections describe 10 examples of how to use the resource and its parameters.
Example Usage from GitHub
Type: AWS::Batch::ComputeEnvironment
Properties:
Type: MANAGED
State: ENABLED
ServiceRole: !Ref BatchServiceRole
ComputeEnvironmentName: "#{AWS::StackName}-multifish-ondemand-small-cpu-compute-env"
Type: AWS::Batch::ComputeEnvironment
Properties:
ServiceRole:
Fn::ImportValue: !Sub "${RoleStackName}:BatchServiceRole"
Type: MANAGED
State: ENABLED
Type: AWS::Batch::ComputeEnvironment
Properties:
ServiceRole:
Fn::ImportValue: !Sub "${RoleStackName}:BatchServiceRole"
Type: MANAGED
State: ENABLED
Type: AWS::Batch::ComputeEnvironment
Properties:
ServiceRole:
Fn::ImportValue: !Sub "${RoleStackName}:BatchServiceRole"
Type: MANAGED
State: ENABLED
Type: AWS::Batch::ComputeEnvironment
Properties:
ServiceRole:
Fn::ImportValue: !Sub "${RoleStackName}:BatchServiceRole"
Type: MANAGED
State: ENABLED
"Type": "AWS::Batch::ComputeEnvironment",
"Properties": {
"Type": "MANAGED",
"ServiceRole": {
"Ref": "RoleBatch"
},
"Type" : "AWS::Batch::ComputeEnvironment",
"Properties" : {
"Type" : "MANAGED",
"ComputeResources" : {
"Type" : "EC2",
"MinvCpus" : 0,
"Type" : "AWS::Batch::ComputeEnvironment",
"Properties" : {
"Type" : "MANAGED",
"ComputeResources" : {
"Type" : "EC2",
"MinvCpus" : 0,
"Type": "AWS::Batch::ComputeEnvironment"
},
"GPUJobQueue": {
"Properties": {
"ComputeEnvironmentOrder": [
{
"Type": "AWS::Batch::ComputeEnvironment"
},
"GPUJobQueue": {
"Properties": {
"ComputeEnvironmentOrder": [
{
Parameters
-
UnmanagedvCpus
optional - Integer -
Type
required - String -
ServiceRole
optional - String -
ComputeEnvironmentName
optional - String -
ComputeResources
optional - ComputeResources -
State
optional - String -
Tags
optional - Json
Explanation in CloudFormation Registry
The
AWS::Batch::ComputeEnvironment
resource defines your AWS Batch compute environment. You can defineMANAGED
orUNMANAGED
compute environments.MANAGED
compute environments can use Amazon EC2 or AWS Fargate resources.UNMANAGED
compute environments can only use EC2 resources. For more information, see Compute Environments in the AWS Batch User Guide.In a managed compute environment, AWS Batch manages the capacity and instance types of the compute resources within the environment. This is based on the compute resource specification that you define or the launch template that you specify when you create the compute environment. You can choose either to use EC2 On-Demand Instances and EC2 Spot Instances, or to use Fargate and Fargate Spot capacity in your managed compute environment. You can optionally set a maximum price so that Spot Instances only launch when the Spot Instance price is below a specified percentage of the On-Demand price.
Note Multi-node parallel jobs are not supported on Spot Instances.
In an unmanaged compute environment, you can manage your own EC2 compute resources and have a lot of flexibility with how you configure your compute resources. For example, you can use custom AMI. However, you need to verify that your AMI meets the Amazon ECS container instance AMI specification. For more information, see container instance AMIs in the Amazon Elastic Container Service Developer Guide. After you have created your unmanaged compute environment, you can use the DescribeComputeEnvironments operation to find the Amazon ECS cluster that is associated with it. Then, manually launch your container instances into that Amazon ECS cluster. For more information, see Launching an Amazon ECS container instance in the Amazon Elastic Container Service Developer Guide.
Note AWS Batch doesn't upgrade the AMIs in a compute environment after it's created. For example, it doesn't update the AMIs when a newer version of the Amazon ECS optimized AMI is available. Therefore, you're responsible for the management of the guest operating system (including updates and security patches) and any additional application software or utilities that you install on the compute resources. To use a new AMI for your AWS Batch jobs, complete these steps: Create a new compute environment with the new AMI.
Add the compute environment to an existing job queue.
Remove the earlier compute environment from your job queue.
Delete the earlier compute environment.
Frequently asked questions
What is AWS Batch Compute Environment?
AWS Batch Compute Environment is a resource for Batch of Amazon Web Service. Settings can be wrote in Terraform and CloudFormation.
Where can I find the example code for the AWS Batch Compute Environment?
For Terraform, the AlexsLemonade/alsf-scpca source code example is useful. See the Terraform Example section for further details.
For CloudFormation, the JaneliaSciComp/multifish, jbaden15/aws_batch_genomics and aws-samples/aws-batch-genomics source code examples are useful. See the CloudFormation Example section for further details.