AWS Amazon EMR Cluster
This page shows how to write Terraform and CloudFormation for Amazon EMR Cluster and write them securely.
aws_emr_cluster (Terraform)
The Cluster in Amazon EMR can be configured in Terraform with the resource name aws_emr_cluster
. The following sections describe 2 examples of how to use the resource and its parameters.
Example Usage from GitHub
resource "aws_emr_cluster" "cluster_ok" {
name = "emr-test-arn"
release_label = "emr-4.6.0"
applications = ["Spark"]
ec2_attributes {
resource "aws_emr_cluster" "cluster_ok" {
name = "emr-test-arn"
release_label = "emr-4.6.0"
applications = ["Spark"]
ec2_attributes {
Parameters
-
additional_info
optional - string -
applications
optional - set of string -
arn
optional computed - string -
autoscaling_role
optional - string -
cluster_state
optional computed - string -
configurations
optional - string -
configurations_json
optional - string -
custom_ami_id
optional - string -
ebs_root_volume_size
optional - number -
id
optional computed - string -
keep_job_flow_alive_when_no_steps
optional computed - bool -
log_uri
optional - string -
master_public_dns
optional computed - string -
name
required - string -
release_label
required - string -
scale_down_behavior
optional computed - string -
security_configuration
optional - string -
service_role
required - string -
step
optional computed - list of object-
action_on_failure
- string -
hadoop_jar_step
- list of object-
args
- list of string -
jar
- string -
main_class
- string -
properties
- map from string to string
-
-
name
- string
-
-
step_concurrency_level
optional - number -
tags
optional - map from string to string -
termination_protection
optional computed - bool -
visible_to_all_users
optional - bool -
bootstrap_action
list block -
core_instance_fleet
list block-
id
optional computed - string -
name
optional - string -
provisioned_on_demand_capacity
optional computed - number -
provisioned_spot_capacity
optional computed - number -
target_on_demand_capacity
optional - number -
target_spot_capacity
optional - number -
instance_type_configs
set block-
bid_price
optional - string -
bid_price_as_percentage_of_on_demand_price
optional - number -
instance_type
required - string -
weighted_capacity
optional - number -
configurations
set block-
classification
optional - string -
properties
optional - map from string to string
-
-
ebs_config
set block-
iops
optional - number -
size
required - number -
type
required - string -
volumes_per_instance
optional - number
-
-
-
launch_specifications
list block-
on_demand_specification
list block-
allocation_strategy
required - string
-
-
spot_specification
list block-
allocation_strategy
required - string -
block_duration_minutes
optional - number -
timeout_action
required - string -
timeout_duration_minutes
required - number
-
-
-
-
core_instance_group
list block-
autoscaling_policy
optional - string -
bid_price
optional - string -
id
optional computed - string -
instance_count
optional - number -
instance_type
required - string -
name
optional - string -
ebs_config
set block-
iops
optional - number -
size
required - number -
type
required - string -
volumes_per_instance
optional - number
-
-
-
ec2_attributes
list block-
additional_master_security_groups
optional - string -
additional_slave_security_groups
optional - string -
emr_managed_master_security_group
optional computed - string -
emr_managed_slave_security_group
optional computed - string -
instance_profile
required - string -
key_name
optional - string -
service_access_security_group
optional computed - string -
subnet_id
optional computed - string -
subnet_ids
optional computed - set of string
-
-
kerberos_attributes
list block-
ad_domain_join_password
optional - string -
ad_domain_join_user
optional - string -
cross_realm_trust_principal_password
optional - string -
kdc_admin_password
required - string -
realm
required - string
-
-
master_instance_fleet
list block-
id
optional computed - string -
name
optional - string -
provisioned_on_demand_capacity
optional computed - number -
provisioned_spot_capacity
optional computed - number -
target_on_demand_capacity
optional - number -
target_spot_capacity
optional - number -
instance_type_configs
set block-
bid_price
optional - string -
bid_price_as_percentage_of_on_demand_price
optional - number -
instance_type
required - string -
weighted_capacity
optional - number -
configurations
set block-
classification
optional - string -
properties
optional - map from string to string
-
-
ebs_config
set block-
iops
optional - number -
size
required - number -
type
required - string -
volumes_per_instance
optional - number
-
-
-
launch_specifications
list block-
on_demand_specification
list block-
allocation_strategy
required - string
-
-
spot_specification
list block-
allocation_strategy
required - string -
block_duration_minutes
optional - number -
timeout_action
required - string -
timeout_duration_minutes
required - number
-
-
-
-
master_instance_group
list block-
bid_price
optional - string -
id
optional computed - string -
instance_count
optional - number -
instance_type
required - string -
name
optional - string -
ebs_config
set block-
iops
optional - number -
size
required - number -
type
required - string -
volumes_per_instance
optional - number
-
-
Explanation in Terraform Registry
Provides an Elastic MapReduce Cluster, a web service that makes it easy to process large amounts of data efficiently. See Amazon Elastic MapReduce Documentation for more information. To configure Instance Groups for task nodes, see the
aws_emr_instance_group
resource.
AWS::EMR::Cluster (CloudFormation)
The Cluster in EMR can be configured in CloudFormation with the resource name AWS::EMR::Cluster
. The following sections describe 10 examples of how to use the resource and its parameters.
Example Usage from GitHub
Type: 'AWS::EMR::Cluster'
crmTransformationPipelineStepFunctions:
Type: 'AWS::StepFunctions::StateMachine'
Properties:
RoleArn: ''
Type: 'AWS::EMR::Cluster'
crmTransformationPipelineStepFunctions:
Type: 'AWS::StepFunctions::StateMachine'
Properties:
RoleArn: ''
Type: AWS::EMR::Cluster
Properties:
Instances:
MasterInstanceGroup:
InstanceCount: 1
InstanceType: m1.medium
Type: AWS::EMR::Cluster
Properties:
Instances:
MasterInstanceGroup:
InstanceCount: 1
InstanceType: m1.medium
Type: 'AWS::EMR::Cluster'
Properties:
Instances:
MasterInstanceGroup:
InstanceCount: 1
InstanceType: m5.xlarge
"Type": "AWS::EMR::Cluster",
"Properties": {
"Instances": {
"Ec2KeyName" : "ukmyjorgeawstestkeypair",
"HadoopVersion" : "Amazon 2.8.5",
"MasterInstanceGroup": {
"resourceType" : "AWS::EMR::Cluster",
"properties" : [ {
"propertyName" : "AdditionalInfo",
"propertyType" : "JsonObject",
"required" : false
}, {
"Type": "AWS::EMR::Cluster",
"Properties": {
"Instances": {
"MasterInstanceGroup": {
"InstanceCount": 1,
"InstanceType": "m1.medium",
"Type": "AWS::EMR::Cluster",
"Properties": {
"Instances": {
"MasterInstanceGroup": {
"InstanceCount": 1,
"InstanceType": "m1.medium",
"Type":"AWS::EMR::Cluster::InstanceType"
},
"CoreInstanceType":{
"Description":"Instance type to be used for core instances.",
"Type":"String",
"AllowedValues":[
Parameters
-
AdditionalInfo
optional - Json -
Applications
optional - List of Application -
AutoScalingRole
optional - String -
BootstrapActions
optional - List of BootstrapActionConfig -
Configurations
optional - List of Configuration -
CustomAmiId
optional - String -
EbsRootVolumeSize
optional - Integer -
Instances
required - JobFlowInstancesConfig -
JobFlowRole
required - String -
KerberosAttributes
optional - KerberosAttributes -
LogEncryptionKmsKeyId
optional - String -
LogUri
optional - String -
ManagedScalingPolicy
optional - ManagedScalingPolicy -
Name
required - String -
ReleaseLabel
optional - String -
ScaleDownBehavior
optional - String -
SecurityConfiguration
optional - String -
ServiceRole
required - String -
StepConcurrencyLevel
optional - Integer -
Steps
optional - List of StepConfig -
Tags
optional - List of Tag -
VisibleToAllUsers
optional - Boolean
Explanation in CloudFormation Registry
The
AWS::EMR::Cluster
resource specifies an Amazon EMR cluster. This cluster is a collection of Amazon EC2 instances that run open source big data frameworks and applications to process and analyze vast amounts of data. For more information, see the Amazon EMR Management Guide.
Frequently asked questions
What is AWS Amazon EMR Cluster?
AWS Amazon EMR Cluster is a resource for Amazon EMR of Amazon Web Service. Settings can be wrote in Terraform and CloudFormation.
Where can I find the example code for the AWS Amazon EMR Cluster?
For Terraform, the SnidermanIndustries/checkov-fork and melscoop-test/check source code examples are useful. See the Terraform Example section for further details.
For CloudFormation, the demeritiusg/data_warehouse-pipelines, demeritiusg/AWS-Examples and stelligent/cfn_nag source code examples are useful. See the CloudFormation Example section for further details.