AWS Amazon EMR Cluster
This page shows how to write Terraform and CloudFormation for Amazon EMR Cluster and write them securely.
aws_emr_cluster (Terraform)
The Cluster in Amazon EMR can be configured in Terraform with the resource name aws_emr_cluster. The following sections describe 2 examples of how to use the resource and its parameters.
Example Usage from GitHub
resource "aws_emr_cluster" "cluster_ok" {
name = "emr-test-arn"
release_label = "emr-4.6.0"
applications = ["Spark"]
ec2_attributes {
resource "aws_emr_cluster" "cluster_ok" {
name = "emr-test-arn"
release_label = "emr-4.6.0"
applications = ["Spark"]
ec2_attributes {
Parameters
-
additional_infooptional - string -
applicationsoptional - set of string -
arnoptional computed - string -
autoscaling_roleoptional - string -
cluster_stateoptional computed - string -
configurationsoptional - string -
configurations_jsonoptional - string -
custom_ami_idoptional - string -
ebs_root_volume_sizeoptional - number -
idoptional computed - string -
keep_job_flow_alive_when_no_stepsoptional computed - bool -
log_urioptional - string -
master_public_dnsoptional computed - string -
namerequired - string -
release_labelrequired - string -
scale_down_behavioroptional computed - string -
security_configurationoptional - string -
service_rolerequired - string -
stepoptional computed - list of object-
action_on_failure- string -
hadoop_jar_step- list of object-
args- list of string -
jar- string -
main_class- string -
properties- map from string to string
-
-
name- string
-
-
step_concurrency_leveloptional - number -
tagsoptional - map from string to string -
termination_protectionoptional computed - bool -
visible_to_all_usersoptional - bool -
bootstrap_actionlist block -
core_instance_fleetlist block-
idoptional computed - string -
nameoptional - string -
provisioned_on_demand_capacityoptional computed - number -
provisioned_spot_capacityoptional computed - number -
target_on_demand_capacityoptional - number -
target_spot_capacityoptional - number -
instance_type_configsset block-
bid_priceoptional - string -
bid_price_as_percentage_of_on_demand_priceoptional - number -
instance_typerequired - string -
weighted_capacityoptional - number -
configurationsset block-
classificationoptional - string -
propertiesoptional - map from string to string
-
-
ebs_configset block-
iopsoptional - number -
sizerequired - number -
typerequired - string -
volumes_per_instanceoptional - number
-
-
-
launch_specificationslist block-
on_demand_specificationlist block-
allocation_strategyrequired - string
-
-
spot_specificationlist block-
allocation_strategyrequired - string -
block_duration_minutesoptional - number -
timeout_actionrequired - string -
timeout_duration_minutesrequired - number
-
-
-
-
core_instance_grouplist block-
autoscaling_policyoptional - string -
bid_priceoptional - string -
idoptional computed - string -
instance_countoptional - number -
instance_typerequired - string -
nameoptional - string -
ebs_configset block-
iopsoptional - number -
sizerequired - number -
typerequired - string -
volumes_per_instanceoptional - number
-
-
-
ec2_attributeslist block-
additional_master_security_groupsoptional - string -
additional_slave_security_groupsoptional - string -
emr_managed_master_security_groupoptional computed - string -
emr_managed_slave_security_groupoptional computed - string -
instance_profilerequired - string -
key_nameoptional - string -
service_access_security_groupoptional computed - string -
subnet_idoptional computed - string -
subnet_idsoptional computed - set of string
-
-
kerberos_attributeslist block-
ad_domain_join_passwordoptional - string -
ad_domain_join_useroptional - string -
cross_realm_trust_principal_passwordoptional - string -
kdc_admin_passwordrequired - string -
realmrequired - string
-
-
master_instance_fleetlist block-
idoptional computed - string -
nameoptional - string -
provisioned_on_demand_capacityoptional computed - number -
provisioned_spot_capacityoptional computed - number -
target_on_demand_capacityoptional - number -
target_spot_capacityoptional - number -
instance_type_configsset block-
bid_priceoptional - string -
bid_price_as_percentage_of_on_demand_priceoptional - number -
instance_typerequired - string -
weighted_capacityoptional - number -
configurationsset block-
classificationoptional - string -
propertiesoptional - map from string to string
-
-
ebs_configset block-
iopsoptional - number -
sizerequired - number -
typerequired - string -
volumes_per_instanceoptional - number
-
-
-
launch_specificationslist block-
on_demand_specificationlist block-
allocation_strategyrequired - string
-
-
spot_specificationlist block-
allocation_strategyrequired - string -
block_duration_minutesoptional - number -
timeout_actionrequired - string -
timeout_duration_minutesrequired - number
-
-
-
-
master_instance_grouplist block-
bid_priceoptional - string -
idoptional computed - string -
instance_countoptional - number -
instance_typerequired - string -
nameoptional - string -
ebs_configset block-
iopsoptional - number -
sizerequired - number -
typerequired - string -
volumes_per_instanceoptional - number
-
-
Explanation in Terraform Registry
Provides an Elastic MapReduce Cluster, a web service that makes it easy to process large amounts of data efficiently. See Amazon Elastic MapReduce Documentation for more information. To configure Instance Groups for task nodes, see the
aws_emr_instance_groupresource.
AWS::EMR::Cluster (CloudFormation)
The Cluster in EMR can be configured in CloudFormation with the resource name AWS::EMR::Cluster. The following sections describe 10 examples of how to use the resource and its parameters.
Example Usage from GitHub
Type: 'AWS::EMR::Cluster'
crmTransformationPipelineStepFunctions:
Type: 'AWS::StepFunctions::StateMachine'
Properties:
RoleArn: ''
Type: 'AWS::EMR::Cluster'
crmTransformationPipelineStepFunctions:
Type: 'AWS::StepFunctions::StateMachine'
Properties:
RoleArn: ''
Type: AWS::EMR::Cluster
Properties:
Instances:
MasterInstanceGroup:
InstanceCount: 1
InstanceType: m1.medium
Type: AWS::EMR::Cluster
Properties:
Instances:
MasterInstanceGroup:
InstanceCount: 1
InstanceType: m1.medium
Type: 'AWS::EMR::Cluster'
Properties:
Instances:
MasterInstanceGroup:
InstanceCount: 1
InstanceType: m5.xlarge
"Type": "AWS::EMR::Cluster",
"Properties": {
"Instances": {
"Ec2KeyName" : "ukmyjorgeawstestkeypair",
"HadoopVersion" : "Amazon 2.8.5",
"MasterInstanceGroup": {
"resourceType" : "AWS::EMR::Cluster",
"properties" : [ {
"propertyName" : "AdditionalInfo",
"propertyType" : "JsonObject",
"required" : false
}, {
"Type": "AWS::EMR::Cluster",
"Properties": {
"Instances": {
"MasterInstanceGroup": {
"InstanceCount": 1,
"InstanceType": "m1.medium",
"Type": "AWS::EMR::Cluster",
"Properties": {
"Instances": {
"MasterInstanceGroup": {
"InstanceCount": 1,
"InstanceType": "m1.medium",
"Type":"AWS::EMR::Cluster::InstanceType"
},
"CoreInstanceType":{
"Description":"Instance type to be used for core instances.",
"Type":"String",
"AllowedValues":[
Parameters
-
AdditionalInfooptional - Json -
Applicationsoptional - List of Application -
AutoScalingRoleoptional - String -
BootstrapActionsoptional - List of BootstrapActionConfig -
Configurationsoptional - List of Configuration -
CustomAmiIdoptional - String -
EbsRootVolumeSizeoptional - Integer -
Instancesrequired - JobFlowInstancesConfig -
JobFlowRolerequired - String -
KerberosAttributesoptional - KerberosAttributes -
LogEncryptionKmsKeyIdoptional - String -
LogUrioptional - String -
ManagedScalingPolicyoptional - ManagedScalingPolicy -
Namerequired - String -
ReleaseLabeloptional - String -
ScaleDownBehavioroptional - String -
SecurityConfigurationoptional - String -
ServiceRolerequired - String -
StepConcurrencyLeveloptional - Integer -
Stepsoptional - List of StepConfig -
Tagsoptional - List of Tag -
VisibleToAllUsersoptional - Boolean
Explanation in CloudFormation Registry
The
AWS::EMR::Clusterresource specifies an Amazon EMR cluster. This cluster is a collection of Amazon EC2 instances that run open source big data frameworks and applications to process and analyze vast amounts of data. For more information, see the Amazon EMR Management Guide.
Frequently asked questions
What is AWS Amazon EMR Cluster?
AWS Amazon EMR Cluster is a resource for Amazon EMR of Amazon Web Service. Settings can be wrote in Terraform and CloudFormation.
Where can I find the example code for the AWS Amazon EMR Cluster?
For Terraform, the SnidermanIndustries/checkov-fork and melscoop-test/check source code examples are useful. See the Terraform Example section for further details.
For CloudFormation, the demeritiusg/data_warehouse-pipelines, demeritiusg/AWS-Examples and stelligent/cfn_nag source code examples are useful. See the CloudFormation Example section for further details.