Azure HDInsight Hadoop Cluster
This page shows how to write Terraform and Azure Resource Manager for HDInsight Hadoop Cluster and write them securely.
azurerm_hdinsight_hadoop_cluster (Terraform)
The Hadoop Cluster in HDInsight can be configured in Terraform with the resource name azurerm_hdinsight_hadoop_cluster
. The following sections describe 6 examples of how to use the resource and its parameters.
Example Usage from GitHub
resource "azurerm_hdinsight_hadoop_cluster" "with_edge" {
name = "example-hdicluster"
resource_group_name = azurerm_resource_group.example.name
location = "eastus"
cluster_version = "3.6"
tier = "Standard"
resource "azurerm_hdinsight_hadoop_cluster" "demohadoop" {
name = var.hadoopclus_name
resource_group_name = var.rg_name
location = var.rg_loc
cluster_version = var.clus_version
tier = var.clus_tier
resource "azurerm_hdinsight_hadoop_cluster" "with_edge" {
name = "example-hdicluster"
resource_group_name = azurerm_resource_group.example.name
location = "eastus"
cluster_version = "3.6"
tier = "Standard"
resource "azurerm_hdinsight_hadoop_cluster" "this" {
cluster_version = var.cluster_version
location = var.location
name = var.name
resource_group_name = var.resource_group_name
tags = var.tags
resource "azurerm_hdinsight_hadoop_cluster" "this" {
cluster_version = var.cluster_version
location = var.location
name = var.name
resource_group_name = var.resource_group_name
tags = var.tags
resource "azurerm_hdinsight_hadoop_cluster" "example" {
name = "projet-hdicluster"
resource_group_name = azurerm_resource_group.rg.name
location = azurerm_resource_group.rg.location
cluster_version = "3.6"
tier = "Standard"
Parameters
-
cluster_version
required - string -
https_endpoint
optional computed - string -
id
optional computed - string -
location
required - string -
name
required - string -
resource_group_name
required - string -
ssh_endpoint
optional computed - string -
tags
optional - map from string to string -
tier
required - string -
tls_min_version
optional - string -
component_version
list block-
hadoop
required - string
-
-
gateway
list block -
metastores
list block-
ambari
list block-
database_name
required - string -
password
required - string -
server
required - string -
username
required - string
-
-
hive
list block-
database_name
required - string -
password
required - string -
server
required - string -
username
required - string
-
-
oozie
list block-
database_name
required - string -
password
required - string -
server
required - string -
username
required - string
-
-
-
monitor
list block-
log_analytics_workspace_id
required - string -
primary_key
required - string
-
-
roles
list block-
edge_node
list block-
target_instance_count
required - number -
vm_size
required - string -
install_script_action
list block
-
-
head_node
list block -
worker_node
list block-
min_instance_count
optional computed - number -
password
optional - string -
ssh_keys
optional - set of string -
subnet_id
optional - string -
target_instance_count
required - number -
username
required - string -
virtual_network_id
optional - string -
vm_size
required - string
-
-
zookeeper_node
list block
-
-
storage_account
list block-
is_default
required - bool -
storage_account_key
required - string -
storage_container_id
required - string
-
-
storage_account_gen2
list block-
filesystem_id
required - string -
is_default
required - bool -
managed_identity_resource_id
required - string -
storage_resource_id
required - string
-
-
timeouts
single block
Explanation in Terraform Registry
Manages a HDInsight Hadoop Cluster.
Microsoft.HDInsight/clusters (Azure Resource Manager)
The clusters in Microsoft.HDInsight can be configured in Azure Resource Manager with the resource name Microsoft.HDInsight/clusters
. The following sections describe how to use the resource and its parameters.
Example Usage from GitHub
"type": "Microsoft.HDInsight/clusters",
"location": "West US",
"etag": "f0212a39-b827-45e0-9ffa-4f5232e58851",
"tags": { "key1": "val1" },
"properties": {
"clusterVersion": "3.5.1000.0",
"type": "Microsoft.HDInsight/clusters",
"location": "West US",
"etag": "f0212a39-b827-45e0-9ffa-4f5232e58851",
"tags": { "key1": "val1" },
"properties": {
"clusterVersion": "3.5.1000.0",
"type": "Microsoft.HDInsight/clusters",
"location": "West US",
"etag": "f0212a39-b827-45e0-9ffa-4f5232e58851",
"tags": { "key1": "val1" },
"properties": {
"clusterVersion": "3.5.1000.0",
"type": "Microsoft.HDInsight/clusters",
"location": "West US",
"etag": "f0212a39-b827-45e0-9ffa-4f5232e58851",
"tags": { "key1": "val1" },
"properties": {
"clusterVersion": "3.5.1000.0",
"type": "Microsoft.HDInsight/clusters",
"location": "West US",
"etag": "f0212a39-b827-45e0-9ffa-4f5232e58851",
"tags": { "key1": "val1" },
"properties": {
"clusterVersion": "3.5.1000.0",
"type": "Microsoft.HDInsight/clusters",
"location": "West US",
"etag": "f0212a39-b827-45e0-9ffa-4f5232e58851",
"tags": { "key1": "val1" },
"properties": {
"clusterVersion": "3.5.1000.0",
"type": "Microsoft.HDInsight/clusters",
"location": "West US",
"etag": "f0212a39-b827-45e0-9ffa-4f5232e58851",
"tags": { "key1": "val1" },
"properties": {
"clusterVersion": "3.5.1000.0",
"type": "Microsoft.HDInsight/clusters",
"location": "West US",
"etag": "f0212a39-b827-45e0-9ffa-4f5232e58851",
"tags": { "key1": "val1" },
"properties": {
"clusterVersion": "3.5.1000.0",
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"clusterName": {
"type": "Microsoft.HDInsight/clusters",
"location": "West US",
"etag": "f0212a39-b827-45e0-9ffa-4f5232e58851",
"tags": {
"key1": "val1"
},
Parameters
apiVersion
required - stringidentity
optionaltype
optional - stringThe type of identity used for the cluster. The type 'SystemAssigned, UserAssigned' includes both an implicitly created identity and a set of user assigned identities.
userAssignedIdentities
optional - undefinedThe list of user identities associated with the cluster. The user identity dictionary key references will be ARM resource ids in the form: '/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.ManagedIdentity/userAssignedIdentities/{identityName}'.
location
optional - stringThe location of the cluster.
name
required - stringThe name of the cluster.
properties
requiredclusterDefinition
optionalblueprint
optional - stringThe link to the blueprint.
componentVersion
optional - stringThe versions of different services in the cluster.
configurations
optional - objectThe cluster configurations.
kind
optional - stringThe type of cluster.
clusterVersion
optional - stringThe version of the cluster.
computeIsolationProperties
optionalenableComputeIsolation
optional - booleanThe flag indicates whether enable compute isolation or not.
hostSku
optional - stringThe host sku.
computeProfile
optionalroles
optional arrayautoscale
optionalcapacity
optionalmaxInstanceCount
optional - integerThe maximum instance count of the cluster
minInstanceCount
optional - integerThe minimum instance count of the cluster
recurrence
optionalschedule
optional arraydays
optional - arrayDays of the week for a schedule-based autoscale rule
timeAndCapacity
optionalmaxInstanceCount
optional - integerThe maximum instance count of the cluster
minInstanceCount
optional - integerThe minimum instance count of the cluster
time
optional - string24-hour time in the form xx:xx
timeZone
optional - stringThe time zone for the autoscale schedule times
dataDisksGroups
optional arraydisksPerNode
optional - integerThe number of disks per node.
encryptDataDisks
optional - booleanIndicates whether encrypt the data disks.
hardwareProfile
optionalvmSize
optional - stringThe size of the VM
minInstanceCount
optional - integerThe minimum instance count of the cluster.
name
optional - stringThe name of the role.
osProfile
optionallinuxOperatingSystemProfile
optionalpassword
optional - stringThe password.
sshProfile
optionalpublicKeys
optional arraycertificateData
optional - stringThe certificate for SSH.
username
optional - stringThe username.
scriptActions
optional arrayname
required - stringThe name of the script action.
parameters
required - stringThe parameters for the script provided.
uri
required - stringThe URI to the script.
targetInstanceCount
optional - integerThe instance count of the cluster.
virtualNetworkProfile
optionalid
optional - stringThe ID of the virtual network.
subnet
optional - stringThe name of the subnet.
VMGroupName
optional - stringThe name of the virtual machine group.
diskEncryptionProperties
optionalencryptionAlgorithm
optional - stringAlgorithm identifier for encryption, default RSA-OAEP.
encryptionAtHost
optional - booleanIndicates whether or not resource disk encryption is enabled.
keyName
optional - stringKey name that is used for enabling disk encryption.
keyVersion
optional - stringSpecific key version that is used for enabling disk encryption.
msiResourceId
optional - stringResource ID of Managed Identity that is used to access the key vault.
vaultUri
optional - stringBase key vault URI where the customers key is located eg. https://myvault.vault.azure.net
encryptionInTransitProperties
optionalisEncryptionInTransitEnabled
optional - booleanIndicates whether or not inter cluster node communication is encrypted in transit.
kafkaRestProperties
optionalclientGroupInfo
optionalgroupId
optional - stringThe AAD security group id.
groupName
optional - stringThe AAD security group name.
configurationOverride
optional - stringThe configurations that need to be overriden.
minSupportedTlsVersion
optional - stringThe minimal supported tls version.
networkProperties
optionalprivateLink
optional - stringIndicates whether or not private link is enabled.
resourceProviderConnection
optional - stringThe direction for the resource provider connection.
osType
optional - stringThe type of operating system.
privateLinkConfigurations
optional arrayname
required - stringThe name of private link configuration.
properties
requiredgroupId
required - stringThe HDInsight private linkable sub-resource name to apply the private link configuration to. For example, 'headnode', 'gateway', 'edgenode'.
ipConfigurations
required arrayname
required - stringThe name of private link IP configuration.
properties
optionalprimary
optional - booleanIndicates whether this IP configuration is primary for the corresponding NIC.
privateIPAddress
optional - stringThe IP address.
privateIPAllocationMethod
optional - stringThe method that private IP address is allocated.
subnet
optionalid
optional - stringThe azure resource id.
securityProfile
optionalaaddsResourceId
optional - stringThe resource ID of the user's Azure Active Directory Domain Service.
clusterUsersGroupDNs
optional - arrayOptional. The Distinguished Names for cluster user groups
directoryType
optional - stringThe directory type.
domain
optional - stringThe organization's active directory domain.
domainUsername
optional - stringThe domain user account that will have admin privileges on the cluster.
domainUserPassword
optional - stringThe domain admin password.
ldapsUrls
optional - arrayThe LDAPS protocol URLs to communicate with the Active Directory.
msiResourceId
optional - stringUser assigned identity that has permissions to read and create cluster-related artifacts in the user's AADDS.
organizationalUnitDN
optional - stringThe organizational unit within the Active Directory to place the cluster and service accounts.
storageProfile
optionalstorageaccounts
optional arraycontainer
optional - stringThe container in the storage account, only to be specified for WASB storage accounts.
fileshare
optional - stringThe file share name.
fileSystem
optional - stringThe filesystem, only to be specified for Azure Data Lake Storage Gen 2.
isDefault
optional - booleanWhether or not the storage account is the default storage account.
key
optional - stringThe storage account access key.
msiResourceId
optional - stringThe managed identity (MSI) that is allowed to access the storage account, only to be specified for Azure Data Lake Storage Gen 2.
name
optional - stringThe name of the storage account.
resourceId
optional - stringThe resource ID of storage account, only to be specified for Azure Data Lake Storage Gen 2.
saskey
optional - stringThe shared access signature key.
tier
optional - stringThe cluster tier.
tags
optional - stringThe resource tags.
type
required - stringzones
optional - arrayThe availability zones.
Frequently asked questions
What is Azure HDInsight Hadoop Cluster?
Azure HDInsight Hadoop Cluster is a resource for HDInsight of Microsoft Azure. Settings can be wrote in Terraform.
Where can I find the example code for the Azure HDInsight Hadoop Cluster?
For Terraform, the gilyas/infracost, sowmya3000/Terraform and infracost/infracost source code examples are useful. See the Terraform Example section for further details.
For Azure Resource Manager, the assing/alerts-extension, assing/alerts-extension and assing/alerts-extension source code examples are useful. See the Azure Resource Manager Example section for further details.