Azure Storage Data Lake Gen2 Filesystem

This page shows how to write Terraform for Storage Data Lake Gen2 Filesystem and write them securely.

azurerm_storage_data_lake_gen2_filesystem (Terraform)

The Data Lake Gen2 Filesystem in Storage can be configured in Terraform with the resource name azurerm_storage_data_lake_gen2_filesystem. The following sections describe 10 examples of how to use the resource and its parameters.

Example Usage from GitHub

main.tf#L34
resource "azurerm_storage_data_lake_gen2_filesystem" "raw" {
  name               = "raw"
  storage_account_id = azurerm_storage_account.storageacc.id
  lifecycle {
    prevent_destroy = true
  }
main.tf#L13
resource "azurerm_storage_data_lake_gen2_filesystem" "lake" {
  count              = length(var.container-name)
  name               = var.container-name[count.index]
  storage_account_id = data.azurerm_storage_account.DL.id

  #############Function APP#############################
filesytems.tf#L1
resource "azurerm_storage_data_lake_gen2_filesystem" "datalake_filesystem" {
  count              = length(var.datalake_filesystem_name)
  name               = element(var.datalake_filesystem_name, count.index)
  storage_account_id = azurerm_storage_account.datalake_storage.id
main.tf#L1
resource "azurerm_storage_data_lake_gen2_filesystem" "gen2_filesystem" {
  name               = var.filesystem_name
  storage_account_id = var.storage_account_id
main.tf#L2
resource "azurerm_storage_data_lake_gen2_filesystem" "filesystems_source" {
  for_each           = toset(["raw", "curated", "landing", "normalized"])
  name               = lower(each.value)
  storage_account_id = data.azurerm_storage_account.storage_source.id
}
storage.tf#L18
resource "azurerm_storage_data_lake_gen2_filesystem" "aks-datalake" {
  name               = var.datalake_name
  storage_account_id = azurerm_storage_account.aks-sa.id
}

resource "azurerm_storage_data_lake_gen2_filesystem" "aks-airflow-logs" {
main.tf#L10
resource "azurerm_storage_data_lake_gen2_filesystem" "data_lake_gen2" {
  # count                    = local.data_lake_create_flag
  name               = var.data_lake_name
  storage_account_id = var.storage_account_id
  # properties               = var.resource_tags
}
gen2_filesystem.tf#L1
resource "azurerm_storage_data_lake_gen2_filesystem" "gen2" {
  name               = var.settings.name
  storage_account_id = var.storage_account_id
  properties = {
    for key, value in try(var.settings.properties) : key => base64encode(value)
  }
gen2_filesystem.tf#L1
resource "azurerm_storage_data_lake_gen2_filesystem" "gen2" {
  name               = var.settings.name
  storage_account_id = var.storage_account_id
  properties = {
    for key, value in try(var.settings.properties) : key => base64encode(value)
  }
gen2_filesystem.tf#L1
resource "azurerm_storage_data_lake_gen2_filesystem" "gen2" {
  name               = var.settings.name
  storage_account_id = var.storage_account_id
  properties = {
    for key, value in try(var.settings.properties) : key => base64encode(value)
  }

Review your Terraform file for Azure best practices

Shisho Cloud, our free checker to make sure your Terraform configuration follows best practices, is available (beta).

Parameters

Explanation in Terraform Registry

Manages a Data Lake Gen2 File System within an Azure Storage Account.

NOTE: This resource requires some Storage specific roles which are not granted by default. Some of the built-ins roles that can be attributed are Storage Account Contributor, Storage Blob Data Owner, Storage Blob Data Contributor, Storage Blob Data Reader.

Tips: Best Practices for The Other Azure Storage Resources

In addition to the azurerm_storage_account, Azure Storage has the other resources that should be configured for security reasons. Please check some examples of those resources and precautions.

risk-label

azurerm_storage_account

Ensure to use HTTPS connections

It is better to use HTTPS instead of HTTP, which could be vulnerable to person-in-the-middle attacks.

risk-label

azurerm_storage_account_network_rules

Ensure to allow Trusted Microsoft Services to bypass

It is better to allow Trusted Microsoft Services to bypass. They are not able to access storage account unless rules are set to allow them explicitly.

Review your Azure Storage settings

In addition to the above, there are other security points you should be aware of making sure that your .tf files are protected in Shisho Cloud.

Azure Resource Manager Example

Azure Resource Manager code does not have the related resource.

Frequently asked questions

What is Azure Storage Data Lake Gen2 Filesystem?

Azure Storage Data Lake Gen2 Filesystem is a resource for Storage of Microsoft Azure. Settings can be wrote in Terraform.

Where can I find the example code for the Azure Storage Data Lake Gen2 Filesystem?

For Terraform, the ClimateImpactLab/downscaleCMIP6, Dmeehan81/Terraform-DataLake and shaikis/az_terraform_modules source code examples are useful. See the Terraform Example section for further details.