AWS Amazon MSK Cluster

This page shows how to write Terraform and CloudFormation for Amazon MSK Cluster and write them securely.

aws_msk_cluster (Terraform)

The Cluster in Amazon MSK can be configured in Terraform with the resource name aws_msk_cluster. The following sections describe 5 examples of how to use the resource and its parameters.

Example Usage from GitHub

aws_msk_cluster_logs.tf#L1
resource "aws_msk_cluster" "allowed" {
  logging_info {
    broker_logs {
      cloudwatch_logs {
        enabled = true
      }
positive.tf#L1
resource "aws_msk_cluster" "positive1" {
  cluster_name           = "example"
  kafka_version          = "2.4.1"
  number_of_broker_nodes = 3
}

negative.tf#L1
resource "aws_msk_cluster" "negative1" {
  encryption_info {
    encryption_at_rest_kms_key_arn = aws_kms_key.kms.arn
  }
}

test_msk.tf#L1
resource "aws_msk_cluster" "sensitive_data_cluster_1" {
  encryption_info {
    encryption_in_transit {
      client_broker = "PLAINTEXT" # Noncompliant
    }
  }
test_aws_msk_cluster.tf#L2
resource "aws_msk_cluster" "sensitive_msk" {
  cluster_name = "sensitive_msk"
}

resource "aws_msk_cluster" "sensitive_msk" {
  cluster_name = "sensitive_msk"

Review your Terraform file for AWS best practices

Shisho Cloud, our free checker to make sure your Terraform configuration follows best practices, is available (beta).

Parameters

Explanation in Terraform Registry

Manages AWS Managed Streaming for Kafka cluster

AWS::MSK::Cluster (CloudFormation)

The Cluster in MSK can be configured in CloudFormation with the resource name AWS::MSK::Cluster. The following sections describe 10 examples of how to use the resource and its parameters.

Example Usage from GitHub

MSKClusterWithImports.yml#L88
    Type: AWS::MSK::Cluster
    Condition: MTLSMSKCluster1
    Properties:
      BrokerNodeGroupInfo:
        ClientSubnets:
        - Fn::ImportValue:
kafka_template.yml#L73
    Type: AWS::MSK::Cluster
    Properties:
      ClusterName: KafkaForWaterstream
      KafkaVersion: !Ref KafkaVersion
      NumberOfBrokerNodes: !Ref KafkaNodes
      EnhancedMonitoring: DEFAULT
kafka-cluster.yml#L44
    Type: "AWS::MSK::Cluster"
    Properties:
      BrokerNodeGroupInfo:
        ClientSubnets: !Ref SubnetIds
        InstanceType: "kafka.m5.large"
        SecurityGroups:
msk.cfn.yml#L39
    Type: 'AWS::MSK::Cluster'
    Properties:
      ClusterName: spn-kafka-cluster
      NumberOfBrokerNodes: 3
      KafkaVersion: "2.2.1"
      BrokerNodeGroupInfo:
msk.yml#L38
    Type: AWS::MSK::Cluster
    Properties:
      ClusterName: !Ref ClusterName
      KafkaVersion: 2.2.1
      NumberOfBrokerNodes: 2
      EnhancedMonitoring: PER_BROKER
negative3.json#L6
      "Type": "AWS::MSK::Cluster",
      "Properties": {
        "ClusterName": "ClusterWithRequiredProperties",
        "KafkaVersion": "2.2.1",
        "LoggingInfo": {
          "BrokerLogs": {
positive4.json#L6
      "Type": "AWS::MSK::Cluster",
      "Properties": {
        "ClusterName": "ClusterWithRequiredProperties",
        "KafkaVersion": "2.2.1",
        "NumberOfBrokerNodes": 3,
        "BrokerNodeGroupInfo": {
positive4.json#L6
      "Type": "AWS::MSK::Cluster",
      "Properties": {
        "ClusterName": "ClusterWithRequiredProperties",
        "KafkaVersion": "2.2.1",
        "NumberOfBrokerNodes": 3,
        "BrokerNodeGroupInfo": {
function_with_msk_using_managed_policy.json#L4
  { "LogicalResourceId":"MyMskCluster", "ResourceType":"AWS::MSK::Cluster" },
  { "LogicalResourceId":"MyMskStreamProcessorMyMskEvent", "ResourceType":"AWS::Lambda::EventSourceMapping" }
function_with_msk.json#L4
  { "LogicalResourceId":"MyMskCluster", "ResourceType":"AWS::MSK::Cluster" },
  { "LogicalResourceId":"MyMskStreamProcessorMyMskEvent", "ResourceType":"AWS::Lambda::EventSourceMapping" }

Parameters

Explanation in CloudFormation Registry

The AWS::MSK::Cluster resource creates an Amazon MSK cluster. For more information, see What Is Amazon MSK? in the Amazon MSK Developer Guide.

Frequently asked questions

What is AWS Amazon MSK Cluster?

AWS Amazon MSK Cluster is a resource for Amazon MSK of Amazon Web Service. Settings can be wrote in Terraform and CloudFormation.

Where can I find the example code for the AWS Amazon MSK Cluster?

For Terraform, the snyk-labs/infrastructure-as-code-goof, leonidweinbergcx/mykics and leonidweinbergcx/mykics source code examples are useful. See the Terraform Example section for further details.

For CloudFormation, the aws-samples/mirrormaker2-msk-migration, simplematter/waterstream-aws-ecs-cloudformation and michael-robbins/aws-streaming-session source code examples are useful. See the CloudFormation Example section for further details.