AWS Amazon MSK Cluster
This page shows how to write Terraform and CloudFormation for Amazon MSK Cluster and write them securely.
aws_msk_cluster (Terraform)
The Cluster in Amazon MSK can be configured in Terraform with the resource name aws_msk_cluster. The following sections describe 5 examples of how to use the resource and its parameters.
Example Usage from GitHub
resource "aws_msk_cluster" "allowed" {
logging_info {
broker_logs {
cloudwatch_logs {
enabled = true
}
resource "aws_msk_cluster" "positive1" {
cluster_name = "example"
kafka_version = "2.4.1"
number_of_broker_nodes = 3
}
resource "aws_msk_cluster" "negative1" {
encryption_info {
encryption_at_rest_kms_key_arn = aws_kms_key.kms.arn
}
}
resource "aws_msk_cluster" "sensitive_data_cluster_1" {
encryption_info {
encryption_in_transit {
client_broker = "PLAINTEXT" # Noncompliant
}
}
resource "aws_msk_cluster" "sensitive_msk" {
cluster_name = "sensitive_msk"
}
resource "aws_msk_cluster" "sensitive_msk" {
cluster_name = "sensitive_msk"
Parameters
-
arnoptional computed - string -
bootstrap_brokersoptional computed - string -
bootstrap_brokers_sasl_scramoptional computed - string -
bootstrap_brokers_tlsoptional computed - string -
cluster_namerequired - string -
current_versionoptional computed - string -
enhanced_monitoringoptional - string -
idoptional computed - string -
kafka_versionrequired - string -
number_of_broker_nodesrequired - number -
tagsoptional - map from string to string -
zookeeper_connect_stringoptional computed - string -
broker_node_group_infolist block-
az_distributionoptional - string -
client_subnetsrequired - list of string -
ebs_volume_sizerequired - number -
instance_typerequired - string -
security_groupsrequired - list of string
-
-
client_authenticationlist block-
sasllist block-
scramoptional - bool
-
-
tlslist block-
certificate_authority_arnsoptional - set of string
-
-
-
configuration_infolist block -
encryption_infolist block-
encryption_at_rest_kms_key_arnoptional computed - string -
encryption_in_transitlist block-
client_brokeroptional - string -
in_clusteroptional - bool
-
-
-
logging_infolist block-
broker_logslist block-
cloudwatch_logslist block -
firehoselist block-
delivery_streamoptional - string -
enabledrequired - bool
-
-
s3list block
-
-
-
open_monitoringlist block-
prometheuslist block-
jmx_exporterlist block-
enabled_in_brokerrequired - bool
-
-
node_exporterlist block-
enabled_in_brokerrequired - bool
-
-
-
Explanation in Terraform Registry
Manages AWS Managed Streaming for Kafka cluster
AWS::MSK::Cluster (CloudFormation)
The Cluster in MSK can be configured in CloudFormation with the resource name AWS::MSK::Cluster. The following sections describe 10 examples of how to use the resource and its parameters.
Example Usage from GitHub
Type: AWS::MSK::Cluster
Condition: MTLSMSKCluster1
Properties:
BrokerNodeGroupInfo:
ClientSubnets:
- Fn::ImportValue:
Type: AWS::MSK::Cluster
Properties:
ClusterName: KafkaForWaterstream
KafkaVersion: !Ref KafkaVersion
NumberOfBrokerNodes: !Ref KafkaNodes
EnhancedMonitoring: DEFAULT
Type: "AWS::MSK::Cluster"
Properties:
BrokerNodeGroupInfo:
ClientSubnets: !Ref SubnetIds
InstanceType: "kafka.m5.large"
SecurityGroups:
Type: 'AWS::MSK::Cluster'
Properties:
ClusterName: spn-kafka-cluster
NumberOfBrokerNodes: 3
KafkaVersion: "2.2.1"
BrokerNodeGroupInfo:
Type: AWS::MSK::Cluster
Properties:
ClusterName: !Ref ClusterName
KafkaVersion: 2.2.1
NumberOfBrokerNodes: 2
EnhancedMonitoring: PER_BROKER
"Type": "AWS::MSK::Cluster",
"Properties": {
"ClusterName": "ClusterWithRequiredProperties",
"KafkaVersion": "2.2.1",
"NumberOfBrokerNodes": 3,
"BrokerNodeGroupInfo": {
"Type": "AWS::MSK::Cluster",
"Properties": {
"ClusterName": "ClusterWithRequiredProperties",
"KafkaVersion": "2.2.1",
"NumberOfBrokerNodes": 3,
"BrokerNodeGroupInfo": {
"Type": "AWS::MSK::Cluster",
"Properties": {
"ClusterName": "ClusterWithRequiredProperties",
"KafkaVersion": "2.2.1",
"LoggingInfo": {
"BrokerLogs": {
{ "LogicalResourceId":"MyMskCluster", "ResourceType":"AWS::MSK::Cluster" },
{ "LogicalResourceId":"MyMskStreamProcessorMyMskEvent", "ResourceType":"AWS::Lambda::EventSourceMapping" }
{ "LogicalResourceId":"MyMskCluster", "ResourceType":"AWS::MSK::Cluster" },
{ "LogicalResourceId":"MyMskStreamProcessorMyMskEvent", "ResourceType":"AWS::Lambda::EventSourceMapping" }
Parameters
-
BrokerNodeGroupInforequired - BrokerNodeGroupInfo -
EnhancedMonitoringoptional - String -
KafkaVersionrequired - String -
NumberOfBrokerNodesrequired - Integer -
EncryptionInfooptional - EncryptionInfo -
OpenMonitoringoptional - OpenMonitoring -
ClusterNamerequired - String -
ClientAuthenticationoptional - ClientAuthentication -
LoggingInfooptional - LoggingInfo -
Tagsoptional - Json -
ConfigurationInfooptional - ConfigurationInfo
Explanation in CloudFormation Registry
The
AWS::MSK::Clusterresource creates an Amazon MSK cluster. For more information, see What Is Amazon MSK? in the Amazon MSK Developer Guide.
Frequently asked questions
What is AWS Amazon MSK Cluster?
AWS Amazon MSK Cluster is a resource for Amazon MSK of Amazon Web Service. Settings can be wrote in Terraform and CloudFormation.
Where can I find the example code for the AWS Amazon MSK Cluster?
For Terraform, the snyk-labs/infrastructure-as-code-goof, leonidweinbergcx/mykics and leonidweinbergcx/mykics source code examples are useful. See the Terraform Example section for further details.
For CloudFormation, the aws-samples/mirrormaker2-msk-migration, simplematter/waterstream-aws-ecs-cloudformation and michael-robbins/aws-streaming-session source code examples are useful. See the CloudFormation Example section for further details.
