AWS Kinesis Data Firehose Firehose Delivery Stream
This page shows how to write Terraform and CloudFormation for Kinesis Data Firehose Firehose Delivery Stream and write them securely.
aws_kinesis_firehose_delivery_stream (Terraform)
The Firehose Delivery Stream in Kinesis Data Firehose can be configured in Terraform with the resource name aws_kinesis_firehose_delivery_stream
. The following sections describe 3 examples of how to use the resource and its parameters.
Example Usage from GitHub
resource "aws_kinesis_firehose_delivery_stream" "withAllTags" {
name = "terraform-kinesis-firehose-test-stream"
destination = "splunk"
extended_s3_configuration {
role_arn = aws_iam_role.firehose.arn
bucket_arn = aws_s3_bucket.bucket.arn
resource "aws_kinesis_firehose_delivery_stream" "kinesisSseEnabled" {
name = "kinesisSseEnabled"
destination = "s3"
server_side_encryption {
enabled = true
resource "aws_kinesis_firehose_delivery_stream" "user_event_stream" {
count = terraform.workspace == "streaming" ? 1 : 0
name = "kinesis-firehose-extended-s3-user-event-stream"
destination = "extended_s3"
Parameters
-
arn
optional computed - string -
destination
required - string -
destination_id
optional computed - string -
id
optional computed - string -
name
required - string -
tags
optional - map from string to string -
version_id
optional computed - string -
elasticsearch_configuration
list block-
buffering_interval
optional - number -
buffering_size
optional - number -
cluster_endpoint
optional - string -
domain_arn
optional - string -
index_name
required - string -
index_rotation_period
optional - string -
retry_duration
optional - number -
role_arn
required - string -
s3_backup_mode
optional - string -
type_name
optional - string -
cloudwatch_logging_options
list block-
enabled
optional - bool -
log_group_name
optional - string -
log_stream_name
optional - string
-
-
processing_configuration
list block-
enabled
optional - bool -
processors
list block-
type
required - string -
parameters
list block-
parameter_name
required - string -
parameter_value
required - string
-
-
-
-
vpc_config
list block-
role_arn
required - string -
security_group_ids
required - set of string -
subnet_ids
required - set of string -
vpc_id
optional computed - string
-
-
-
extended_s3_configuration
list block-
bucket_arn
required - string -
buffer_interval
optional - number -
buffer_size
optional - number -
compression_format
optional - string -
error_output_prefix
optional - string -
kms_key_arn
optional - string -
prefix
optional - string -
role_arn
required - string -
s3_backup_mode
optional - string -
cloudwatch_logging_options
list block-
enabled
optional - bool -
log_group_name
optional - string -
log_stream_name
optional - string
-
-
data_format_conversion_configuration
list block-
enabled
optional - bool -
input_format_configuration
list block-
deserializer
list block-
hive_json_ser_de
list block-
timestamp_formats
optional - list of string
-
-
open_x_json_ser_de
list block-
case_insensitive
optional - bool -
column_to_json_key_mappings
optional - map from string to string -
convert_dots_in_json_keys_to_underscores
optional - bool
-
-
-
-
output_format_configuration
list block-
serializer
list block-
orc_ser_de
list block-
block_size_bytes
optional - number -
bloom_filter_columns
optional - list of string -
bloom_filter_false_positive_probability
optional - number -
compression
optional - string -
dictionary_key_threshold
optional - number -
enable_padding
optional - bool -
format_version
optional - string -
padding_tolerance
optional - number -
row_index_stride
optional - number -
stripe_size_bytes
optional - number
-
-
parquet_ser_de
list block-
block_size_bytes
optional - number -
compression
optional - string -
enable_dictionary_compression
optional - bool -
max_padding_bytes
optional - number -
page_size_bytes
optional - number -
writer_version
optional - string
-
-
-
-
schema_configuration
list block-
catalog_id
optional computed - string -
database_name
required - string -
region
optional computed - string -
role_arn
required - string -
table_name
required - string -
version_id
optional - string
-
-
-
processing_configuration
list block-
enabled
optional - bool -
processors
list block-
type
required - string -
parameters
list block-
parameter_name
required - string -
parameter_value
required - string
-
-
-
-
s3_backup_configuration
list block-
bucket_arn
required - string -
buffer_interval
optional - number -
buffer_size
optional - number -
compression_format
optional - string -
kms_key_arn
optional - string -
prefix
optional - string -
role_arn
required - string -
cloudwatch_logging_options
list block-
enabled
optional - bool -
log_group_name
optional - string -
log_stream_name
optional - string
-
-
-
-
http_endpoint_configuration
list block-
access_key
optional - string -
buffering_interval
optional - number -
buffering_size
optional - number -
name
optional - string -
retry_duration
optional - number -
role_arn
optional - string -
s3_backup_mode
optional - string -
url
required - string -
cloudwatch_logging_options
list block-
enabled
optional - bool -
log_group_name
optional - string -
log_stream_name
optional - string
-
-
processing_configuration
list block-
enabled
optional - bool -
processors
list block-
type
required - string -
parameters
list block-
parameter_name
required - string -
parameter_value
required - string
-
-
-
-
request_configuration
list block-
content_encoding
optional - string -
common_attributes
list block
-
-
-
kinesis_source_configuration
list block-
kinesis_stream_arn
required - string -
role_arn
required - string
-
-
redshift_configuration
list block-
cluster_jdbcurl
required - string -
copy_options
optional - string -
data_table_columns
optional - string -
data_table_name
required - string -
password
required - string -
retry_duration
optional - number -
role_arn
required - string -
s3_backup_mode
optional - string -
username
required - string -
cloudwatch_logging_options
list block-
enabled
optional - bool -
log_group_name
optional - string -
log_stream_name
optional - string
-
-
processing_configuration
list block-
enabled
optional - bool -
processors
list block-
type
required - string -
parameters
list block-
parameter_name
required - string -
parameter_value
required - string
-
-
-
-
s3_backup_configuration
list block-
bucket_arn
required - string -
buffer_interval
optional - number -
buffer_size
optional - number -
compression_format
optional - string -
kms_key_arn
optional - string -
prefix
optional - string -
role_arn
required - string -
cloudwatch_logging_options
list block-
enabled
optional - bool -
log_group_name
optional - string -
log_stream_name
optional - string
-
-
-
-
s3_configuration
list block-
bucket_arn
required - string -
buffer_interval
optional - number -
buffer_size
optional - number -
compression_format
optional - string -
kms_key_arn
optional - string -
prefix
optional - string -
role_arn
required - string -
cloudwatch_logging_options
list block-
enabled
optional - bool -
log_group_name
optional - string -
log_stream_name
optional - string
-
-
-
server_side_encryption
list block -
splunk_configuration
list block-
hec_acknowledgment_timeout
optional - number -
hec_endpoint
required - string -
hec_endpoint_type
optional - string -
hec_token
required - string -
retry_duration
optional - number -
s3_backup_mode
optional - string -
cloudwatch_logging_options
list block-
enabled
optional - bool -
log_group_name
optional - string -
log_stream_name
optional - string
-
-
processing_configuration
list block-
enabled
optional - bool -
processors
list block-
type
required - string -
parameters
list block-
parameter_name
required - string -
parameter_value
required - string
-
-
-
-
Explanation in Terraform Registry
Provides a Kinesis Firehose Delivery Stream resource. Amazon Kinesis Firehose is a fully managed, elastic service to easily deliver real-time data streams to destinations such as Amazon S3 and Amazon Redshift. For more details, see the [Amazon Kinesis Firehose Documentation][1].
AWS::KinesisFirehose::DeliveryStream (CloudFormation)
The DeliveryStream in KinesisFirehose can be configured in CloudFormation with the resource name AWS::KinesisFirehose::DeliveryStream
. The following sections describe how to use the resource and its parameters.
Example Usage from GitHub
An example could not be found in GitHub.
Parameters
-
DeliveryStreamEncryptionConfigurationInput
optional - DeliveryStreamEncryptionConfigurationInput -
DeliveryStreamName
optional - String -
DeliveryStreamType
optional - String -
ElasticsearchDestinationConfiguration
optional - ElasticsearchDestinationConfiguration -
AmazonopensearchserviceDestinationConfiguration
optional - AmazonopensearchserviceDestinationConfiguration -
ExtendedS3DestinationConfiguration
optional - ExtendedS3DestinationConfiguration -
KinesisStreamSourceConfiguration
optional - KinesisStreamSourceConfiguration -
RedshiftDestinationConfiguration
optional - RedshiftDestinationConfiguration -
S3DestinationConfiguration
optional - S3DestinationConfiguration -
SplunkDestinationConfiguration
optional - SplunkDestinationConfiguration -
HttpEndpointDestinationConfiguration
optional - HttpEndpointDestinationConfiguration -
Tags
optional - List of Tag
Explanation in CloudFormation Registry
The
AWS::KinesisFirehose::DeliveryStream
resource specifies an Amazon Kinesis Data Firehose (Kinesis Data Firehose) delivery stream that delivers real-time streaming data to an Amazon Simple Storage Service (Amazon S3), Amazon Redshift, or Amazon Elasticsearch Service (Amazon ES) destination. For more information, see Creating an Amazon Kinesis Data Firehose Delivery Stream in the Amazon Kinesis Data Firehose Developer Guide.
Frequently asked questions
What is AWS Kinesis Data Firehose Firehose Delivery Stream?
AWS Kinesis Data Firehose Firehose Delivery Stream is a resource for Kinesis Data Firehose of Amazon Web Service. Settings can be wrote in Terraform and CloudFormation.
Where can I find the example code for the AWS Kinesis Data Firehose Firehose Delivery Stream?
For Terraform, the gilyas/infracost, storebot/pr_demo_flat and MundiCollins/multi-account-kinesis-to-redshift source code examples are useful. See the Terraform Example section for further details.