Amazon Kinesis Data Stream Integration
A fully managed, distributed data streaming platform to help you collect and store data streams from hundreds of thousands of producers. With Site24x7's integration you can monitor and alert on basic stream level metrics with ease.
Setup and configuration
- If you haven't done it already, enable access to your AWS resource by creating Site24x7 as an IAM user or by creating a cross-account IAM role between your account and Site24x7's AWS account. Learn more.
- Next, In the Integrate AWS Account page, please make sure the Kinesis Data Stream checkbox is selected in the Services to be discovered field. Learn more.
Policies and Permissions
Please make sure the following read level permissions are present in the IAM policy assigned to the Site24x7 IAM entity. Learn more.
- "kinesis:DescribeStreamSummary",
- "kinesis:ListStreams",
- "kinesis:ListTagsForStream",
- "kinesis:DescribeStream"
Polling frequency
Performance metrics are collected as per the poll frequency set (1 minute to a day), by querying the CloudWatch API. Learn more.
IT Automations
You can add automations for the AWS services supported by Site24x7. Log in to Site24x7 and go to Admin > IT Automation Templates (+) > Add Automation Templates. Once automations are added, you can schedule them to be executed one after the other.
You can now push data to kinesis data streams using Amazon Kinesis Data Stream automations.
To view metrics.
- Log in to the Site24x7 console, click on AWS > Monitored AWS account and select Kinesis Data Stream from the drop down
- From the list of monitored data streams, choose the stream for which you want to access metrics for.
- The various graphs applicable for your Amazon Kinesis Data stream are displayed in the Summary tab.
Supported performance metrics
The following basic stream level performance metrics are collected:
| Attribute | Description | Data type | Statistics |
|---|---|---|---|
| GetRecords.Bytes | Measures the number of bytes retrieved from the Kinesis stream. | Bytes | Sum |
| GetRecords.IteratorAgeMilliseconds | Measures the age of the last record in all GetRecords calls made against an Kinesis stream. | Milliseconds | Maximum |
| GetRecords.Latency | Measures the time taken per GetRecords operation. | Milliseconds | Average |
| GetRecords.Records | Measures the number of records retrieved from the shard | Count | Sum |
| GetRecords.Success | Measures the number of successful GetRecords operations per stream. | Count | Average |
| IncomingBytes | Measures the number of bytes successfully put to the Kinesis stream. | Bytes | Sum |
| IncomingRecords | Measures the number of records successfully put to the Kinesis stream. | Count | Sum |
| PutRecord.Bytes | Measures the number of bytes put to the Kinesis stream using the PutRecord operation. | Bytes | Sum |
| PutRecord.Latency | Measures the time taken per PutRecord operation. | Milliseconds | Average |
| PutRecord.Success | Measures the number of successful PutRecord operations per Kinesis stream. | Count | Average |
| PutRecords.Bytes | Measures the number of bytes put to the Kinesis stream using the PutRecords operation | Bytes | Sum |
| PutRecords.Latency | Measures the time taken per PutRecords operation | Milliseconds | Average |
| PutRecords.Records | Measures the number of successful records in a PutRecords operation per Kinesis stream | Count | Sum |
| PutRecords.Success | Measures the number of PutRecords operations where at least one record succeeded. | Count | Average |
| ReadProvisionedThroughputExceeded | Measures the number of GetRecords calls throttled. | Count | Average |
| WriteProvisionedThroughputExceeded | Measures the number of records rejected due to throttling. | Count | Average |
| PutRecords.TotalRecords | The total number of records sent in a PutRecords operation per Kinesis data stream, measured over the specified time period. | Count | Sum |
| PutRecords.SuccessfulRecords | The number of successful records in a PutRecords operation per Kinesis data stream, measured over the specified time period. | Count | Sum |
| PutRecords.FailedRecords | The number of records rejected due to internal failures in a PutRecords operation. | Count | Sum |
| PutRecords.ThrottledRecords | The number of records rejected due to throttling in a PutRecords operation per Kinesis data stream, measured over the specified time period. | Count | Sum |
| PutRecords Successful | The percentage of successful PutRecords Operation | Percentage | Average |
| PutRecords Failed | The percentage of failed PutRecords Operation | Percentage | Average |
| PutRecordsThrottled | The percentage of throttled PutRecords Operation | Percentage | Average |
Configuration details
The following config data is collected on a per data stream basis
| Attribute | Description |
|---|---|
| Stream name | Displays the name of the data stream |
| Stream ARN | Displays the Amazon Resource Name of the data stream |
| Created time | Displays the time stamp for data stream creation |
| Stream status | Displays the current status of the data stream |
| Encrypted Type | The type of encryption in user – server side or client side |
| Number of Shards | Displays the number of shards associated with the stream |
| Shard Level Metrics | Shows whether shard level metrics is enabled or not |
| Data Retention Period | Displays the retention period in hours. |