Skip to content

Commit 0b2bee6

Browse files
pbsinclair42Paul Sinclair
andauthored
feat(KinesisFirehoseMonitoring): add isDataFormatConversionEnabled prop (#627)
The conversions widget is useless for any Firehose which doesn't have data format conversions enabled. Default is true for backwards compatability. Implementation is based on the similar `isIterator` parameter for Lambda Functions. Note that the AWS docs are not consistent about what they call this feature, [sometimes calling it data format conversion, sometimes calling it record format conversion](https://docs.aws.amazon.com/firehose/latest/dev/enable-record-format-conversion.html). CDK calls it [data format conversion](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_kinesisfirehose.CfnDeliveryStream.DataFormatConversionConfigurationProperty.html), hence my choice to use that terminology for the parameter too. The library doesn't currently support enabling alarms on any of the data format conversion metrics, so we don't need additional checks there. I've not written any new tests validating if this widget is shown based on the param, since the similar `isIterator` parameter also has no tests for this; happy to add some if you wish, but if so I'd appreciate a pointer to a similar existing test to help me understand this repo's testing approach. --- _By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license_ Co-authored-by: Paul Sinclair <pssincl@amazon.com>
1 parent f501905 commit 0b2bee6

File tree

4 files changed

+190
-8
lines changed

4 files changed

+190
-8
lines changed

API.md

Lines changed: 47 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

lib/monitoring/aws-kinesis/KinesisFirehoseMonitoring.ts

Lines changed: 40 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -23,13 +23,23 @@ import {
2323
RecordsThrottledThreshold,
2424
FirehoseStreamLimitThreshold,
2525
TimeAxisMillisFromZero,
26+
FullWidth,
27+
ThirdWidth,
2628
} from "../../common";
2729
import {
2830
MonitoringHeaderWidget,
2931
MonitoringNamingStrategy,
3032
} from "../../dashboard";
3133

3234
export interface KinesisFirehoseMonitoringOptions extends BaseMonitoringProps {
35+
/**
36+
* Indicates that the Kinesis Firehose has record format conversion enabled.
37+
* This impacts what widgets are shown.
38+
*
39+
* @default - true
40+
* @see https://docs.aws.amazon.com/firehose/latest/dev/enable-record-format-conversion.html
41+
*/
42+
readonly isDataFormatConversionEnabled?: boolean;
3343
readonly addRecordsThrottledAlarm?: Record<string, RecordsThrottledThreshold>;
3444
readonly addIncomingBytesExceedThresholdAlarm?: Record<
3545
string,
@@ -60,8 +70,11 @@ export class KinesisFirehoseMonitoring extends Monitoring {
6070
readonly incomingBytesMetric: MetricWithAlarmSupport;
6171
readonly incomingRecordsMetric: MetricWithAlarmSupport;
6272
readonly throttledRecordsMetric: MetricWithAlarmSupport;
73+
74+
readonly isDataFormatConversionEnabled: boolean;
6375
readonly successfulConversionMetric: MetricWithAlarmSupport;
6476
readonly failedConversionMetric: MetricWithAlarmSupport;
77+
6578
readonly putRecordLatency: MetricWithAlarmSupport;
6679
readonly putRecordBatchLatency: MetricWithAlarmSupport;
6780
readonly incomingBytesToLimitRate: MetricWithAlarmSupport;
@@ -94,6 +107,9 @@ export class KinesisFirehoseMonitoring extends Monitoring {
94107
this.incomingBytesMetric = metricFactory.metricIncomingBytes();
95108
this.incomingRecordsMetric = metricFactory.metricIncomingRecordCount();
96109
this.throttledRecordsMetric = metricFactory.metricThrottledRecordCount();
110+
111+
this.isDataFormatConversionEnabled =
112+
props.isDataFormatConversionEnabled ?? true;
97113
this.successfulConversionMetric =
98114
metricFactory.metricSuccessfulConversionCount();
99115
this.failedConversionMetric = metricFactory.metricFailedConversionCount();
@@ -167,21 +183,37 @@ export class KinesisFirehoseMonitoring extends Monitoring {
167183
}
168184

169185
summaryWidgets(): IWidget[] {
170-
return [
186+
const widgetWidth = this.isDataFormatConversionEnabled
187+
? HalfWidth
188+
: FullWidth;
189+
const widgets = [
171190
this.createTitleWidget(),
172-
this.createIncomingRecordWidget(HalfWidth, DefaultSummaryWidgetHeight),
173-
this.createConversionWidget(HalfWidth, DefaultSummaryWidgetHeight),
191+
this.createIncomingRecordWidget(widgetWidth, DefaultSummaryWidgetHeight),
174192
];
193+
if (this.isDataFormatConversionEnabled) {
194+
widgets.push(
195+
this.createConversionWidget(widgetWidth, DefaultSummaryWidgetHeight),
196+
);
197+
}
198+
return widgets;
175199
}
176200

177201
widgets(): IWidget[] {
178-
return [
202+
const widgetWidth = this.isDataFormatConversionEnabled
203+
? QuarterWidth
204+
: ThirdWidth;
205+
let widgets = [
179206
this.createTitleWidget(),
180-
this.createIncomingRecordWidget(QuarterWidth, DefaultGraphWidgetHeight),
181-
this.createLatencyWidget(QuarterWidth, DefaultGraphWidgetHeight),
182-
this.createConversionWidget(QuarterWidth, DefaultGraphWidgetHeight),
183-
this.createLimitWidget(QuarterWidth, DefaultGraphWidgetHeight),
207+
this.createIncomingRecordWidget(widgetWidth, DefaultGraphWidgetHeight),
208+
this.createLatencyWidget(widgetWidth, DefaultGraphWidgetHeight),
184209
];
210+
if (this.isDataFormatConversionEnabled) {
211+
widgets.push(
212+
this.createConversionWidget(widgetWidth, DefaultGraphWidgetHeight),
213+
);
214+
}
215+
widgets.push(this.createLimitWidget(widgetWidth, DefaultGraphWidgetHeight));
216+
return widgets;
185217
}
186218

187219
createTitleWidget() {

test/monitoring/aws-kinesis/KinesisFirehoseMonitoring.test.ts

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,20 @@ test("snapshot test: all alarms", () => {
5959
expect(Template.fromStack(stack)).toMatchSnapshot();
6060
});
6161

62+
test("snapshot test: data format conversion disabled", () => {
63+
const stack = new Stack();
64+
65+
const scope = new TestMonitoringScope(stack, "Scope");
66+
67+
const monitoring = new KinesisFirehoseMonitoring(scope, {
68+
deliveryStreamName: "my-firehose-delivery-stream",
69+
isDataFormatConversionEnabled: false,
70+
});
71+
72+
addMonitoringDashboardsToStack(stack, monitoring);
73+
expect(Template.fromStack(stack)).toMatchSnapshot();
74+
});
75+
6276
test("test: validation error if incoming traffic usage alarm threshold equal to 1", () => {
6377
const stack = new Stack();
6478

test/monitoring/aws-kinesis/__snapshots__/KinesisFirehoseMonitoring.test.ts.snap

Lines changed: 89 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)