You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: src/README.md
+31-9Lines changed: 31 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -24,14 +24,20 @@ components:
24
24
```
25
25
26
26
## Conventions
27
-
- Treat datadog like a separate cloud provider with integrations ([datadog-integration](https://docs.cloudposse.com/components/library/aws/datadog-integration)) into your accounts.
27
+
28
+
- Treat datadog like a separate cloud provider with integrations
29
+
([datadog-integration](https://docs.cloudposse.com/components/library/aws/datadog-integration)) into your accounts.
28
30
29
31
- Use the `catalog` convention to define a step of alerts. You can use ours or define your own.
Since there are so many parameters that may be adjusted for a given monitor, we define all monitors through YAML. By convention, we define the **default monitors** that should apply to all environments, and then adjust the thresholds per environment. This is accomplished using the `datadog-monitor` components variable `datadog_monitors_config_paths` which defines the path to the YAML configuration files. By passing a path for `dev` and `prod`, we can define configurations that are different per environment.
36
+
Since there are so many parameters that may be adjusted for a given monitor, we define all monitors through YAML. By
37
+
convention, we define the **default monitors** that should apply to all environments, and then adjust the thresholds per
38
+
environment. This is accomplished using the `datadog-monitor` components variable `datadog_monitors_config_paths` which
39
+
defines the path to the YAML configuration files. By passing a path for `dev` and `prod`, we can define configurations
40
+
that are different per environment.
35
41
36
42
For example, you might have the following settings defined for `prod` and `dev` stacks that override the defaults.
37
43
@@ -47,6 +53,7 @@ components:
47
53
- catalog/monitors/*.yaml
48
54
- catalog/monitors/dev/*.yaml # note this line
49
55
```
56
+
50
57
For `prod` stack:
51
58
52
59
```
@@ -60,7 +67,8 @@ components:
60
67
- catalog/monitors/prod/*.yaml # note this line
61
68
```
62
69
63
-
Behind the scenes (with `atmos`) we fetch all files from these glob patterns, template them, and merge them by key. If we peek into the `*.yaml` and `dev/*.yaml` files above you could see an example like this:
70
+
Behind the scenes (with `atmos`) we fetch all files from these glob patterns, template them, and merge them by key. If
71
+
we peek into the `*.yaml` and `dev/*.yaml` files above you could see an example like this:
The important thing to note here is that the default yaml is applied to every stage that it's deployed to. For dev specifically however, we want to override the thresholds and priority for this monitor. This merging is done by key of the monitor, in this case `elb-lb-httpcode-5xx-notify`.
132
+
133
+
The important thing to note here is that the default yaml is applied to every stage that it's deployed to. For dev
134
+
specifically however, we want to override the thresholds and priority for this monitor. This merging is done by key of
135
+
the monitor, in this case `elb-lb-httpcode-5xx-notify`.
124
136
125
137
### Templating
126
-
The second thing to note is `${ dd_env }`. This is **terraform** templating in action. While double braces (`{{ env }}`) refers to datadog templating, `${ dd_env }` is a template variable we pass into our monitors. in this example we use it to specify a grouping int he message. This value is passed in and can be overridden via stacks.
138
+
139
+
The second thing to note is `${ dd_env }`. This is **terraform** templating in action. While double braces (`{{ env }}`)
140
+
refers to datadog templating, `${ dd_env }` is a template variable we pass into our monitors. in this example we use it
141
+
to specify a grouping int he message. This value is passed in and can be overridden via stacks.
127
142
128
143
We pass a value via:
129
144
@@ -140,6 +155,7 @@ components:
140
155
datadog_monitors_config_parameters:
141
156
dd_env: "dev"
142
157
```
158
+
143
159
This allows us to further use inheritance from stack configuration to keep our monitors dry, but configurable.
144
160
145
161
Another available option is to use our catalog as base monitors and then override them with your specific fine tuning.
@@ -156,10 +172,14 @@ components:
156
172
157
173
## Other Gotchas
158
174
159
-
Our integration action that checks for `'source_type_name' equals 'Monitor Alert'` will also be true for synthetics. Whereas if we check for `'event_type' equals 'query_alert_monitor'`, that's only true for monitors, because synthetics will only be picked up by an integration action when `event_type` is `synthetics_alert`.
175
+
Our integration action that checks for `'source_type_name' equals 'Monitor Alert'` will also be true for synthetics.
176
+
Whereas if we check for `'event_type' equals 'query_alert_monitor'`, that's only true for monitors, because synthetics
177
+
will only be picked up by an integration action when `event_type` is `synthetics_alert`.
160
178
161
-
This is important if we need to distinguish between monitors and synthetics in OpsGenie, which is the case when we want to ensure clean messaging on OpsGenie incidents in Statuspage.
179
+
This is important if we need to distinguish between monitors and synthetics in OpsGenie, which is the case when we want
180
+
to ensure clean messaging on OpsGenie incidents in Statuspage.
162
181
182
+
<!-- prettier-ignore-start -->
163
183
<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
164
184
## Requirements
165
185
@@ -230,7 +250,7 @@ No resources.
230
250
|------|-------------|
231
251
| <a name="output_datadog_monitor_names"></a> [datadog\_monitor\_names](#output\_datadog\_monitor\_names) | Names of the created Datadog monitors |
232
252
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
233
-
253
+
<!-- prettier-ignore-end -->
234
254
235
255
## Related How-to Guides
236
256
@@ -240,10 +260,12 @@ No resources.
240
260
- [How to Implement SRE with Datadog](https://docs.cloudposse.com/reference-architecture/how-to-guides/tutorials/how-to-implement-sre-with-datadog)
0 commit comments