Skip to content
This repository was archived by the owner on Mar 20, 2025. It is now read-only.

Telegraf synthetic monitoring #2235

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

Telegraf synthetic monitoring #2235

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
d34ddd4
Telegraf synthetic monitoring
Jan 22, 2023
c22e9cf
Update X509.md
nico-shishkin Jan 25, 2023
6da0453
Update X509.md
nico-shishkin Jan 25, 2023
1ab1c40
Update api-status.md
nico-shishkin Jan 25, 2023
dc5e457
Update ping-statistics.md
nico-shishkin Jan 25, 2023
c874be7
Update X509.md
nico-shishkin Jan 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
136 changes: 136 additions & 0 deletions _source/logzio_collections/_synthetic-monitoring-sources/X509.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,17 @@ shipping-tags:
order: 1380
---

<!-- tabContainer:start -->
<div class="branching-container">

* [Overview](#overview)
* [Using a Lambda functiion](#lambda)
* [Telegraf on a local host](#local)
{:.branching-tabs}

<!-- tab:start -->
<div id="overview">

### Overview

Deploy this integration to collect X509 certificate metrics from URLs and send them to Logz.io. The following metrics are collected by this integration:
Expand All @@ -25,6 +36,13 @@ Deploy this integration to collect X509 certificate metrics from URLs and send t
* x509_start_date (in seconds passed since 1.1.1970)
* x509_end_date (in seconds passed since 1.1.1970)

</div>

<!-- tab:end -->

<!-- tab:start -->
<div id="lambda">

The integration is based on a Lambda function that will be auto-deployed together with the layer [LogzioLambdaExtensionLogs](https://github.com/logzio/logzio-lambda-extensions/tree/main/logzio-lambda-extensions-logs).


Expand Down Expand Up @@ -98,4 +116,122 @@ Give your metrics some time to get from your system to ours, and then open [Kiba


</div>
</div>

<!-- tab:end -->

<!-- tab:start -->
<div id="local">

The integration uses a locally hosted Telegraf agent with a Prometheus remote write plugin to send metrics to Logz.io.

<!-- logzio-inject:install:grafana:dashboards ids=["19AIOkwkFLQCZWmUSINGXT"] -->

<div class="tasklist">

##### Set up Telegraf v1.17 or higher:

{% include metric-shipping/telegraf-setup.md %}

##### Enable the inputs.x509_cert plug-in

Add the **inputs.x509_cert** section to the configuration file:

``` ini
[[processors.rename]]
[[processors.rename.replace]]
measurement = "x509_cert"
dest = "x509"

[[inputs.x509_cert]]
## List certificate sources, support wildcard expands for files
## Prefix your entry with 'file://' if you intend to use relative paths
sources = ["tcp://example.org:443", "https://influxdata.com:443",
"smtp://mail.localhost:25", "udp://127.0.0.1:4433",
"/etc/ssl/certs/ssl-cert-snakeoil.pem",
"/etc/mycerts/*.mydomain.org.pem", "file:///path/to/*.pem"]

## Timeout for SSL connection
timeout = "5s"

## Pass a different name into the TLS request (Server Name Indication).
## This is synonymous with tls_server_name, and only one of the two
## options may be specified at one time.
## example: server_name = "myhost.example.org"
# server_name = "myhost.example.org"

## Only output the leaf certificates and omit the root ones.
# exclude_root_certs = false

## Optional TLS Config
# tls_ca = "/etc/telegraf/ca.pem"
# tls_cert = "/etc/telegraf/cert.pem"
# tls_key = "/etc/telegraf/key.pem"
# tls_server_name = "myhost.example.org"

## Set the proxy URL
# use_proxy = true
# proxy_url = "http://localhost:8888"
```

##### Configure the **iinputs.x509_cert** plugin

Specify the plugin parameters as per the table below:


| Parameter | Description | Required/Optional | Default |
| --- | --- | --- | --- |
| sources | Comma-separated list of your the URLs to collect metrics from. | Required | - |
| timeout | Your response timeout (seconds). | Required | `5 (seconds)` |
| server_name | Server Name Indication used as a different name into the TLS request | Optional | - |
| exclude_root_certs | Selector to output the leaf certificates and omit the root ones | Optional | false |
| tls_ca | Optional TLS Config setting | Optional | - |
| tls_cert | Optional TLS Config setting | Optional | - |
| tls_key | Optional TLS Config setting | Optional | - |
| tls_server_name | Use the given name as the SNI server name on each URL | Optional | - |
| use_proxy | Selector to enable proxy URL. | Optional | true |
| proxy _url| HTTP proxy URL. | Optional | - |

##### Add the outputs.http plug-in

After you create a config file for Telegraf, configure the output plug-in to enable your data to be sent to Logz.io in Prometheus-format and add the following code to the configuration file:


``` yaml
[[outputs.http]]
url = "https://<<LISTENER-HOST>>:8053"
data_format = "prometheusremotewrite"
[outputs.http.headers]
Content-Type = "application/x-protobuf"
Content-Encoding = "snappy"
X-Prometheus-Remote-Write-Version = "0.1.0"
Authorization = "Bearer <<PROMETHEUS-METRICS-SHIPPING-TOKEN>>"
```

{% include general-shipping/replace-placeholders-prometheus.html %}

<!-- info-box-start:info -->
The full list of data scraping and configuring options can be found [here](https://docs.influxdata.com/telegraf/v1.18/plugins/).
{:.info-box.note}
<!-- info-box-end -->

##### Start Telegraf

{% include metric-shipping/telegraf-run.md %}


##### Check Logz.io for your metrics

{% include metric-shipping/custom-dashboard.html %} Install the pre-built dashboard to enhance the observability of your metrics.

<!-- logzio-inject:install:grafana:dashboards ids=["32X5zm8qW7ByLlp1YPFkrJ"] -->

{% include metric-shipping/generic-dashboard.html %}

</div>

</div>
<!-- tab:end -->

</div>
<!-- tabContainer:end -->
207 changes: 207 additions & 0 deletions _source/logzio_collections/_synthetic-monitoring-sources/api-status.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,31 @@ shipping-tags:
order: 1380
---

<!-- tabContainer:start -->
<div class="branching-container">

* [Overview](#overview)
* [Using a Lambda functiion](#lambda)
* [Telegraf on a local host](#local)
{:.branching-tabs}

<!-- tab:start -->
<div id="overview">

### Overview

Deploy this integration to collect API status metrics of user API and send them to Logz.io.

</div>

<!-- tab:end -->

<!-- tab:start -->
<div id="lambda">




The integration is based on a Lambda function that will be auto-deployed together with the layer [LogzioLambdaExtensionLogs](https://github.com/logzio/logzio-lambda-extensions/tree/main/logzio-lambda-extensions-logs).


Expand Down Expand Up @@ -102,5 +123,191 @@ Give your metrics some time to get from your system to ours, and then open [Kiba



</div>
</div>

<!-- tab:end -->

<!-- tab:start -->
<div id="local">

The integration uses a locally hosted Telegraf agent with a Prometheus remote write plugin to send metrics to Logz.op.
Copy link
Contributor

@Doron-Bargo Doron-Bargo Jan 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logz.io



<!-- logzio-inject:install:grafana:dashboards ids=["1rNO8llFw8Cm9N8U3M3vCQ"] -->

<div class="tasklist">

##### Set up Telegraf v1.17 or higher:

{% include metric-shipping/telegraf-setup.md %}

##### Enable the inputs.http_response plug-in

Add the **inputs.http_response** section to the configuration file:

``` ini
[[inputs.http_response]]
## List of urls to query.
urls = ["http://localhost"]

## Set http_proxy.
## Telegraf uses the system wide proxy settings if it's is not set.
# http_proxy = "http://localhost:8888"

## Set response_timeout (default 5 seconds)
response_timeout = "5s"

## HTTP Request Method
method = "GET"

## HTTP response status codes that are considered a success
response_status_success = [200,201,202,203,204]

## Headers to send with HTTP requests
headers = {"Content-Type": "application/json",
"Authorization": "Bearer YOUR_LOGZIO_API_TOKEN"}

## JSON data path, e.g. foo.bar.baz
json_query = "metrics"


## Whether to follow redirects from the server (defaults to false)
# follow_redirects = false

## Optional file with Bearer token
## file content is added as an Authorization header
# bearer_token = "/path/to/file"

## Optional HTTP Basic Auth Credentials
# username = "username"
# password = "pa$$word"

## Optional HTTP Request Body
# body = '''
# {'fake':'data'}
# '''

## Optional name of the field that will contain the body of the response.
## By default it is set to an empty String indicating that the body's
## content won't be added
# response_body_field = ''

## Maximum allowed HTTP response body size in bytes.
## 0 means to use the default of 32MiB.
## If the response body size exceeds this limit a "body_read_error" will
## be raised.
# response_body_max_size = "32MiB"

## Optional substring or regex match in body of the response (case sensitive)
# response_string_match = "\"service_status\": \"up\""
# response_string_match = "ok"
# response_string_match = "\".*_status\".?:.?\"up\""

## Expected response status code.
## The status code of the response is compared to this value. If they match,
## the field "response_status_code_match" will be 1, otherwise it will be 0.
## If the expected status code is 0, the check is disabled and the field
## won't be added.
# response_status_code = 0

## Optional TLS Config
# tls_ca = "/etc/telegraf/ca.pem"
# tls_cert = "/etc/telegraf/cert.pem"
# tls_key = "/etc/telegraf/key.pem"
## Use TLS but skip chain & host verification
# insecure_skip_verify = false
## Use the given name as the SNI server name on each URL
# tls_server_name = ""
## TLS renegotiation method, choose from "never", "once", "freely"
# tls_renegotiation_method = "never"

## HTTP Request Headers (all values must be strings)
# [inputs.http_response.headers]
# Host = "github.com"

## Optional setting to map response http headers into tags
## If the http header is not present on the request, no corresponding tag will
## be added. If multiple instances of the http header are present, only the
## first value will be used.
# http_header_tags = {"HTTP_HEADER" = "TAG_NAME"}

## Interface to use when dialing an address
# interface = "eth0"
```

##### Configure the **inputs.http_response** plugin

Specify the plugin parameters as per the table below:


| Parameter | Description | Required/Optional | Default |
| --- | --- | --- | --- |
| urls | Comma-separated list of your API URLs to collect status from (for example: ["http://sample1", "http://sample2"]. | Required | - |
| http_proxy | HTTP proxy URL. | Optional | - |
| response_timeout | Your API response timeout (seconds). | Required | `5 (seconds)` |
| method | Your API HTTP request method. Can be `GET` or `POST` | Required | `GET` |
| response_status_success | HTTP response status codes that are considered a success | Required | - |
| headers | Headers to send with HTTP requests. | Required | - |
| json_query | JSON data path. | Required | - |
| follow_redirects | Defines whether to follow redirects from the server (defaults to false) | Optional | false |
| bearer_token | Optional path to a file where the bearer token is stored. The bearer token is specified | Optional | false |
| Username | Your API username. | Optional | - |
| Password | Your API password. | Optional | - |
| body | Your API HTTP request body. | Optional | - |
| response_body_field | Optional name of the field that will contain the body of the response. | Optional | - |
| response_body_max_size | Maximum allowed HTTP response body size in bytes. | Optional | - |
| response_string_match | Optional substring or regex match in body of the response (case sensitive). | Optional | - |
| response_status_code | Expected response status code. | Optional | - |
| tls_ca | Optional TLS Config setting | Optional | - |
| tls_cert | Optional TLS Config setting | Optional | - |
| tls_key | Optional TLS Config setting | Optional | - |
| insecure_skip_verify | Use TLS but skip chain & host verification | Optional | - |
| tls_server_name | Use the given name as the SNI server name on each URL | Optional | - |
| tls_renegotiation_method | TLS renegotiation method, choose from "never", "once", "freely" | Optional | - |
| inputs.http_response.headers | HTTP Request Headers (all values must be strings) | Optional | - |
| http_header_tags | Optional setting to map response http headers into tags | Optional | - |
| interface | Interface to use when dialing an address | Optional | - |

##### Add the outputs.http plug-in

After you create a config file for Telegraf, configure the output plug-in to enable your data to be sent to Logz.io in Prometheus-format and add the following code to the configuration file:


``` yaml
[[outputs.http]]
url = "https://<<LISTENER-HOST>>:8053"
data_format = "prometheusremotewrite"
[outputs.http.headers]
Content-Type = "application/x-protobuf"
Content-Encoding = "snappy"
X-Prometheus-Remote-Write-Version = "0.1.0"
Authorization = "Bearer <<PROMETHEUS-METRICS-SHIPPING-TOKEN>>"
```

{% include general-shipping/replace-placeholders-prometheus.html %}

<!-- info-box-start:info -->
The full list of data scraping and configuring options can be found [here](https://docs.influxdata.com/telegraf/v1.18/plugins/).
{:.info-box.note}
<!-- info-box-end -->

##### Start Telegraf

{% include metric-shipping/telegraf-run.md %}

##### Check Logz.io for your metrics

{% include metric-shipping/custom-dashboard.html %} Install the pre-built dashboard to enhance the observability of your metrics.

<!-- logzio-inject:install:grafana:dashboards ids=["32X5zm8qW7ByLlp1YPFkrJ"] -->

{% include metric-shipping/generic-dashboard.html %}

</div>

</div>
<!-- tab:end -->

</div>
<!-- tabContainer:end -->
Loading