Skip to content

Commit 0dde8cd

Browse files
albinsureshalbinsuresh
authored
Merge pull request #3829 from albinsuresh/doc/extensible-log-managament-documentation
doc: Log management plugins documentation
2 parents 0478c6a + 5735e8f commit 0dde8cd

File tree

2 files changed

+216
-0
lines changed

2 files changed

+216
-0
lines changed

‎docs/src/extend/log-management.md‎

Lines changed: 207 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,207 @@
1+
---
2+
title: Extensible Log Management
3+
tags: [Extend, Log Management]
4+
sidebar_position: 2
5+
description: Extend log management capabilities with plugins
6+
---
7+
8+
This document describes how to extend the log management capabilities of %%te%% using custom log plugins.
9+
By default, %%te%% supports retrieving file-based logs.
10+
But, it can be extended with plugins to collect logs from other sources like systemd journal,
11+
docker containers or any custom log sources.
12+
13+
## Overview
14+
15+
The %%te%% agent supports extensible log management through a plugin system:
16+
17+
* The built-in `file` plugin handles traditional file-based logs (the default behavior)
18+
* Additional plugins can be installed to support other log sources
19+
* Each plugin can provide multiple log types
20+
* Plugins are discovered and executed automatically by `tedge-agent`
21+
22+
## How Log Plugins Work
23+
24+
A log plugin is an executable that implements a simple command-line interface with the following requirements:
25+
26+
* Implement two sub-commands:
27+
1. **`list`** - Returns all log types supported by the plugin (one per line)
28+
2. **`get <log-type> [--since <timestamp>] [--until <timestamp>]`** - Retrieves logs for the specified type within the given time range
29+
* Must exit with code 0 for successful `list` command (used to validate the plugin).
30+
* Should output logs to stdout for the `get` command.
31+
* Time filters `--since` and `--until` are passed as seconds since epoch.
32+
* Should handle errors gracefully and exit with non-zero codes on failure.
33+
34+
The agent automatically:
35+
* Discovers plugins at startup by running their `list` command to gather their supported log types.
36+
* Publishes the supported log types to the device's `log_upload` command metadata topic (e.g: `te/device/main///cmd/log_upload`)
37+
with a plugin suffix in the format `<log-type>::<plugin-name>` (e.g., `mosquitto::journald`)
38+
* Routes `log_upload` requests to the `get` command of the appropriate plugin based on the type suffix.
39+
* The `dateFrom` and `dateTo` parameters in the command are passed to the plugin as `--since` and `--until` arguments.
40+
* Further filtering by `searchText` and tail `lines` are done by the agent itself.
41+
* Detects any new plugin installations dynamically.
42+
* Refresh the supported log types by reloading the plugins when any new software is installed or configuration is updated.
43+
44+
Log plugins are installed at `/usr/share/tedge/log-plugins`, by default.
45+
46+
This plugin root directory can be changed using:
47+
```sh
48+
sudo tedge config set log.plugin_paths /usr/local/share/tedge/log-plugins,/usr/share/tedge/log-plugins
49+
```
50+
51+
Multiple plugin directories can be specified if a layered directory structure is desired,
52+
where plugins from directories earlier in the list gets precedence over the latter ones.
53+
For example, with the above configuration, a `file` plugin in `/usr/local/share/tedge/log-plugins`
54+
would override the one with the same name in `/usr/share/tedge/log-plugins`.
55+
This mechanism can be used to override the factory plugins when needed.
56+
57+
## Permissions
58+
59+
Plugins are executed by the agent with `sudo` privileges.
60+
The agent automatically creates the following sudoers entry,
61+
giving sudo rights to all plugins installed at `/usr/local/lib/tedge/log-plugins`:
62+
63+
```
64+
tedge ALL = (ALL) NOPASSWD:SETENV: /usr/local/lib/tedge/log-plugins/[a-zA-Z0-9]*
65+
```
66+
67+
If the `log.plugin_paths` config is updated with additional directories as shown in the previous section,
68+
then sudoers entries must be created for those directories as well.
69+
70+
Additionally, ensure your plugin has appropriate permissions to access the log sources it needs.
71+
72+
## Creating a Custom Log Plugin
73+
74+
### Example: docker plugin
75+
76+
Here's a `docker` plugin example that can retrieve logs from containers using the `docker logs` command:
77+
78+
```sh
79+
#!/bin/sh
80+
set -eu
81+
82+
help() {
83+
cat <<EOT
84+
docker log plugin to retrieve the logs from containers using the docker cli
85+
86+
$0 <SUBCOMMAND>
87+
88+
SUBCOMMANDS
89+
list
90+
get <type> [--since <timestamp>] [--until <timestamp>]
91+
EOT
92+
}
93+
94+
list_log_types() {
95+
docker ps -a --format "{{.Names}}"
96+
}
97+
98+
get_log_by_type() {
99+
log_type="$1"
100+
shift
101+
102+
# Parse option defaults
103+
since="24h"
104+
until="0s"
105+
106+
while [ $# -gt 0 ]; do
107+
case "$1" in
108+
--since)
109+
since="$2"
110+
shift
111+
;;
112+
--until)
113+
until="$2"
114+
shift
115+
;;
116+
esac
117+
shift
118+
done
119+
120+
# Retrieve logs using docker logs
121+
docker logs "$log_type" \
122+
--since "$since" \
123+
--until "$until"
124+
}
125+
126+
if [ $# -lt 1 ]; then
127+
echo "Missing required subcommand" >&2
128+
help
129+
exit 1
130+
fi
131+
132+
SUBCOMMAND="$1"
133+
shift
134+
135+
case "$SUBCOMMAND" in
136+
list)
137+
list_log_types
138+
;;
139+
get)
140+
get_log_by_type "$@"
141+
;;
142+
*)
143+
echo "Unsupported command" >&2
144+
exit 1
145+
;;
146+
esac
147+
```
148+
149+
* The `list` command of this plugin will output all container names:
150+
```
151+
nginx
152+
mosquitto
153+
```
154+
155+
* The `get` command retrieves the logs for the target container using the `docker logs` command.
156+
157+
### Installation
158+
159+
Copy the plugin to the plugins directory and make it executable:
160+
161+
```sh
162+
sudo cp /path/to/docker /usr/share/tedge/log-plugins/
163+
sudo chmod +x /usr/share/tedge/log-plugins/docker
164+
```
165+
166+
### Testing the Plugin
167+
168+
List all log types the plugin supports:
169+
170+
```sh
171+
sudo /usr/share/tedge/log-plugins/docker list
172+
```
173+
174+
Retrieve logs for a specific service:
175+
176+
```sh
177+
sudo /usr/share/tedge/log-plugins/docker get ssh
178+
```
179+
180+
Retrieve logs with time filters (timestamps in seconds since epoch):
181+
182+
```sh
183+
sudo /usr/share/tedge/log-plugins/docker get tedge-agent --since 1696250000 --until 1696260000
184+
```
185+
186+
## Dynamic log type discovery
187+
188+
To ensure that any newly installed services or log sources are immediately available for log collection,
189+
`tedge-agent` automatically reloads the plugins and refreshes the supported log types on the following events:
190+
191+
* A new log plugin is installed in the plugin directory (`/usr/share/tedge/log-plugins`)
192+
* The `tedge-log-plugin.toml` file is updated
193+
* A new software is installed with the agent (via the `software_update` command)
194+
* A new configuration is installed or updated with the agent (via the `config_update` command)
195+
196+
If the new software or configuration is installed/updated externally, and not via `tedge-agent`,
197+
a refresh can be triggered by just touching the plugin configuration file:
198+
199+
```
200+
touch /etc/tedge/plugins/tedge-log-plugin.toml
201+
```
202+
203+
## Factory Plugins
204+
205+
* The default `file` plugin is included in the `tedge` installation package itself on all distributions.
206+
* A `journald` plugin that can gather systemd service logs using the `journalctl` command is also included
207+
in the `tedge` packages for systemd based distributions like Debian, Ubuntu, RHEL etc.

‎docs/src/operate/c8y/log-management.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,3 +33,12 @@ Cumulocity UI:
3333
<p align="center">
3434
<img width="40%" src={require('../../images/tedge-log-plugin_log-types.png').default} alt="Log request dropdown" />
3535
</p>
36+
37+
## Plugin support for non-file sources
38+
39+
The log management capabilities of %%te%% can be extended using custom log plugins,
40+
to fetch logs from other sources like systemd journal, Docker containers etc.
41+
A `journald` plugin is even provided out-of-the-box for systemd based distributions (e.g: Debian, RHEL etc).
42+
43+
Instructions on how to write and install a new plugin are documented [here](../../extend/log-management.md).
44+

0 commit comments

Comments
 (0)