Add an OpenTelemetry Collector for logs and metrics #4087
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: Benjamin Blattberg <ben.blattberg@crunchydata.com> Co-authored-by: Drew Sessler <drew.sessler@crunchydata.com> Co-authored-by: Tony Landreth <tony.landreth@crunchydata.com>
Issue: PGO-2043
cbandy
force-pushed
the
otel-collector
branch
from
c86ec3b to
154646b
Compare
benjaminjb
approved these changes
Feb 6, 2025
Contributor
benjaminjb
left a comment
benjaminjb
left a comment
There was a problem hiding this comment.
LGTM, but might like to hear that it works still
A generator converts YAML with comments to JSON to avoid errors at runtime and comments in the binary. Co-authored-by: Chris Bandy <chris.bandy@crunchydata.com> Issue: PGO-2054
Postgres can log in two structured formats: CSV and JSON since Postgres 15. The two formats are very similar semantically, so this parses them in a shared OTTL transform processor. Co-authored-by: Chris Bandy <chris.bandy@crunchydata.com> Issue: PGO-2033 Issue: PGO-2065
Issue: PGO-2059
Issue: PGO-2056
Collect JSON-formatted logs from pgAdmin when the feature gate is enabled. Issue: PGO-2057
Issue: PGO-2058
This strips comments and formatting from SQL files. It might be useful, and we already have the necessary dependencies.
cbandy
force-pushed
the
otel-collector
branch
from
154646b to
f92e114
Compare
In testing, we found that a running pgbackrest container wouldn't get the permissions adjusted with a mkdir; so we're switching to an install for now.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Checklist:
Type of Changes:
What is the current behavior (link to any open issues here)?
Postgres and pgBackRest are multi-process and log to files. Metrics are collected using postgres_exporter.
What is the new behavior (if this is a feature change)?
Postgres and pgBackRest logs are parsed using the OTel Collector. Patroni and PgBouncer metrics are exported in the Prometheus format. Other exporters and formats will come later.
Other Information:
Issue: PGO-16
Issue: PGO-2033
Issue: PGO-2043
Issue: PGO-2056
Issue: PGO-2057
Issue: PGO-2058
Issue: PGO-2059
Issue: PGO-2065