-
Notifications
You must be signed in to change notification settings - Fork 810
Add multi-target support #1063
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add multi-target support #1063
Conversation
b3486c1 to
8bafbac
Compare
sysadmind
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I need to spend more time with the code to provide all of my feedback, but want to provide what I have ready while I gave it the first pass.
sysadmind
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the guts of this are really solid. It did take me some time to give a proper review so once the changes are pushed up, I will need to allocate some time to re-review. Thanks for submitting this, I think it will be a good improvement.
sysadmind
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All of the new files are also missing the copyright assignment/license header. You should be able to copy paste (make sure there are no dates) from any other go file. I don't love the extra requests for the two collectors but I think that can be cleaned up after #1052 is merged.
sysadmind
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The DCO check is also failing on at least one commit
Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
…'s empty and if so, don't parse it Signed-off-by: pincher95 <yuri.tsuprun@logz.io> Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
Update examples/auth_modules.yml Fix main.go to apply userpass credentials only if the module type is explicitly set to userpass. Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
Keep light-weight validation for the probe params during runtime Add AWS SigV4 authentication module support Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
Add TLS config validation Update config test to include TLS config Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
…when no clusterinfo retriever is attached, avoiding the previous "unknown_cluster" label. Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
…sts to /probe. Updated auth_modules.yml to include AWS SigV4 signing and mTLS support. Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
…theus-community#1065) Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.22.0 to 1.23.0. - [Release notes](https://github.com/prometheus/client_golang/releases) - [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md) - [Commits](prometheus/client_golang@v1.22.0...v1.23.0) --- updated-dependencies: - dependency-name: github.com/prometheus/client_golang dependency-version: 1.23.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
Add tls auth type support in multi-target mode Update README.md, examples/auth_modules.yml, tests Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
955d9ee to
bd1c0a8
Compare
Signed-off-by: Yuri Tsuprun <51751791+pincher95@users.noreply.github.com>
Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
Update roundtripper.go to use region from config or environment resolver if not provided in config file (AWS_REGION) Update probe.go to accept module even if region omitted; environment resolver can provide it Update config.go to use region as optional field Update main.go to use region from config or environment resolver if not provided in config file (AWS_REGION) and update roundtripper.go to use region from config or environment resolver if not provided in config file (AWS_REGION) Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
|
Thanks! Great stuff! I appreciate all of your hard work on this one to get it ready to ship. |
* Add multi-target support Signed-off-by: pincher95 <yuri.tsuprun@logz.io> * Update example-prometheus.yml Signed-off-by: pincher95 <yuri.tsuprun@logz.io> * Make `es.uri` optional by setting default to empty string check if it's empty and if so, don't parse it Signed-off-by: pincher95 <yuri.tsuprun@logz.io> Signed-off-by: pincher95 <yuri.tsuprun@logz.io> * Update README.md Signed-off-by: pincher95 <yuri.tsuprun@logz.io> * Add sanity target scheme validation Signed-off-by: pincher95 <yuri.tsuprun@logz.io> * Change yaml package to go.yaml.in/yaml/v3 Signed-off-by: pincher95 <yuri.tsuprun@logz.io> * Update yaml package to go.yaml.in/yaml/v3 Signed-off-by: pincher95 <yuri.tsuprun@logz.io> * Update CHANGELOG.md Signed-off-by: pincher95 <yuri.tsuprun@logz.io> * Remove whitespaces from README.md Signed-off-by: pincher95 <yuri.tsuprun@logz.io> * Add testing for apikey authentication module Update examples/auth_modules.yml Fix main.go to apply userpass credentials only if the module type is explicitly set to userpass. Signed-off-by: pincher95 <yuri.tsuprun@logz.io> * Add Load-time validation for the auth module config file during startup Keep light-weight validation for the probe params during runtime Add AWS SigV4 authentication module support Signed-off-by: pincher95 <yuri.tsuprun@logz.io> * Expose error in the logger Signed-off-by: pincher95 <yuri.tsuprun@logz.io> * Add TLS config per target support Add TLS config validation Update config test to include TLS config Signed-off-by: pincher95 <yuri.tsuprun@logz.io> * Indices and Shards collectors now fetch cluster_name once from GET / when no clusterinfo retriever is attached, avoiding the previous "unknown_cluster" label. Signed-off-by: pincher95 <yuri.tsuprun@logz.io> * Removed the special-case logic that redirected /metrics?target= requests to /probe. Updated auth_modules.yml to include AWS SigV4 signing and mTLS support. Signed-off-by: pincher95 <yuri.tsuprun@logz.io> * Add license headers to all new files Signed-off-by: pincher95 <yuri.tsuprun@logz.io> * Fixes for relative paths in multi-target mode Signed-off-by: pincher95 <yuri.tsuprun@logz.io> * Bump github.com/prometheus/client_golang from 1.22.0 to 1.23.0 (prometheus-community#1065) Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.22.0 to 1.23.0. - [Release notes](https://github.com/prometheus/client_golang/releases) - [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md) - [Commits](prometheus/client_golang@v1.22.0...v1.23.0) --- updated-dependencies: - dependency-name: github.com/prometheus/client_golang dependency-version: 1.23.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: pincher95 <yuri.tsuprun@logz.io> * Add target schema validation, http/https only Add tls auth type support in multi-target mode Update README.md, examples/auth_modules.yml, tests Signed-off-by: pincher95 <yuri.tsuprun@logz.io> * Cleanup Signed-off-by: pincher95 <yuri.tsuprun@logz.io> * Fix tls auth type validation Signed-off-by: pincher95 <yuri.tsuprun@logz.io> * Remove aws.region validation Signed-off-by: pincher95 <yuri.tsuprun@logz.io> * Add temp file cleanup in config_test.go Signed-off-by: pincher95 <yuri.tsuprun@logz.io> * Add copyright header to config_test.go Signed-off-by: pincher95 <yuri.tsuprun@logz.io> * Add version metric to the per-probe registry Update roundtripper.go to use region from config or environment resolver if not provided in config file (AWS_REGION) Update probe.go to accept module even if region omitted; environment resolver can provide it Update config.go to use region as optional field Update main.go to use region from config or environment resolver if not provided in config file (AWS_REGION) and update roundtripper.go to use region from config or environment resolver if not provided in config file (AWS_REGION) Signed-off-by: pincher95 <yuri.tsuprun@logz.io> --------- Signed-off-by: pincher95 <yuri.tsuprun@logz.io> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Yuri Tsuprun <51751791+pincher95@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: pincher95 <yuri.tsuprun@logz.io>
PR: Add multi-target
/probeendpoint + auth-modules (fixes #1062)This PR implements the feature described in issue #1062 (“multi-target scraping”).
───────────────────────────────
What’s new
───────────────────────────────
/probeendpointAccepts
?target=and (optionally)auth_module=.For each request the exporter spins up a throw-away Prometheus registry, registers the usual collectors with the supplied ES URL, gathers, and streams metrics.
Auth-module support (
--config.file=)Simple YAML schema modelled after
postgres_exporter&mysqld_exporter.Alias on
/metricsRequests that already contain
target=are internally forwarded to the probe handler, so users may keeppath: /metricsin ServiceMonitors if they wish.Per-module API key support
type: apikeyinjects theAuthorization: ApiKey …header for that probe only.Docs & examples
examples/auth_modules.ymlshowsuserpass,apikey.Unit tests
configloader round-trip.validateProbeParams.───────────────────────────────
Compatibility
───────────────────────────────
/metricswithouttarget=still returns the startup cluster.───────────────────────────────
Prometheus sample config
───────────────────────────────
───────────────────────────────
Commit outline
───────────────────────────────
Feedback / suggestions welcome!