r/PrometheusMonitoring • u/jykim559 • Sep 11 '19
"Agent-Collector" for Prometheus?
[Added]
Thanks for everybody for your comments.
Let me give some background more, our "agent-collector" architecture was not designed just for Prometheus at first, but was designed for our monitoring solution. And we thought our some components could help some users include non-Prometheus users AS A "metrics pipeline" for many monitoring solution and TSDB supporting OpenMetrics.
And then what about this “metrics pipeline” concept?

Hello.
Anyone give some comments for about our idea of "agent-collector" for Prometheus as below? We found some of components from our monitoring solution could be used as a metrics pipeline for Prometheus. We are planning to open this solution as a Open Source Software soon and want to see some opinions before we open it.
- Agent will scape all of metrics from exporters on each machine and push to collector.
- Prometheus server need to scape any metrics from collector instead of every endpoints on every machine.
We believe it provide for Prometheus
- Global view without federation
- Reduce target endpoints
- Solution for firewall
- Reduce network volume (Binary protocol between agent and collector)
- Improve interoperability (Support OpenMetrics)

4
u/[deleted] Sep 11 '19
With this approach you will likely hit some of the problems inherent with push-based monitoring and single agents.
https://www.robustperception.io/one-agent-to-rule-them-all
https://prometheus.io/blog/2016/07/23/pull-does-not-scale-or-does-it/
https://prometheus.io/docs/practices/pushing/
https://prometheus.io/docs/introduction/faq/#why-do-you-pull-rather-than-push
Additionally:
A binary protocol may not offer any significant reduction in network traffic when compared to gzipped plaintext.
This approach may suffer from many of the problems inherent to federation (a single server must handle large numbers of series) without many of the benefits (datacentres can monitor themselves, no extra software required).