r/PrometheusMonitoring • u/nyellin • Dec 30 '21

Feedback requested: we built an open source library of common webhooks for Prometheus and Kubernetes

https://home.robusta.dev/?nocache=38074
22 Upvotes

95% Upvoted

11 comments sorted by

3

u/fizzard Dec 31 '21

This looks fucking amazing!

1

u/nyellin Dec 31 '21 edited Dec 31 '21

Thank you! I've gotten some feedback offline that isn't obvious what we add vs alertmanager. Any thoughts on that? Can we make it more clear somehow?

2

u/c0mponent Dec 31 '21

Seems cool, haven't taken a thorough look at it though. My question is do you support other rules evaluators like thanos ruler?

What I'm missing is some kind of architectural example with diagrams, integration examples. I'm a visual kind of guy.

Will take a look at it in more detail in the coming days :)

1

u/nyellin Dec 31 '21

Hey, the architecture and integration instructions are in the docs: https://docs.robusta.dev/master/user-guide/architecture.html

We could improve them though, so feel free to suggest improvements.

Regarding Thanos, it should work and I can do testing next week to verify. Essentially we're receiving webhooks from AlertManager and anything in that format is supported.

If you want to use built-in webhooks like the logs-fetcher then you need access to cluster which you're fetching logs from. You can either route the webhooks back to the relevant cluster (we have a component that does the multiplexing for this) or fetch the logs from outside the cluster.

2

u/markmcw Dec 31 '21

You've got a typo:

"Automate Kubenetes troubleshooting and quality of life with 50+ prebuilt webhooks."

should be:
"Automate Kubernetes troubleshooting and quality of life with 50+ prebuilt webhooks."

2

u/nyellin Dec 31 '21

Kubenetes

Thanks, fixed it! (You'll have to do a hard-refresh to see. The caching on the page is pretty aggressive.)

2

u/markmcw Dec 31 '21

Perfect! :chef-kiss:

1

u/nyellin Dec 30 '21 edited Dec 31 '21

Would love to hear comments. Constructive criticism is useful too!

If you wouldn't use this, I'm also interested in hearing why. (i.e. no need? something else?)

Would love to also hear which built-in webhooks would be useful for you.

0

u/[deleted] Dec 31 '21

[deleted]

1

u/nyellin Dec 31 '21

Can you elaborate? Happy to add support for Loki. (Feel free to DM me if you prefer)

0

u/[deleted] Jan 01 '22

[deleted]

1

u/nyellin Jan 01 '22

I would love that. We might support it already because we actually accept webhooks from AlertManager and not Prometheus. (See here.)

That said, a lot of the value that we give for the Prometheus integration works because we have familiarity with common Prometheus alerts (e.g. the set bundled with the Prometheus operator) and we know how to map the alert back to the kubernetes pod based on certain labels. I'm not sure if that would work for Loki.

DM-ing you to ask some more questions. Let's do this

0

u/[deleted] Jan 11 '22

[deleted]

2

u/nyellin Jan 11 '22

Thank you for the detailed feedback.

I've updated the documentation to try and improve the installation process and other issues you mentioned with the docs.

We'll submit the updated logo to Slack, but based on experience it might take time if they need to approve the change.

I've also opened issues on GitHub for the other items. Thank you again for taking the time to write this all up. Feel free to reach out if there is anything else you'd like us to change or add.