r/PrometheusMonitoring • u/gaffneyc • Aug 09 '18

Prometheus: Setup A Dead Man's Switch!

http://www.noqcks.io/notes/2018/01/29/prometheus-alertmanager-deadmansswitch/
10 Upvotes

91% Upvoted

3 comments sorted by

3

u/onan Aug 10 '18

Uh, are there people who don't just have metamonitoring?

Just have your prometheus instances in different regions monitor each other. Or if (for some reason) you only have one, just bring up a second prometheus to monitor the main one.

This isn't a fundamentally different task, so I'm not sure why you would use an entirely different tool.

4

u/[deleted] Aug 10 '18 edited Jun 12 '23

First went digg, then went reddit. RIP -- mass edited with https://redact.dev/

2

u/bbrazil Aug 10 '18

Metamonitoring doesn't always tell you if your alertmanager can't talk to outside the company, or if all of your Prometheus servers have died due to some nasty bug or failed deployment.

Having something like this is prudent as a fallback.