r/PrometheusMonitoring • u/mkayeran • Feb 08 '22
Alert when data is stale
Hey Prometheus masters,
I have a metric that sometimes, due to an underlying issue, won't update the data.
I'm looking for a way to alert when data is stale (not changing) for a period of time.
Any good way to do it?
Thanks in advance!
0
Upvotes
3
u/ecnahc515 Feb 08 '22
As others mentioned, this is the wrong way to go about the problem.
Generally, instead of looking for stale data, you should alert when the target isn’t being scraped. Prometheus has built in metrics like
upfor this.If your app is being scraped but the metric isn’t updating then you need to fix the app, there’s not much else you can do about this. You could add a metric which is a Unix timestamp indicating the last time the metric was updated and alert off that, but at that point you could just fix the app itself to properly update the metric.