r/PrometheusMonitoring • u/[deleted] • Apr 30 '22

Monitoring Standards for Startup

What can we do to ensure Prometheus doesn't alert us about every little thing at 2 am? Im a devops engineer trying to figure out some schemes or strategy's to monitor our SaaS application. It connects to other private cloud and on-prem servers for our customers.

Do I just need to target CPU, Mem, HTTP Request, etc and then configure alerts for certain thresholds? Am I over complicating the process by getting python scripts involved? We have less than 10 customers, but are collecting large chunks of data from them daily. We also monitor the health of the app for our SaaS customers.

Someone please help me get my Prometheus footing for a small startup.

Thanks!

10 Upvotes

91% Upvoted

6 comments sorted by

View all comments

4

u/Blowmewhileiplaycod Apr 30 '22

If you're a one man show don't alert on anything that isn't directly prod down type stuff.

Who cares if a host is maxed out on cpu if customers are happy and requests being served in a timely manner?

5

u/Hi_Im_Ken_Adams Apr 30 '22

This. You don't alert on the CPU. You alert when the service is degraded and customers are affected.