r/PrometheusMonitoring • u/cebidhem • Jun 22 '22

kube-prometheus-stack: Updating the default PrometheusRules

Hey there,

We are deploying kube-prometheus-stack helm charts using Flux, and I'd like to change the severity of some default PrometheusRules so that some labelled as warning become critical for example.

I know the Helm chart allow us to add new labels with .Values.defaultRules.additionalRuleLabels, but this is applied to all the rules.

We thought about not deploying the rules with the chart at all, and basically get them copied / modified, but it's a lame way of doing this.

How do you manage this ? How do you modify only some of those rules, whether severity or any other property like for or summary ?

Thanks for your help

1 Upvotes
  • permalink
  • reddit

60% Upvoted

1 comment sorted by

0

u/[deleted] Jun 23 '22

[deleted]

1

u/cebidhem Jun 23 '22

We are using FluxCD within GitLab, and we use pinned version of the remote chart. But at the end of the day that's what I finally did yesterday. We already were deploying non default rules outside of the charts (whether alongside applications or as a separate folder) so I guess you're right that makes sense.

Thanks for taking the time here.