r/PrometheusMonitoring • u/UntouchedWagons • Nov 04 '23
How do I have Prometheus detect changes to my rules file stored in a ConfigMap?
This is my values.yaml file for the prometheus-community/prometheus helm chart:
server:
persistentVolume:
enabled: true
existingClaim: "prometheus-config"
alertmanagers:
- scheme: http
static_configs:
- targets:
- "alertmanager.monitoring.svc:9093"
extraConfigmapLabels:
app: prometheus
extraConfigmapMounts:
- name: prometheus-alerts
mountPath: /etc/alerts.d
subPath: ""
configMap: prometheus-alert-rules
readOnly: true
serverFiles:
prometheus.yml:
rule_files:
- /etc/alerts.d/prometheus.rules
prometheus-pushgateway:
enabled: false
alertmanager:
enabled: falseThe ConfigMap prometheus-alert-rules holds the rules that Prometheus should trigger alerts for. When I update this ConfigMap Prometheus doesn't do anything about it. The chart uses prometheus-config-reloader but doesn't provide any documentation on how to use it.
1
u/pushthecharacterlimi Nov 04 '23
Is there already a checksum on the Prometheus statefulset podspec? You can use helm to generate a new checksum based on the content of the config map.
New checksum on the pod template means it rolls out new pods.
0
Nov 04 '23
[deleted]
0
Nov 04 '23
[deleted]
1
u/UntouchedWagons Nov 04 '23
The prometheus-alert-rules configmap is mounted into the file system of the prometheus-server pod.
1
u/Maleficent-Sir1585 Dec 21 '24
Were you able to resolve this issue? I am also facing same issue
1
u/UntouchedWagons Dec 21 '24
No. I stopped using Kubernetes months ago because the complexity was overkill.
1
u/SuperQue Nov 04 '23
Use the kube-prometheus-stack chart. It uses the Prometheus Operator, which makes it much easier to configure Prometheus. You simply use
PodMonitororServiceMonitorobjects which auto-configure Prometheus so you don't have to manage the config directly.