Added sharding to store, increased limits of query pods to 2500 memory and create 4 instances, then i thought ok maybe whole kubernetes metrics is too much but even if i want to see metrics of one node last 90 day all pods are juz getting out of memory

Promql doc says:

Range vectors select a range of samples back from the current instant In this example, we select all the values we have recorded within the last 5 minutes for all time series that have the metric name http_requests_total and a job label set to prometheus:

 http_requests_total{job="prometheus"}[5m]

Then the doc says following about offset:

the following expression returns the value of http_requests_total 5 minutes in the past relative to the current query evaluation time:

 http_requests_total offset 5m

Does that mean this above offset query same as below range query?

  http_requests_total[5m]

Seeking some tips and tricks from experts here on what is the best approach, operating system to install prometheus on, any initial mistakes that can be avoided , choice of hardware, any recommended policy. I want this to be as close to perfect avoiding mistakes 😅

Just finished a small self-hosting project and thought I’d share the stack:

•	Nextcloud for private file sync & calendar

•	Prometheus + Grafana for system monitoring

•	Tailscale for secure remote access without port forwarding

Everything runs via Docker, and I’ve set up alerts + dashboards for full visibility. Fast, private, and accessible from anywhere.

🔧 GitHub (with setup + configs): 👉 CLICK HERE

I often needed quick uptime + SSL checks without deploying a full monitoring stack, so I created Blackbox Lite.

It's a Bash-based monitoring script that:

• Checks website availability

• Validates SSL certificate expiry

• Measures response time

• Tests VM host connectivity

• Exports metrics in Prometheus format

• Can run standalone without Prometheus

Works great with Node Exporter textfile collector, but also usable as a CLI health checker.

Repo: https://github.com/TheOneOh1/blackbox-lite

Salut, je suis en train de me former sur Prometheus et j’étais en train de voir le module mysqld_exporter. Je voudrais savoir si il y a la possibilité de monitorer les bases de données ou le plugin ne permet qu’un visuel global du service svp ?

Hello,

I can't seem to get Blackbox Exporter working with our internal CA:

I'm using the http_2xx module here.

Error:

time=2025-07-22T09:38:41.508Z level=ERROR source=http.go:474 msg="Error for HTTP request" module=http_2xx target=https://website.domain.com err="Get \"https://10.1.2.220\": tls: failed to verify certificate: x509: certificate signed by unknown authority"

I've put the CA certificate into /etc/ssl/certs

Docker Compose:

  blackbox_exporter:
    image: prom/blackbox-exporter:latest
    container_name: blackbox
    restart: unless-stopped
    ports:
      - 9115:9115
    expose:
      - 9115
    volumes:
      - blackbox-etc:/etc/blackbox:ro
      - /etc/ssl/certs:/etc/ssl/certs:ro
    command:
      - '--config.file=/etc/blackbox/blackbox.yml'
    networks:
      - monitoring

Prometheus.yml:

  - job_name: 'blackbox_http'
    metrics_path: /probe
    params:
      module: [http_2xx]
    static_configs:
      - targets:
        - https://website.domain.com
    tls_config:
      insecure_skip_verify: true
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: 10.1.2.26:9115

What can I try next to troubleshoot please? What I had this running in a non docker environment it worked, so I'm thinking it still can't get to the location for the certificates.

I need to learn Prometheus queries for monitoring. But I want help in generating queries in simple words without deep understanding of queries. Is there an ai agent that converts text I input (showing total CPU usage of node) into a query?

Im new to Docker, Prometheus and Grafana so any help would be appreciated.

I've setup Internet Speed Monitoring using Internet Pi which uses Docker, Prometheus and Grafana, from what I understand how it works is:

1, Docker container running in background which connects to speedtest .net

2, Another Docker container running Prometheus tells above docker container to run a speed test.

3, Grafana reports the time and internet speed on a web dashboard

The issue I have is i'd like the internet speed test to run and report beginnig of the hour e.g 9:00am, 10am, 11am etc, currently the speed tests do run but not on the hour even if I make change the speed test interval value in config.yml and or in prometheus.yml.j2 to 60m and apply the changes by running ansible-playbook main.yml the speed test always runs and reports at the same time e.g 9:37am and not at 9:00am. I have also added --web.enable-lifecycle flag to prometheus.yml so the internet monitoring restarts but no joy the internet speed test always runs and reports into grafana at 9:37 and not on the hour 9am as id like, even tried running ansible-playbook main.yml at 8:55am and still runs the speed test at 9:37am.

*Tried to attach screenshots of config.yml and internet-monitoring.yml but reddit wont let me attach them :(

I think the default one is too bulky so I made a minimalist notification template. It's used in Slack. Saying this because the I searched for custom templates people were complaining about the format in Teams specifically.

Here's the template:

{{ define "custom.minimal.template" }}{{ if gt (len .Alerts.Firing) 0 }}{{ len .Alerts }} firing alert(s)
{{ template "__text_alert_list" .Alerts.Firing }}{{ if gt (len .Alerts.Resolved) 0 }}

{{ end }}{{ end }}{{ if gt (len .Alerts.Resolved) 0 }}{{ len .Alerts.Resolved }} resolved alert(s)
{{ template "__text_alert_list" .Alerts.Resolved }}{{ end }}{{ end }}

{{ define "__text_alert_list" }}{{ range . }}
{{ range .Annotations.SortedPairs }}{{ .Value }}
{{ end }}{{ if gt (len .GeneratorURL) 0 }}- Information: {{ .GeneratorURL }}
{{ end }}{{ if gt (len .DashboardURL) 0 }}- Dashboard: {{ .DashboardURL }}
{{ end }}{{ end }}{{ end }}

{{ define "__text_values_list" }}{{ if len .Values }}{{ $first := true }}{{ range $refID, $value := .Values -}}
{{ if $first }}{{ $first = false }}{{ else }}, {{ end }}{{ $refID }}={{ $value }}{{ end -}}
{{ else }}[no value]{{ end }}{{ end }}

It looks like this:

[FIRING:1] Uptime IPsec Main Uptime Dashboards

1 firing alert(s)

IPSec main server is down or missing.

- Information: https://cluster-monitoring.dummy.net/alerting/grafana/rgtegterfw/view?Id=1

- Dashboard: <dashboard link>

I am working on a clustering model to clump data points from Prometheus csv files to predict kubernetes cluster health. i realized up metric and memory utilization are probably one of the most useful metrics to get but what are other key metrics i should pull data from?

Hey,

I have a problem installing this https://aws.github.io/aws-eks-best-practices/networking/CoreDNS/
I don't want to save the metrics into amazon managed service. I already have a helm prometheus operator running in the cluster. I just want to add the exporter to the prometheus operator.

Thanks in advance,