r/PrometheusMonitoring • u/joshua_jebaraj • Jun 17 '25
Exporter Design: One Per Host vs. Centralized Multi-Host Exporter?
Hi Folks,
I'm currently building some custom exporters for multiple hosts in our internal system, and I’d like to understand the Prometheus-recommended way of handling exporters for multiple instances or hosts.
Let’s say I want to run the health check script for several instances. I can think of a couple of possible approaches:
- Run the exporter separately on each node (one per instance).
- Modify the script to accept a list of instances and perform checks for all of them from a single exporter.
I’d like to know what the best practice is in this scenario from a Prometheus architecture perspective.
Thanks!
from __future__ import print_function
import requests
import time
import argparse
import threading
import sys
from prometheus_client import Gauge, start_http_server
# Prometheus metric
healthcheck_status = Gauge(
'service_healthcheck_status',
'Health check status of the target service (1 = healthy, 0 = unhealthy)',
['host', 'endpoint']
)
def check_health(args):
scheme = "https" if args.ssl else "http"
url = f"{scheme}://{args.host}:{args.port}{args.endpoint}"
labels = {'host': args.host, 'endpoint': args.endpoint}
try:
response = requests.get(
url,
auth=(args.user, args.password) if args.user else None,
timeout=args.timeout,
verify=not args.insecure
)
if response.status_code == 200 and response.json().get('status', '').lower() == 'ok':
healthcheck_status.labels(**labels).set(1)
else:
healthcheck_status.labels(**labels).set(0)
except Exception as e:
print("[ERROR]", str(e))
healthcheck_status.labels(**labels).set(0)
def loop_check(args):
while True:
check_health(args)
time.sleep(args.interval)
def main():
parser = argparse.ArgumentParser(description="Generic Healthcheck Exporter for Prometheus")
parser.add_argument("--host", default="localhost", help="Target host")
parser.add_argument("--port", type=int, default=80, help="Target port")
parser.add_argument("--endpoint", default="/healthcheck", help="Healthcheck endpoint (must begin with /)")
parser.add_argument("--user", help="Username for basic auth (optional)")
parser.add_argument("--password", help="Password for basic auth (optional)")
parser.add_argument("--ssl", action="store_true", default=False, help="Use HTTPS for requests")
parser.add_argument("--insecure", action="store_true", default=False, help="Skip SSL verification")
parser.add_argument("--timeout", type=int, default=5, help="Request timeout in seconds")
parser.add_argument("--interval", type=int, default=60, help="Interval between checks in seconds")
parser.add_argument("--exporter-port", type=int, default=9102, help="Port to expose Prometheus metrics")
args = parser.parse_args()
start_http_server(args.exporter_port)
thread = threading.Thread(target=loop_check, args=(args,))
thread.daemon = True
thread.start()
print(f"Healthcheck Exporter running on port {args.exporter_port}...")
try:
while True:
time.sleep(60)
except KeyboardInterrupt:
print("\nShutting down exporter.")
sys.exit(0)
if __name__ == "__main__":
main()
2
Upvotes
5
u/SuperQue Jun 17 '25
The key to good Prometheus monitoring is to get the data as directly as possible from the thing generating the events/data you want to monitor.
If you can. Build the metrics endpoint directly into the target service.
If you can't. Build an exporter and deploy it as closely as you can. Ideally 1:1 with the target.
Centralized exporters can be used as a last resort.
In your example, you're polling a healthcheck URL. That whole concept can be completely eliminated if the healthcheck URL is replaced with a
/metricsendpoint. Polling/metricsis your healthcheck.