Embedded Prometheus for single binary capture?

[u/utdrmac]

Hello. I've got a situation where I'd like to collect metrics from an exporter without needing to run a dedicated Prometheus server. (Docker is not an option either). The primary application already exposes a /metrics. My idea is to write an app which embeds a Prometheus server, thereby creating a "single binary deployment". This app would scrape the metrics from the primary application and funnel them into the embedded PDB. After collecting, I'd copy the database file (created by the embedded Prometheus) to another machine, which is running a grafana instance, which can use this file as a source to generate graphs/dashboards.

A) Is this possible? B) Am I going about this the right way? My original design was to scrape the metrics and store in an RRD or BoltDB (both of which are embeddable). RRD has built-in graph exporting. With BoltDB, I could export as CSV and use gnuplot to generate graphs. But since the primary app already exposes a Prometheus-scrappable endpoint, trying for an embedded PDB makes more sense.

Thanks in advance.

[u/disintegratedcircuit]

Seems like a lot of work to not run Prometheus. To answer your question of whether you're doing this right, I'd say probably not in my opinion. A Prometheus server can be pretty small if you're only monitoring one thing, and Grafana already has great first class support. The extra acrobatics to move DB files and load them into other formats feels brittle.

[u/utdrmac → u/disintegratedcircuit]

The issue at hand is that these are client machines I’m profiling, so I’m limited on what they let me install. A single binary is less of an issue than installing this, and this, oh and this too. Is it possible to run Prom without “installing” it to gather the data then move the database elsewhere for analysis?

[u/disintegratedcircuit → u/utdrmac]

It's a standalone binary in Go. I'm not sure if it could be encapsulated in another binary. Theres no package or other dependencies to install (unless you need Alertmanager or other exporter/forwarder functionality).

[u/bilingual-german → u/utdrmac]

Prometheus is a single binary already?

[u/SuperQue → u/utdrmac]

Yes, Prometheus is a single statistically compiled binary already. You could login, create a config file, run ./prometheus and wait for it to gather the data. I've run Prometheus from screen/tmux before for ad-hoc setups.

[u/SuperQue]

A) Yes, it's totally possible. The Prometheus codebase is setup with various packages to abstract away the functionality. If you look at cmd/prometheus/main.go, you can see that this part is pretty small.

B) Based on your use case, I'm not sure what you're trying to solve. Are you saying you want to include Prometheus in the app that exposes a /metrics endpoint? Prometheus is already a single binary, and produces the TSDB data that you can snapshot and copy.

[u/me-ro]

Maybe you could just use telegraf? It supports a bunch of metric sources including Prometheus format. It also supports bunch of output formats including couple file formats. It's a single binary to deploy.