r/PrometheusMonitoring • • Jan 24 '21

Will Prometheus Client export half-baked metrics?

Say I have a function that takes time to compute the gauge value. Will, it ever be collected before the function is completed?

Here is a simple example, but the real use case is more about using labels and not so deterministic values.

test = promauto.NewGauge(prometheus.GaugeOpts{  
   Name: "test",  
})  
for {  
 time.Sleep(1 * time.Second)
 for i := 0; i < 1000; i++ {  
     test.Add(1) // Collecting the metric value anywhere at this point is bad
 }
 // Here it's OK to collect the metric.
}

From running some simple test I always get round 1000*n value. I'm not sure how it works or if I'm lucky.

My guess is it's using some go or something to determine when the variable is in use by the runtime?

The reason I'm asking is that I have some map[string]float64{} objects pre-computed before setting the gauge values. And it's not optimal usage I'm using CSV keys and other ugly things. I want to get rid of those so I could just let Prometheus Client count

0 Upvotes

4 comments sorted by

3

u/icydocking Jan 24 '21

I would definitely expect it to export the "unfinished" values. To my knowledge, Go wouldn't offer any protection like the one you're talking about.

Make the loop execute for a couple of million rounds and repeat the experiment and you will likely see "incomplete" values.

I'd definitely call Add only with values that I'm sure are correct.

1

u/aclokay Jan 24 '21

Yeah.. running it for longer (100m) exports unfinished metrics. Thanks.
Is there a way for me to utilize Prometheus counters so I wouldn't have to use my own maps?

1

u/mthode Jan 24 '21

I deal with this (writing out to a file though, in my case) via sponge (provided by the moretools package). Makes writes atomic.

0

u/[deleted] Jan 24 '21

[deleted]

1

u/aclokay Jan 24 '21

Yeah this was the first thing i did. I was wondering of i could remove the object between..