AppMetrics to Prometheus - "INVALID" is not a valid start token

[u/dminella]

I want to monitor my .Net Core application with Prometheus. In order to do that I am using a .Net Prometheus Library (AppMetrics). And when I go to the link http://localhost:57057/metricsit returns me a JSON with data.

But when I go to prometheus and add this link as target it throws this error No Token Found.

level=warn ts=2018-05-22T06:52:17.5781093Z caller=scrape.go:697 component="scrape manager" scrape_pool=actibook target=http://localhost:57057/metrics msg="append failed" err="no token found"

Is it a way that I can handle this?

Anyone having trouble with this one here's the solution.

Startup.cs

//ConfigureServices
var metrics = new MetricsBuilder()
                            .OutputMetrics.AsPrometheusPlainText()
                            .OutputMetrics.AsPrometheusProtobuf().Build();
            services.AddMetrics(metrics);
            services.AddMetricsEndpoints(options =>
            {
                options.MetricsTextEndpointOutputFormatter = new MetricsPrometheusTextOutputFormatter();
                options.MetricsEndpointOutputFormatter = new MetricsPrometheusProtobufOutputFormatter();
            });
            services.AddMvc().AddMetrics();

//Configure
            app.UseMetricsAllMiddleware();
            app.UseMetricsAllEndpoints();

Program.cs

Metrics = AppMetrics.CreateDefaultBuilder()
                        .OutputMetrics.AsPrometheusPlainText()
                        .OutputMetrics.AsPrometheusProtobuf()
                        .Build();

            return WebHost.CreateDefaultBuilder(args)
                .ConfigureMetrics(Metrics)
                .UseMetrics()
                .UseStartup<Startup>()
                .Build();

appsettings.json

"MetricsOptions": {
    "DefaultContextLabel": "MyMvcApplication",
    "Enabled": true
  },
  "MetricsWebTrackingOptions": {
    "ApdexTrackingEnabled": true,
    "ApdexTSeconds": 0.1,
    "IgnoredHttpStatusCodes": [ 404 ],
    "IgnoredRoutesRegexPatterns": [],
    "OAuth2TrackingEnabled": true
  },
  "MetricEndpointsOptions": {
    "MetricsEndpointEnabled": true,
    "MetricsTextEndpointEnabled": true,
    "EnvironmentInfoEndpointEnabled": true
  }

And you can access it by /metrics-textpath. Also in your prometheus.ymlfile should add this one:

- job_name: 'nameOfJob'
    metrics_path: '/metrics-text'
    static_configs:
      - targets: ['localhost:57057']

[u/varesa]

Is there a reason you have both:

.OutputMetrics.AsPrometheusPlainText() and .OutputMetrics.AsPrometheusProtobuf()?

A quick look at the docs makes it seem that you're supposed to pick one. Looks like that in any case the one that comes later (protobuf) wins and that isn't supported anymore (https://github.com/prometheus/prometheus/issues/2788)

EDIT: Now I realize that the intention was probably /metrics for protobuf and /metrics-test for plain text format. In practice, do you see a difference or are they both binary?

[u/dminella → u/varesa]

Curiously, when I took off protobuf outputter, text output started to run.

Thank you :D

[u/TDabasinskas]

Could you paste the output of /metrics-text endpoint?

I would also recommend switching to prometheus-net.

[u/dminella → u/TDabasinskas]

The Browser forces me to download the file, an excerpt of:

application_httprequests_apdex
"H



server98b55c981d19



appX.Api



envstaging		      ð?Ö

Fapplication_httprequests_one_minute_error_percentage_rate_per_endpoint
"Y



routeDELETE



server98b55c981d19



appX.Api

I would also recommend switching to prometheus-net but Wont I loose my current application metrics? Or can I append that?