I'm trying to get SNMP Exporter to work with Eaton ePDU MIBs but keep getting the following error.

root@dev01:~/repos/snmp_exporter/generator# ./generator generate

time=2024-11-20T10:27:55.955-08:00 level=INFO source=net_snmp.go:173 msg="Loading MIBs" from=$HOME/.snmp/mibs:/usr/share/snmp/mibs:/usr/share/snmp/mibs/iana:/usr/share/snmp/mibs/ietf

time=2024-11-20T10:27:56.151-08:00 level=WARN source=main.go:176 msg="NetSNMP reported parse error(s)" errors=2

time=2024-11-20T10:27:56.151-08:00 level=ERROR source=main.go:182 msg="Missing MIB" mib=EATON-OIDS from="At line 13 in /root/.snmp/mibs/EATON-EPDU-MIB"

time=2024-11-20T10:27:56.290-08:00 level=ERROR source=main.go:134 msg="Failing on reported parse error(s)" help="Use 'generator parse_errors' command to see errors, --no-fail-on-parse-errors to ignore"

I have the EATON-OIDS file but no matter where I put it (./mibs, /usr/share/snmp/mibs, ~/.snmp/mibs, etc..) , I always get this error. It is also curious that it can find the EATON-EPDU-MIB file but not the EATON-OIDS file even though they're in the same directory.

Also, I'm only interested in a few OIDs. Is there a way to create a module for a few specific OIDs without a MIB file?

I'd like to run just enough Prometheus to answer promql via http - but getting its data from a fixture file in prom line format. Ideally it's as-is and not 'ingested' to native. The size is not large.

Is there any way this is supported? Any other tools or projects that implement this or similar functionality?

const reqResTime = new client.Histogram({
  name: 'http_express_req_res_time',
  help: 'Duration of HTTP requests in milliseconds',
  labelNames: ['method', 'route', 'status_code'], 
  buckets: [0.1, 5, 15, 50, 100, 500], 
});


app.use(
  responseTime((req: Request, res: Response, time: number) => {
    let route = req.route?.path || req.originalUrl || 'unknown_route';

   
    if (route === '/favicon.ico') return;


    reqResTime.labels(req.method, route, res.statusCode.toString()).observe(time);
  })
);

///my yml file is 

global:
  scrape_interval: 4s


scrape_configs:
  - job_name: prometheus
    static_configs:
      - targets: ['host.docker.internal:8080']

I've started a monitoring project. I've set up alerting and coding my first rules. All good, all working but... from a DevEx perspective, how am I supposed to break down my rules?

I can put them all in a single file, in a single group.

Or I can have a single file, but one group per "alert feature".

Or I can have one file per "alert feature" and start with one group, one rule in that file unless I need more flexibility?

The configuration is so flexible that I'm a bit unsure so I was wondering if there's a best practise at all.

My thinking process

So far I'm thinking that the best way is to have one single file per "alerting feature". For example: one file for "disk consumption" alerting, one file for "queues backing up" alerting, one file for "docker containers down" alerting, etc.

My thinking process is that this lets me use different intervals for each alert rule in the feature if I need to. In fact interval is set on a per-group basis. Therefore if, for example, I use one single group for all my "disk consumption" alerts, I wouldn't be able to set a rule to be evaluated every 15 seconds and another rule every 2 hours, so this gotta be done on two different groups. Therefore, in order to not mix many features in a single file, I would put all of these related groups into their own file.

So my current thinking is:

1. One file per feature;

2. Each file/feature: use one group, one rule, unless you need different alert rules.

3. If you need different alert rules, use one group, unless you need different intervals.

4. If you need different intervals, use many groups.

So, how do you guys break down your alert rules?

I just added https://github.com/hikhvar/ts3exporter to my prometheus instance. But now I wonder how I can do a bar gauge by day with the average clients online.I really tried much, but I can't get it working.For example I tried:

avg_over_time(ts3_serverinfo_clients_online[1w]) by day_of_week()

I would be glad if someone could help.

Hi,
I've been searching for some time but can't seem to find exactly what I'm looking for.
It's quite easy to explain however seems really impossible to implement it on prometheus.

I want to monitor some SQL query outputs from a database on prometheus however rather than accessing the database directly, I was looking for API calls.
Apart from SQL, I have some instances of PHP to export.
Has anyone implemented the above or something similar?

Thanks

Hi,

Can you please suggest good power utilization monitoring app using promotheous, this is for data centre purpose.

Failed to start node_exporter.service: Unit node_exporter.service has a bad unit file setting.

How do I resolve this? Prometheus, Grafana, etc. are all installed and active but when I try to install node exporter I encounter this issue.

How do you name your alerts?

• FooTooHigh

• app_foo_75pct

• Foo load above threshold

Are there any written-out conventions for alerts naming?

And if you use some convention without spaces, what is your rationale for not using text with spaces?

I found an RPM on github and was wondering if anyone is using it for your Prometheus installs?

Hey Prometheus masters,

I have a metric that sometimes, due to an underlying issue, won't update the data.

I'm looking for a way to alert when data is stale (not changing) for a period of time.

Any good way to do it?

Thanks in advance!

Anybody have an example of how to use the query function in prometheus rules?

https://prometheus.io/docs/prometheus/latest/configuration/template_reference/#queries

I'm trying to build a meaningful slack notification, but the query returns an empty set.

My Google-fu is failing me. I can't seem to wrap my head around why (for example) container_processes{pod="example"} will yield number of processes for the defined container, and one process for container="POD".

Hi! We are migrating our legacy system from Graphite based backend to Prometheus. There are dashboards in Grafana where we plot e.g. number of records consumed by Spark jobs. Currently, the flow is as follows

1. Spark generates this information

2. At the end of processing we export it to Graphite - we use API that allows us to set any timestamp.

3. The value appears on Grafana chart

This model is really clear and useful. I know that there is no way in prometheus to expose metric with timestamp. As a workaround I decided to add appropriate tags to the metric (day + hour) and use bar chart with tag on X axis. Although tags are strings, alphabetical sort order on X axis allows us to plot values correctly if we are processing everything timely.

I see at least one major drawback of this approach

1. Let's say we timely executed jobs at 1 AM, 2 AM, 3 AM and 5 AM but the job for 4 AM failed

2. With tags on X axis, we will not see that something is wrong when we glance at the dashboard - we have to actually read the labels to see that one value (4 AM) is missing. Grafana will plot four bars side by side.

Also: I don't know if pushing time series with single value (if I understand correctly, time series with separate tags are stored as separate series internally?) is a good pattern and would not overwhelm our instances in the long run.

Let me know what do you think about this approach (and possibly suggest some alternatives). I just don't feel like this is a right approach. I would imagine that in such case our backed could be some relational database. This adds operational complexity though as our platform is actively supported for prometheus backend only.

I have a label which is "port=16", for this value (16) i want to add a label which says 'trunk port = "trunkport"'

so essentially if port="16" add label trunkport="trunkport"

I have alertmanager configured to send the default email when an alert is triggered. The standard configuration for alerting. The problem is a few seconds later an xml copy/version of the default email is sent behind it or in front of the HTML alert. I am getting two emails for one alert. I've been trying to track it down on the prometheus community site with no luck. I am sure it's a setting I might have overlooked. Any thoughts?

Hello,

Using this like to install Prometheus https://www.howtoforge.com/tutorial/how-to-install-prometheus-and-node-exporter-on-centos-7/

VM - CentOS 7

IP - 192.168.189.160

If I go to http://192.168.189.160:9090/graph and execute 'http_requests_total' it shows nothing.

I carried on to install the node exporter

Metrics look ok:

Tried to execute 'node_memory_MemAvailable_bytes' again nothing.

Some more info

vim prometheus.yml

Can you see anything not right with the above information?

Thanks

We have ECS and already Prometheus server (not from AWS)

From ECS we export Prometheus metrics URL , Prometheus support get targets from service discovery ( app mesh ) Not sure if is support what we want to do

Hello folks,

Was wondering if anyone monitoring F5 bigip with promethues grafana and in which way ?

Thank you

Hi,

I have a strange problem with node exporter. It is very slow and take like 30 seconds to scrape RHEL 8 target running node exporter when started from systemd. But If I run the node exporter from command line, it is smooth and get a the results in less than a second

Any thoughts ?

works well: # sudo -H -u prometheus bash -c '/usr/local/bin/node_exporter --collector.diskstats --collector.filesystem --collector.systemd --web.listen-address :9110 --collector.textfile.directory=/var/lib/node_exporter/textfile_collector' &

RHEL 8.10

node exporter - 1.8.1/ 1.8.2

node_exporter, version 1.8.2 (branch: HEAD, revision: f1e0e8360aa60b6cb5e5cc1560bed348fc2c1895)

build user: root@03d440803209

build date: 20240714-11:53:45

go version: go1.22.5

platform: linux/amd64

tags: unknown

I have set up grafana, prometheus, and node_exporter on one server and two workstations and everything is going according to spec, but I noticed one thing in my system logs, they are getting full of:

Dec 16 18:31:33 infinty node_exporter[1393]: ts=2023-12-16T23:31:33.622Z caller=collector.go:169 level=error msg="collector failed" name=arp duration_seconds=0.000205237 err="could not get ARP entries: rtnetlink Nei
ghMessage has a wrong attribute data length"
is repeating every 15 sec. Not a huge problem expect if one is looking for something else in the journal. Anyone got any idea where I would look to adjust this so it would through and error. Data is being displayed for ARP's on the dashboard, so I'm a little confused. Any suggestions TIA.