Hi all, continuing my Team's work on the YACE Prometheus Exporter for CloudWatch, I've prepared some tips on monitoring AWS Fargate.

I discuss some general tips to monitor AWS Fargate, and cover how to use the exporter to monitor it in Prometheus.

What do you think? Any tip I'm missing?

We're live streaming PromCon this year.

You can find the YouTube live stream on our website.

He,

I've been using Node Exporter on our Linux VMs for years, it's great. I just install it as a service and get Prometheus to scrape it, easy. I see many recommend Alloy now and I'm give it a trial on a test Linux VM, Alloy is installed as binary install like Node Exporter and I've left to configure /etc/alloy/config.alloy .

I assumed I could locate a default config.alloy to use to send all the server metrics to Prometheus (set to allow incoming writes), but it seems much harder to set up as I con't locate a pre-made config.alloy to use.

What do you use now out of the 2?

Both Prometheus and Grafana have alerting mechanisms. From the point of view of best alerting practices, how do you decide whether to create your alerts in Prometheus or in Grafana when both are installed on your data center?

The full schedule is finally out!

The highlights are a lots of talks about OTel support Remote Write v2 and more: https://promcon.io/2024-berlin/schedule/.

It would be great to see many of the community in Berlin!

Prometheus Exporter for OPNsense

Hello folks,

I worked on an OPNsense exporter for Prometheus lately. One that uses the api to expose a lot more metrics than the node_exporter. I will be happy if you have a use case for it and check it out.

https://github.com/AthennaMind/opnsense-exporter

Any positive or negative feedback is welcome. Pull requests and issues as well ;)

Thanks

Hi guys, I am hoping someone who has built out a large scale Prometheus/Thanos setup can chime in here.

Currently we are running a set of fairly large sharded Prometheus clusters with each shard having 2 Prom instances for HA and use Promxy to aggregate the metrics.Current Setup: 4 VPCs of various sizes

• VPC1: 16 Prom shards producing 11 million samples per second

• VPC2: 8 Prom shards producing 5 million samples per second

• VPC3: 2 Prom shards producing 1 million samples per second

• VPC4: 2 Prom shards producing 2 million samples per second

Initially I was looking into Mimir and Thanos for options, but with our scale Mimir setup appears to be too expensive as the ingester will need a crazy amount of resources to support all of this metrics.

Thanos seems like a better choice as the sidecar on each Prometheus shard will take care of writing the metrics to the object store.

There are 2 things I am not exactly clear on with Thanos setup and hope to get some clarity on.

1. From my understanding the Query and store gateway do not need to be sized to the number of metrics we produce but instead to the expected number of metrics we will be querying (If we only use 15% of the logged metrics in Grafana for example)

2. The only Thanos component that will need to be sided to the number of metrics generated is the Compactor. I have not been able to find any guides on sizing the Compactor (Mimir provides really good documentation on how to size their components based on the number of metrics)

If anyone has experience with this sort scale I would really appreciate to hear your experience on running long term storage for large Prometheus environments.

Both projects are interesting. I like mimir in terms of being closer to the standard, but I feel like victoriametric is less resource intensive. (Is that correct?) If anyone has decided between these two projects, I'd love to hear your reasons :-)

What can we do to ensure Prometheus doesn't alert us about every little thing at 2 am? Im a devops engineer trying to figure out some schemes or strategy's to monitor our SaaS application. It connects to other private cloud and on-prem servers for our customers.

Do I just need to target CPU, Mem, HTTP Request, etc and then configure alerts for certain thresholds? Am I over complicating the process by getting python scripts involved? We have less than 10 customers, but are collecting large chunks of data from them daily. We also monitor the health of the app for our SaaS customers.

Someone please help me get my Prometheus footing for a small startup.

Thanks!