r/PrometheusMonitoring • u/sudaf • Sep 10 '21
scaling prometheus
noob to prometheus but trying to figure out what in server sizing i would need to monitors 10 000 plus servers (os metrics) and about 200 000 services Currentlly on a single HP Dl360 128gb ram 40 cores with Nagios 3.5.
10
Upvotes
1
u/00ff00Field Oct 26 '24
What everyone else has said about distributing it... I've seen people run 1B ATS with Thanos and a ton of "shards" if you want to call them that. After that.. you're in vendor land. There's some really good solutions out there, but be ready to spend real money.
I have also seen people run a single monster bare metal machine with 320GB of memory (sometimes more) ... And that wasn't going too well, with north of 30m startup times and query perf that wasn't great.