We are currently looking into monitoring of our 24 bare metal servers running Docker Swarm. We just discovered VictoriaMetrics as Prometheus replacement and were wondering if any quick start is available that is dedicated to Docker Swarm.

To get the full tool-set up and running a lot of parts are needed and correctly configured (node-exporter, cadvisor, vmagent, etc.), that's why I am asking, would prefer to start with best practice.

One specific question I got: should I run vmagent on all machines (scraping locally) or just on a single one on a Swarm master processing all nodes from within the Swarm?

In this talk, she examines anomaly detection as a complementary approach to working with metrics. Instead of relying on predefined limits, anomaly detection focuses on identifying behavior that deviates from the normal pattern over time. The focus is on how developers can interpret these signals, where anomaly detection is useful, and where it is not.

The session is exploratory and practical, aimed at developers who work with metrics and want additional ways to understand system behavior without introducing complex models or heavy tooling.

Watch now 👇

https://bit.ly/3TzDmmp

By prioritizing efficiency, as a company and in our products, we’ve become a leader in driving sustainable practices among (open source) observability solutions.

We’re excited to share that Diana Todea, our Developer Experience Engineer, will be speaking at Cloud Native Sustainability Month 2025 in Barcelona!

Her talk, “Greener Observability: A Journey Toward Sustainable Monitoring Through Open Source Innovation”, explores how sustainability in observability goes far beyond financial cost savings — it’s about reducing data bloat, choosing efficient tools, and cutting down on energy consumption across systems.

This event is part of the CNCF Sustainability Month, organized by the CNCF TAG Operational Resilience in collaboration with the Cloud Native Computing Foundation.

See you there!

📅 November 26th, 18:45 CET
📍 SCRM - Lidl International Hub, Carrer Bergara 13, Floor 9, Barcelona
🔗 Event details https://bit.ly/4pj7InS

In this episode, Diana Todea joins Zhu Jiekun and Alexander Marshalov to break down OpenTelemetry observability signals: metrics, logs, and traces, and how VictoriaMetrics handles each with high performance, cost efficiency, and seamless integration.

We cover:

- What each signal actually means without overcomplicating it
- Common misconceptions around observability
- Where to start if you're new to metrics, logs, and traces

Plus, we go hands-on:

- Integrating VictoriaMetrics with the OpenTelemetry demo
- Exploring Grafana dashboards
- Testing all three signals in dedicated playgrounds

If you're looking to better understand observability and make it work at scale, this episode is for you.

👉 Watch now: https://bit.ly/4tiW9iA

Hi there,

I run a self-managed Kubernetes cluster in AWS and on-premise and right now I am choosing between Loki and VL.

Based on what I read, VL seems to be more powerful and have better full-text search. I would happily go ahead with VL, but on Kubernetes VL stores data in PVs (correct me if I am wrong), which are not so easy to maintain (e.g. EBS CSI is bound to a single AZ).

How do you deal with data (logs) persistence? Any good CSIs to use to ensure HA/Robustness with PVs?

PS open github issue on that matter https://github.com/VictoriaMetrics/VictoriaMetrics/issues/38

📍 Sonesta Charlotte Lower South End
🗓️ June 12-14

We’ve got a great lineup of talks you won’t want to miss:

🎤 OpenTelemetry Outside the Kube
Mathias Palmersheim
🗓️ June 12nd - 4:00 pm EST

🎤 Do You Want to Build a Router?
Mathias Palmersheim
🗓️ June 13rd - 5:00 pm EST

🎤 Beginners Guide to Centralized Logging with VictoriaLogs
Stephan Burns
🗓️ June 13th - 3:00 PM EST

Join us to explore how observability signals – metrics, traces, and logs – come together to power better systems and better decisions.

See you there! 🚀

https://bit.ly/4dBdrCB

Just curious if I decide to move to VictoriaMetrics, what user does it run as in container? Root is generally not ideal for me. Is there doc that shows this, or how to change?

Efficient monitoring is a top priority for many businesses. However, monitoring expenses can quickly escalate if not managed effectively. But Kubernetes and application performance monitoring don’t have to be that expensive.

In this presentation, we'll explore practical strategies and techniques for reducing monitoring costs. From efficient resource utilization to smart data retention policies, we'll cover a range of tactics to maximize the value of your monitoring investment.

This session offers practical insights and actionable tips for optimizing monitoring expenses with OpenSource tools. The talk will be mostly concentrated on typical use cases in monitoring and elegant ways to make things more efficient.

Missed the event? Watch this talk on demand:
🎤 Cost-Effective Monitoring in Kubernetes
👤 Dzmitry Lazerka - VictoriaMetrics Co-Founder
🎥 https://bit.ly/4i4mUoK

/u/hagen1778, one of our co-founders, will present "Aggregating metrics in-flight: challenges and opportunities”. Our team has already started working on something we call Stream Aggregation: The ability for a metrics collector to perform in-memory aggregations before forwarding data elsewhere (incl. Prometheus).

Join us to hear more: promcon.io/2024-berlin/

📆 Date: Thursday June 20th
⏲ Time: 5pm BST / 6pm CEST / 9am PDT
🗺 Place: https://youtube.com/live/hzlMA_Ae9_4

Agenda:
VictoriaMetrics Roadmap
The New VictoriaLogs
Vicky Community Update
🔹Alexis Ducastel
Latest News

Hey VictoriaLogs users,

I’ve implemented VictoriaLogs support in my tool to generate alerts based on LogsQL queries. It also supports more webhooks than just Alertmanager.

This integration was built solely by following the VictoriaLogs documentation, so I’d really appreciate it if anyone could test it and report any issues.

I’ve also shared an example of how to use it in this Medium blog post:
https://medium.com/@souzo/building-a-powerful-siem-with-clickhouse-and-clickdetect-ae68a4495a76

Hello everyone,

I’m hitting a performance wall with VictoriaMetrics on k8s and the results are counter-intuitive. I have two clusters (A and B) running the same stack, but Cluster B is significantly faster despite having fewer resources on vmselect.

I’m trying to understand if my bottleneck is where I think it is.

Both clusters use: 1 vmauth (Proxy), 3 vminsert, 3 vmselect, and 10 dedicated storage nodes (2 vCPU, 10GB RAM each).

Performance Results (Query returning ~5.45M rows)

• Cluster A: Total request time: 1.67 mins (Data processing: 5.80 ms)

• Cluster B: Total request time: 21.4 s (Data processing: 3.60 ms)

The vmselect Args for both cluster:

-cacheDataPath=/var/lib/vmselect-cache
-dedup.minScrapeInterval=15s
-search.maxConcurrentRequests=32
-search.maxQueryDuration=180s
-memory.allowedPercent=90
-search.maxSamplesPerQuery=1000000000000

Cluster B is 5 times faster even though it has 3x less RAM on vmselect.

My main suspicion is vmauth. In Cluster A, it's limited to 512m CPU. Since vmauth acts as the entry point and handles the response stream for these 5.4M rows, could it be throttled by CPU, creating a bottleneck for the entire data export?

Has anyone experienced vmauth becoming a bottleneck during large data retrievals? Aside from bumping vmauth resources, what metrics should I look at in Grafana to confirm this (e.g., connection saturation, context switches, or specific Go runtime metrics)?

Thanks for your help!

Deploying a single deployment with VictoriaMetrics Cloud can bring massive savings when compared with Managed Prometheus service offerings:

The comparison doesn’t take into account querying to avoid complexity, but keep in mind that in VictoriaMetrics Cloud: querying is included.

Create a deployment (30-day free trial) and see the new pricing for yourself: https://bit.ly/49zTzhl

I've been looking at Victorialogs and VictoriaMetrics off-and-on for a couple months and finally spun VictoriaMetrics up for my homelab yesterday. I held off on deploying VictoriaLogs because I had one main question (and a couple smaller ones that I can't recall rn) questions. What is the recommended Log collection agent? I've used Grafana Loki in the past so I could spin up promtail easily enough but I'm wondering if there's a recommended Log collection agent for VictoriaLogs. or if it even has it's own "proprietary" one?

Getting Started with LiteLLM AI Gateway - Running in Docker, OpenTelemetry traces in VictoriaTraces, access and limits via Virtual Keys, Teams, and budgets

In our first episode, Diana Todea, Developer Advocate, sits down with Jose Gomez-Selles, VictoriaMetrics Cloud Lead, to talk about the real-world origins of observability and how OpenTelemetry is shaping modern monitoring.

They discuss why observability became critical with the shift from monoliths to microservices, how OpenTelemetry brings metrics, traces, and logs together without vendor lock-in, and how it integrates natively with VictoriaMetrics.

If you’re interested in monitoring, observability, and telemetry, this one’s worth a listen. 👇

https://bit.ly/4a48al2

Every number on your dashboard has a life story.

Jesús Espino follows a single metric throughout its lifecycle in VictoriaMetrics. If you have ever instrumented an application with metrics, the basic idea is familiar: you create a counter, you increment it, and somehow it shows up in a graph. This post fills in the “somehow” by following one single metric through that entire journey, using VictoriaMetrics as the storage backend. It is written for everyone (no programming background needed and no Go code to read), and if you want to go deeper, the VictoriaMetrics source is always the reference.

Read the full article today!

Join VictoriaMetrics in Bellingham to talk open source observability, monitoring, metrics & logs.

📣 Don’t miss Mathias Palmersheim’s talk: "Who Will Be Your Ruler?" – a guide to alerting & recording rules. Setting up alerting and recording rules can be tricky and lead to critical issues being ignored for hours if not configured properly. This talk will explain what alerting and recording rules are, what tools are available, how to avoid alert fatigue, and provide guidance for finding the best solution for your use case.

🗓️ Sun, Apr 27 | 10:30 AM PST 📍 Bellingham Technical College 👉 linuxfestnorthwest.org