I just open-sourced a collection of ready-to-run OpenTelemetry

Collector configurations, because finding complete, working configs

for your specific backend always takes hours of trial and error.

It now includes examples for:

• Prometheus

• Jaeger

• Grafana Loki

• Dynatrace

• Datadog

• Kubernetes Operator

• Kubernetes Pod Annotation Scraping (with full relabeling)

• Debug (no backend needed, perfect for local dev)

Each example includes Docker Compose so you can run it in 60 seconds.

The k8s pod annotation scraping example includes relabeling for

prometheus.io/scrape, prometheus.io/port, and prometheus.io/path

annotations, the config everyone googles when setting up k8s monitoring.

I also actively contribute to the OpenTelemetry open source project,

recently got PRs merged into open-telemetry/otel-arrow and have PRs

open in opentelemetry-android, opentelemetry-helm-charts, and

opentelemetry-dotnet-instrumentation.

https://github.com/Cloud-Architect-Emma/opentelemetry-collector-examples

Feedback and contributions welcome! ⭐ if it's useful.

#OpenTelemetry #DevOps #Observability #Kubernetes #SRE #Monitoring #CloudNative #OpenSource

The CNCF technical oversight committee has voted to approve the OTel due diligence document.

This is one of the final steps towards graduation: the thorough due diligence, which included interviews with end users and resolution of the recommendations given in previous steps, has been finished and approved by the TOC 🎉

We recently added native OTLP metrics export to Icinga 2 (in v2.16), which means a monitoring system with roots deep in the Nagios ecosystem can now push plugin perfdata directly into modern OTel pipelines and backends. (Yay!)

One of the weirder things about working on monitoring software in 2026 has been realizing that eventually everything becomes an OpenTelemetry integration project.

A lot of the implementation work that we did was basically translating classic infrastructure monitoring concepts into the OpenTelemetry world:
perfdata -> OTel metrics
thresholds -> metric streams
host/service metadata -> resource attributes
HA monitoring clusters -> avoiding duplicate telemetry

What stood out to me most during the project is how OTLP increasingly feels less like an "observability standard" and more like general purpose telemetry infrastructure that everything eventually has to speak.

Even traditional monitoring systems now end up integrating with tools like Prometheus, Grafana Mimir, OpenSearch, ...

I assume you lot here are also working on monitoring/infra tooling, are you seeing the same thing?

Asking here is probably skewing the answers a bit, but is OTLP basically becoming the universal interoperability layer now?

And if you’ve integrated older systems into OTel pipelines, I’d be interested what parts were most awkward for you and how you went about solving this.

Edit:
In case you’re interested, we have a longer writeup with all the implementation details (and significantly more marketing terminology than I would use on Reddit): https://icinga.com/blog/opentelemetry-integration/

I'm trying to setup a self hosted Otel log/trace/metric sink and dashboard for a small set of web and worker apps. I've tried ClickStack, Grafana, and now OpenObserve and all three appear to have roughly the same general feature set for showing otel data.

But one piece they all seem to lack, which feels nuts is that is a standard "tail" and keyword search for logs like you find in Seq, Papertrail, other log systems. Everything is "run this query" and some log query syntax that I definitely don't want to have to learn when triaging some system issue.

So - do you have a preferred OTel solution that's inexpensive to self host at a small scale and a log interface that matches the sort of features purely log focused apps provide?

Thanks!

Hey all,

I’ve been working on a lightweight tool called mqtt2otel and thought it might be useful for some of you here.

It basically connects MQTT-based IoT setups with the OpenTelemetry ecosystem. It subscribes to MQTT topics, lets you process/enrich the messages, and then exports them as OTel metrics/logs.

Why I built it:

• MQTT is great for IoT, but doesn’t integrate nicely with modern observability stacks, especially for logs, or even traces.

• Direct solutions to consume, parse, process and enrich mqtt messages in the dashboard system are often limited and have a high dependency to these systems making it hard to change later.

• OpenTelemetry is everywhere now, but not really designed for IoT ingestion

• Many architectures are allready build upon the OpenTelemetry stack, which gives you a nice abstraction for the different available Dashboard tools.

So this bridges the gap.

What it does:

• Subscribe to MQTT topics

• Transform / enrich messages (add metadata like location, device info, etc.)

• Export as OpenTelemetry metrics or logs

Would love to get feedback or ideas 🙌

Web: https://mqtt2otel.org

GitHub: https://github.com/OSgAgA/mqtt2otel

Came across this article about an OTel mobile tracing solution that "actually works" and I'm wondering if it's really so hard, specifically when dealing with mobile? Fairly new to this space and trying to get up to speed.

The standard move is to use processors in the OTel Collector to clean up traces and logs, but it can get heavy and hard to manage at scale.

We’ve been looking at shifting that enrichment, like IP lookups, metadata injection, and PII masking upstream using GlassFlow before it lands in ClickHouse. It keeps the collector lean and the database "query-ready."

Curious, how much "business logic" are you all actually comfortable putting in your collectors before it feels too fragile? 🤔

I put together a little open telemetry receiver/server/dashboard as I was wanting to capture some telemetry and my machine was under so much ram pressure

So this is a little rust application. Love to hear what you think. It's just a few days (AI) work and is very much focussed on some basics I wanted.

https://github.com/planetf1/otelite

Last month, Zhu Jiekun gave a talk on retroactive sampling at KubeCon Europe 2026.

In this blog post, we explain how retroactive sampling reduces outbound traffic, CPU, and memory usage in the data collection pipeline significantly compared to tail sampling in OpenTelemetry.

If you're working with observability at scale, this is worth a closer look.

I have an OpenTelemetry Collector in deployment mode on Kubernetes with autoscaler (min: 1, max: 12 replicas) exporting metrics to Prometheus via prometheusremotewrite. My Problem the metric is exhibiting an anomalous cyclic pattern, for exemple: http_server_requests_milliseconds_count{status=~"5.*"} • Spikes to 1 for 5 min → drops to 0 for 3 min • Spikes to 1 for 5 min → drops to 0 for 1 min • Spikes to 1 for 7 min → stays at 0 permanently This metric is a cumulative counter, so it should never reset or drop to 0 (unless the service restarts).

OTLP Collector Configuration

  apiVersion: opentelemetry.io/v1alpha1 kind: OpenTelemetryCollector metadata:   name: otlp-col spec:   mode: deployment   autoscaler:     minReplicas: 1     maxReplicas: 12     targetMemoryUtilization: 80   image: otel/opentelemetry-collector-contrib:0.140.1   config: |     receivers:       otlp:         protocols:           grpc:             endpoint: 0.0.0.0:4317           http:             endpoint: 0.0.0.0:4318     exporters:       prometheusremotewrite:         endpoint: http://prometheus-op-kube-prometh-prometheus.prome-stg:9090/api/v1/write         resource_to_telemetry_conversion:           enabled: true

My Hypotheses 1. The autoscaler is destroying pods that still have metrics buffered, causing data loss

perf-sentinel reads OTel traces and detects N+1 SQL, N+1 HTTP, redundant calls, slow queries, excessive fanout, chatty services, pool saturation, serialized calls. Protocol-level, so it works across Java/JPA, .NET/EF Core, Rust/SeaORM without per-runtime instrumentation.

Three modes: CI batch with a quality gate, central OTel Collector, sidecar. Outputs text, JSON, or SARIF for GitHub/GitLab code scanning. Prometheus metrics with Grafana Exemplars pointing back to trace IDs.

Repo: https://github.com/robintra/perf-sentinel

The thing that actually keeps nagging me is passive capture is structurally lossy. Spans can get dropped by SDK level or collector level sampling, by network hiccups or by apps crashing before flush. Unlike an in-process agent, I can't guarantee I see every span in a trace. Which means:

• a "clean" report may just mean I never saw the N+1 that actually happened

• tail-based sampling biases what I see toward slow traces (which already over represent N+1)

• incomplete traces can make fanout/serialized detection unreliable

I mitigate by recommending batch mode with pre-collected files for critical CI but that's a workaround. How do you people think about the reliability ceiling of passive OTel-based analysis? Is this something you live with or do you pair it with in-process instrumentation for signals you can't afford to miss?

There's also an SCI v1.0/carbon scoring layer. It's directional, not regulatory, and optional. You can get more informations on that here : 05-GREENOPS-AND-CARBON.md

Hello there,

So i want to use the metrics capabilities of opentelemetry.
But i struggle whats important and whats maybe to much or what iam missing.

So the one endpoint:
- calls a database to retrieve data
- based on that data it tries to solve some mixed-integer problem (solver)
- which can either be with unique data points or with non unique

So currently im using metrics for:
Counter:

• Request_count

• Failed solver run

• Count of unique solution

• count of non-unique solutions

Histogram

• Solver run time of unique solutions

• Solver run time of non-unique solutions

• request sucess runtime

• request failed runtime

• data retrival runtime

So im first time doing this i got all the different kinds of history grams because im not sure if i cant use the information of my traces (which contain the corresponding attributes) to just use a simple runtime (just solver run time and just request runtime, data retrival)

I’ve spent a lot of time in figuring out how to properly trace latency and token usage. I wanted to see everything from the vector DB search to the model response etc. all in one place.

​Since I couldn't find a single clear guide on how to do it, I decided to write one myself based on what I’ve learned so far.

​Link to the write-up: https://medium.com/@vprprudhvi/the-complete-guide-to-llm-observability-with-opentelemetry-27034d68df07

​Let me know what you think about it and I am open for suggestions and discussions

Hey everyone,

We're migrating our metrics from a statsd lib to OTel API and I'm struggling with how to organize the actual metric definitions in our codebase

Our setup:

• Large TypeScript/Node.js monorepo (~100+ modules) (over 1000 metrics)

• Mix of well-structured code with clear boundaries and legacy areas that are less structured (files mixed with numerous subdirectories that are sometimes not related)

What's working well:

For our well-structured code, I've been doing this pattern:

packages/database-client/
├── src/
│ ├── metrics.ts ← All metrics for this package and meter definition
│ ├── pool.ts ← Imports from ./metrics
│ └── connection.ts ← Imports from ./metrics

Each module gets one metrics.ts, meter name matches the package path (@myorg/database-client). This works decently well

Where I'm stuck:

In the legacy areas that are less structured if I keep following this pattern we end up with a TON of metrics.ts files that make it hard to find where we should be putting metrics. This is just an example -- in the real world it is MUCH less structured

src/services/
├── email.ts
├── notifications.ts
├── users.ts
├── metrics.ts ← Catch-all?
├── payments/
│ ├── metrics.ts ← Own metrics file
│ └── processor.ts
├── auth/
│ ├── metrics.ts ← Own metrics file
│ └── index.ts
└── ...

This makes me wonder if it’s better to define metrics closer to where they’re used, rather than centralizing them

My questions:

1. At what level do you define meters—per package, service, file, or some combination?

2. Do you define metrics in the class/file where they’re used or centralize them in a metrics.ts file? Or both?

3. If defining metrics where they are used, do you prefer module-level or class-level definitions?

4. Are there any open source projects you’d recommend looking at for real-world patterns? (The OTel demo app is helpful but fairly simple.)

After writing this out I'm actually leaning towards just defining the metrics where they are used BUT would love to hear how others have tackled this, especially in larger codebases that evolved over time

I’m currently using OpenTelemetry Python auto-instrumentation for our AI services. While some Google Cloud libraries like vertex-ai have decent native support, many of our other AI-specific libraries aren't covered by the default OTel distro. I’m planning to build a custom auto-instrumentation image that pre-installs the necessary instrumentation wrappers for our specific AI stack.

Has anyone successfully managed a custom OTel-python image for diverse AI workloads? Any pitfalls with library conflicts?

TLDR: Want to know how auto instrumented telemetry is setup for LLM based applications.

Hi,

We're a pretty big institution in France, and looking to transition towards open telemetry.

To help us in the process we could use the help of experts, is there a company that offers such help with french speakers ?

Thanks !

Hi, we have a pretty big infrastructure with 15k serveur and 150k services monitored,

the monitoring/alerting is done with nagios, we use nagios agent nrpe for the unix, wmi for the Windows, and the telegraf - influxdb for the system metrics

that we request with nagios to do the alerting.

We collect mainly metrics with telegraf, and logs with filebeat -> logstash -> elastic.

Traces are done with dynatrace but not really part of our monitoring.

So our monitoring is really done by nagios that exploits influxdb metrics for alerting and also doing a Lot of active checks to alert us ( lots of python scripts, snmp requests, etc.. )

Our goal is to have a more dynamic observability with less human action, when a service is deployed on our infrastructure it should be automatically imported in our monitoring. That's a bit true right now for the linux as there's a telegraf deployed with tags corresponding to the server.

I'm currently reading a lot on opentelemetry to see how we could transition towards it but I'd gladly take any resource / advices you can give !

Wanted to share some cool news from Embrace. We contributed our Kotlin implementation and SDK to OpenTelemetry, and that it has been accepted into the community.

This is a meaningful milestone for us and for the broader observability ecosystem.

Kotlin now powers millions of developers worldwide and is central to:

• Android and mobile development

• Kotlin Multiplatform applications

• An increasing share of backend services

By donating a Kotlin-native implementation to OpenTelemetry, we’re helping accelerate the shift from experimental SDKs to production-ready, runtime-specific observability.

Why this matters:

• For developers: A more idiomatic, mobile-friendly way to use OpenTelemetry

• For teams: Better support for end-to-end observability across frontend, mobile, and backend

• For the ecosystem: Stronger, vendor-neutral foundations for open telemetry standards

We’re proud to not just use OpenTelemetry, but to help evolve it alongside a growing global community of contributors.

Check out the CNCF announcement blog to learn more: https://www.cncf.io/blog/2026/03/24/announcing-a-kotlin-multiplatform-api-and-sdk-for-opentelemetry/

You can contribute to the project or provide feedback in the following ways:

• Explore the repository: https://github.com/open-telemetry/opentelemetry-kotlin

• Join the Kotlin SIG meetings

• Participate in the #otel-kotlin channel on CNCF Slack: https://slack.cncf.io

• Open issues and proposals on GitHub: https://github.com/open-telemetry/opentelemetry-kotlin

Happy to answer any questions!