Smile CDR Observability with Grafana, OpenTelemetry, Loki & Tempo

This guide walks you through setting up Smile CDR with a complete observability stack using OpenTelemetry, Grafana, Loki, and Tempo.

Overview

This guide covers:

• Local development setup.
• Logs, metrics, and traces configuration.

Requirements

• Docker & Docker Compose
• Smile CDR distribution with OpenTelemetry support (version 2024.02 or newer)

OpenTelemetry Integration

OpenTelemetry is a framework and toolkit designed to create and manage telemetry data such as traces, metrics, and logs. Starting with the 2024.02 release, Smile CDR includes built-in support for OpenTelemetry Java agent, making it possible to generate telemetry data out-of-the-box.

Currently, Smile CDR generates traces and metrics using the OpenTelemetry Java agent with some Smile-specific customizations.

Full docs: OpenTelemetry Integration Guide.

Step 1: Enable OpenTelemetry in Smile CDR

To enable OpenTelemetry instrumentation in Smile CDR, set the environment variable CDR_OTEL when running Smile CDR.

CDR_OTEL=y OTEL_LOGS_EXPORTER=otlp bin/smilecdr start

Copy

Or export the variables separately:

export CDR_OTEL=y
export OTEL_LOGS_EXPORTER=otlp
export OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4317  # Recommended when running in containers or remote setup
bin/smilecdr start

Copy

Java Agent Setup (Optional)

To override the default agent configuration:

export JAVA_TOOL_OPTIONS="-javaagent:/path/to/opentelemetry-javaagent.jar"
export OTEL_JAVAAGENT_CONFIGURATION_FILE=/path/to/javaagent.properties

Copy

Starting from 2025.05.R01, this override is optional unless using AWS/Azure-specific OpenTelemetry agents.

Sample javaagent.properties file:

otel.service.name=smilecdr
otel.logs.exporter=otlp
otel.javaagent.extensions=./otel/bin/cdr-otel-agent-extension.jar
otel.instrumentation.logback-appender.experimental.capture-mdc-attributes=*
otel.instrumentation.http.server.capture-response-headers=X-Request-ID
otel.jmx.config=./otel/etc/jmx_rules.yaml

Copy

To disable log export:

otel.logs.exporter=none

Copy

Notes on Metrics

Metrics are sent from Smile CDR via OpenTelemetry to Prometheus. Common metrics include:

• smilecdr.session.count
• smilecdr.storage.duration
• smilecdr.authentication.success_count

Step 2: Launch Smile CDR with Observability Stack

Clone the repository

git clone https://gitlab.com/smilecdr-public/smilcdr-grafana.git
cd smilecdr-grafana

Copy

Start the observability stack

docker compose up -d

Copy

This starts:

• OpenTelemetry Collector
• Grafana
• Loki
• Tempo
• Prometheus

Access Grafana

• URL: http://localhost:3000
• Login: admin
• Password: password

Explore Prebuilt Dashboards

In Grafana, go to Dashboards → Smile CDR where you will find:

• Metrics from Prometheus
• Logs from Loki
• Traces from Tempo
• Health and ingestion panels

Troubleshooting

If metrics are missing, check:

• CDR_OTEL=y is set
• OTEL_LOGS_EXPORTER=otlp is set
• otel.logs.exporter=otlp is set in javaagent.properties
• http://localhost:9464/metrics is reachable
• Prometheus targets are up

Example query in Grafana Explore:

smilecdr_session_count

Copy

Useful Docker Commands

docker logs otel-collector
docker logs loki
docker logs grafana

Copy

Summary

With this setup, you have full observability of Smile CDR including logs, metrics, and traces using Grafana, Loki, Tempo, and Prometheus.

Externalized Metrics

Smile CDR Observability

Operating

Monitoring Basics Externalized Metrics Smile CDR Observability Performance Tuning Troubleshooting

Performance Tuning