Skip to main content

Datadog Integration

Datadog provides a full-stack observability platform for monitoring your Ballerina integrations. You can forward metrics, distributed traces, and logs to Datadog using the Datadog Agent or direct API ingestion.

Prerequisites

RequirementDetails
Datadog AccountActive Datadog account with API key
Datadog AgentVersion 7.x or later (installed on the host or as a sidecar)
BallerinaBuilt with --observability-included

Architecture

Ballerina Integration
├── Prometheus metrics ──▶ Datadog Agent ──▶ Datadog Cloud
├── OpenTelemetry traces ──▶ Datadog Agent ──▶ Datadog Cloud
└── Structured logs ──▶ Datadog Agent ──▶ Datadog Cloud

Step 1 -- install the datadog agent

Linux

DD_API_KEY=<your-api-key> DD_SITE="datadoghq.com" \
bash -c "$(curl -L https://install.datadoghq.com/scripts/install_script_agent7.sh)"

Docker

docker run -d --name datadog-agent \
-e DD_API_KEY=<your-api-key> \
-e DD_SITE="datadoghq.com" \
-e DD_APM_ENABLED=true \
-e DD_LOGS_ENABLED=true \
-p 8126:8126 \
-p 8125:8125/udp \
-v /var/run/docker.sock:/var/run/docker.sock:ro \
datadog/agent:latest

Kubernetes

Deploy with the Datadog Operator or Helm chart:

helm install datadog-agent datadog/datadog \
--set datadog.apiKey=<your-api-key> \
--set datadog.apm.portEnabled=true \
--set datadog.logs.enabled=true

Step 2 -- forward metrics to datadog

Configure the Datadog Agent to scrape Prometheus metrics from your Ballerina integration.

Create /etc/datadog-agent/conf.d/openmetrics.d/conf.yaml:

instances:
- openmetrics_endpoint: "http://localhost:9797/metrics"
namespace: "ballerina"
metrics:
- http_requests_total
- http_request_duration_seconds
- http_response_errors_total
- http_active_requests
- ballerina_sql_query_duration_seconds
tags:
- "service:order-service"
- "environment:production"

Restart the agent:

sudo systemctl restart datadog-agent

Step 3 -- forward traces to datadog

Configure Ballerina to send traces via OpenTelemetry to the Datadog Agent:

[ballerina.observe]
tracingEnabled = true
tracingProvider = "jaeger"

[ballerinax.jaeger]
agentHostname = "localhost"
agentPort = 6831

Configure the Datadog Agent to accept Jaeger traces:

# /etc/datadog-agent/datadog.yaml
apm_config:
enabled: true
apm_non_local_traffic: true
receiver_port: 8126

Step 4 -- forward logs to datadog

Configure the Datadog Agent to collect logs from your integration:

# /etc/datadog-agent/conf.d/ballerina.d/conf.yaml
logs:
- type: file
path: /var/log/integrations/*.log
service: order-service
source: ballerina
tags:
- "environment:production"

Enable log collection in the agent config:

# /etc/datadog-agent/datadog.yaml
logs_enabled: true

Step 5 -- create datadog dashboards

Build dashboards in Datadog to visualize your integration metrics:

Useful queries

PanelQuery
Request Ratesum:ballerina.http_requests_total.count{service:order-service}.as_rate()
Error Ratesum:ballerina.http_response_errors_total.count{service:order-service}.as_rate()
p95 Latencyp95:ballerina.http_request_duration_seconds{service:order-service}
Active Connectionsavg:ballerina.http_active_requests{service:order-service}

Alerting

Create Datadog monitors for critical thresholds:

  1. Navigate to Monitors > New Monitor.
  2. Select Metric monitor type.
  3. Define the query: sum:ballerina.http_response_errors_total.count{service:order-service}.as_rate() > 0.05.
  4. Set notification channels.

What's next