Skip to main content

Datadog Integration

Datadog provides a full-stack observability platform for monitoring your Ballerina integrations. You can forward metrics, distributed traces, and logs to Datadog using the Datadog Agent or direct API ingestion.

Prerequisites​

RequirementDetails
Datadog AccountActive Datadog account with API key
Datadog AgentVersion 7.x or later (installed on the host or as a sidecar)
BallerinaBuilt with --observability-included

Architecture​

Ballerina Integration
├── Prometheus metrics ──▶ Datadog Agent ──▶ Datadog Cloud
├── OpenTelemetry traces ──▶ Datadog Agent ──▶ Datadog Cloud
└── Structured logs ──▶ Datadog Agent ──▶ Datadog Cloud

Step 1 -- install the datadog agent​

Linux​

DD_API_KEY=<your-api-key> DD_SITE="datadoghq.com" \
bash -c "$(curl -L https://install.datadoghq.com/scripts/install_script_agent7.sh)"

Docker​

docker run -d --name datadog-agent \
-e DD_API_KEY=<your-api-key> \
-e DD_SITE="datadoghq.com" \
-e DD_APM_ENABLED=true \
-e DD_LOGS_ENABLED=true \
-p 8126:8126 \
-p 8125:8125/udp \
-v /var/run/docker.sock:/var/run/docker.sock:ro \
datadog/agent:latest

Kubernetes​

Deploy with the Datadog Operator or Helm chart:

helm install datadog-agent datadog/datadog \
--set datadog.apiKey=<your-api-key> \
--set datadog.apm.portEnabled=true \
--set datadog.logs.enabled=true

Step 2 -- forward metrics to datadog​

Configure the Datadog Agent to scrape Prometheus metrics from your Ballerina integration.

Create /etc/datadog-agent/conf.d/openmetrics.d/conf.yaml:

instances:
- openmetrics_endpoint: "http://localhost:9797/metrics"
namespace: "ballerina"
metrics:
- http_requests_total
- http_request_duration_seconds
- http_response_errors_total
- http_active_requests
- ballerina_sql_query_duration_seconds
tags:
- "service:order-service"
- "environment:production"

Restart the agent:

sudo systemctl restart datadog-agent

Step 3 -- forward traces to datadog​

Configure Ballerina to send traces via OpenTelemetry to the Datadog Agent:

[ballerina.observe]
tracingEnabled = true
tracingProvider = "jaeger"

[ballerinax.jaeger]
agentHostname = "localhost"
agentPort = 6831

Configure the Datadog Agent to accept Jaeger traces:

# /etc/datadog-agent/datadog.yaml
apm_config:
enabled: true
apm_non_local_traffic: true
receiver_port: 8126

Step 4 -- forward logs to datadog​

Configure the Datadog Agent to collect logs from your integration:

# /etc/datadog-agent/conf.d/ballerina.d/conf.yaml
logs:
- type: file
path: /var/log/integrations/*.log
service: order-service
source: ballerina
tags:
- "environment:production"

Enable log collection in the agent config:

# /etc/datadog-agent/datadog.yaml
logs_enabled: true

Step 5 -- create datadog dashboards​

Build dashboards in Datadog to visualize your integration metrics:

Useful queries​

PanelQuery
Request Ratesum:ballerina.http_requests_total.count{service:order-service}.as_rate()
Error Ratesum:ballerina.http_response_errors_total.count{service:order-service}.as_rate()
p95 Latencyp95:ballerina.http_request_duration_seconds{service:order-service}
Active Connectionsavg:ballerina.http_active_requests{service:order-service}

Alerting​

Create Datadog monitors for critical thresholds:

  1. Navigate to Monitors > New Monitor.
  2. Select Metric monitor type.
  3. Define the query: sum:ballerina.http_response_errors_total.count{service:order-service}.as_rate() > 0.05.
  4. Set notification channels.

What's next​