# Observability

cronpad is instrumented with OpenTelemetry. It exports metrics and traces over OTLP to any collector, can expose a Prometheus `/metrics` endpoint, and writes structured logs in JSON or logfmt in which every line about a run carries the job ID, run ID and trace ID. A Grafana dashboard and Prometheus alert rules ship in the repository.

| Signal | How to get it | Page |
|---|---|---|
| Metrics | OTLP (`telemetry.otlp_endpoint`) or Prometheus (`telemetry.prometheus_addr`) | [Metrics](metrics.md) |
| Logs | stdout, `json` or `logfmt`; optionally OTLP | [Logging](logging.md) |
| Traces | OTLP | below |

## Quick start with Prometheus

```toml
[telemetry]
prometheus_addr = ":9090"
```

The Compose observability profile scrapes `http://cronpad:9090/metrics` directly. Import `deploy/grafana/cronpad-dashboard.json` into Grafana and load `deploy/prometheus/alerts.yml` into Prometheus.

## Quick start with an OpenTelemetry Collector

```toml
[telemetry]
otlp_endpoint = "otel-collector:4317"
otlp_protocol = "grpc"
otlp_logs = true
```

## Traces

Each run attempt is a span named `cronpad.run` with attributes `cronpad.job_id`, `cronpad.run_id`, `cronpad.attempt`, `cronpad.sink` and `cronpad.outcome`. Webhook requests are child spans with standard HTTP attributes, and cronpad sends a `traceparent` header, so if your endpoint is instrumented its trace joins cronpad's. API requests are traced too. `/readyz` reports dependency and dispatcher health separately from the process liveness probe `/healthz`. `telemetry.traces_sample_ratio` (default `0.1`) controls the sampled fraction.

## The four numbers to watch

| Question | Metric |
|---|---|
| Are runs starting on time? | `cronpad.run.start_lag` p99 and `cronpad.runs.sla_breached` |
| Is work piling up? | `cronpad.queue.oldest_due_age` |
| Are workers healthy? | `cronpad.leases.expired` (should stay at 0) |
| Are jobs succeeding? | `cronpad.runs` by `status` |