Skip to content

Production observability with OpenTelemetry ​

This recipe takes a Nwire app from "logs only" to a live trace tree in Tempo, Honeycomb, or Jaeger — without writing any OTel instrumentation in your domain code. The bridge package @nwire/telemetry-otel does the translation for you.

What you'll wire up ​

Nwire's canonical telemetry events (action.dispatched, event.published, reaction.fired, …) get translated into OTLP spans and span events, then shipped out via the standard OTel SDK to your backend of choice — Tempo, Honeycomb, Jaeger, Datadog, or a Vector + GreptimeDB stack.

Install ​

@nwire/telemetry-otel is duck-typed against the OTel Tracer interface, so the only hard runtime dep on Nwire's side is @nwire/forge. You bring the OTel SDK and the exporter for the protocol your collector speaks.

bash
pnpm add @nwire/telemetry-otel \
         @opentelemetry/api \
         @opentelemetry/sdk-trace-node \
         @opentelemetry/exporter-trace-otlp-http \
         @opentelemetry/resources \
         @opentelemetry/semantic-conventions

Pick exporter-trace-otlp-http for HTTP/protobuf collectors (Tempo, Honeycomb, the OTel Collector default). Swap for exporter-trace-otlp-grpc if your collector listens on 4317 gRPC.

Wire it in ​

One file — call it apps/<wire>/otel.ts — boots the SDK and attaches the exporter to app.runtime. Everything else stays untouched.

ts
// apps/learnflow-api/otel.ts
import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node";
import { BatchSpanProcessor } from "@opentelemetry/sdk-trace-base";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
import { Resource } from "@opentelemetry/resources";
import { SemanticResourceAttributes } from "@opentelemetry/semantic-conventions";
import { trace } from "@opentelemetry/api";
import { attachOtelExporter } from "@nwire/telemetry-otel";
import type { App } from "@nwire/forge";

export function attachTracing(app: App, serviceName: string): () => void {
  const provider = new NodeTracerProvider({
    resource: new Resource({
      [SemanticResourceAttributes.SERVICE_NAME]: serviceName,
      [SemanticResourceAttributes.SERVICE_VERSION]: process.env.GIT_SHA ?? "dev",
      [SemanticResourceAttributes.DEPLOYMENT_ENVIRONMENT]: process.env.NODE_ENV ?? "dev",
    }),
  });
  provider.addSpanProcessor(
    new BatchSpanProcessor(
      new OTLPTraceExporter({
        url: process.env.OTLP_URL ?? "http://localhost:4318/v1/traces",
      }),
    ),
  );
  provider.register();
  return attachOtelExporter(app.runtime, { tracer: trace.getTracer(serviceName) });
}

Call it from your wire's entry point:

ts
// apps/learnflow-api/main.ts
import { learnflowApp } from "./learnflow.app";
import { attachTracing } from "./otel";

const app = learnflowApp.create({ /* config */ });
const detachTracing = attachTracing(app, "learnflow-api");

await app.start();

process.on("SIGTERM", async () => {
  detachTracing();
  await app.stop();
});

attachOtelExporter accepts any object that satisfies the duck-typed OtelTracer shape — a single startSpan(name, options) method returning a span with setAttribute(s), addEvent, recordException, setStatus, and end. The real @opentelemetry/api Tracer matches structurally, so you can pass it straight through.

Causation chains ​

Nwire envelopes carry correlationId (the trace) and causationId (the immediate parent message). The exporter uses them to stitch spans into a tree:

  • action.dispatched opens a span and keys it by envelope.messageId.
  • Every downstream record (event.published, actor.transitioned, projection.folded, reaction.failed) carries an envelope whose causationId points back at the originating action's messageId. The exporter does a Map.get(causationId) and attaches the record as a span event on that open action span.
  • reaction.fired and external.call.* open their own spans for sub-operations that have meaningful duration.
  • action.completed / dlq.recorded look the parent up by messageId and close it with OK or ERROR.

If a close record arrives without a matching open span (listener attached mid-flight, or messageId collision after a process restart), the exporter emits an orphan span tagged nwire.orphan = true so you don't silently lose data. The full mapping table lives in the OpenTelemetry guide.

What you'll see ​

Open a single request in Tempo or Jaeger and you'll get a tree like this:

nwire.action submissions.submit-answer            [124 ms]
├── event submissions.answer-submitted             (span event)
├── actor.transitioned submission/abc → submitted  (span event)
├── projection.folded submissions-by-student       (span event)
├── nwire.reaction submissions.answer-submitted   [38 ms]
│   └── nwire.action mastery.recompute-from-event  [22 ms]
│       └── projection.folded mastery-by-student   (span event)
└── nwire.external stripe.create-payment-intent   [311 ms]

Every node carries the full envelope as attributes — nwire.correlation_id, nwire.message_id, nwire.tenant, nwire.user_id, plus any Studio-aware metadata (persona, journeyStep, slo) you declared on defineAction. Click a failing span and you get the error name, message, stack, and attempt count without leaving the trace UI.

Production checklist ​

  • Sampling at the SDK. Wrap your NodeTracerProvider with ParentBasedSampler({ root: new TraceIdRatioBasedSampler(0.1) }) for 10% head sampling. Configure tail sampling at the collector to retain spans containing action.failed, reaction.failed, or dlq.recorded events at 100%.
  • Batch interval. BatchSpanProcessor's defaults (5s schedule delay, 30s export timeout, 2048 max queue) are fine for most workloads. Tune scheduledDelayMillis down to 1s if your end-to-end latency SLO is tight, up to 30s if you're paying per-request to the backend.
  • PII redaction. Add a SpanProcessor that strips sensitive attribute keys before export — emails, tokens, free-text inputs. Nwire never puts the action payload into span attributes by default, but custom event metadata can leak.
  • High-volume kinds. Use the kinds: [...] filter on attachOtelExporter to keep only action.*, external.call.*, and dlq.recorded in production. Studio's local stream stays untouched; only the OTLP export is filtered.
  • Detach before shutdown. Always call the returned detach() before provider.shutdown() so still-open spans are closed with a nwire.unsubscribed event rather than truncated mid-export.
  • Resource attributes. Always set service.name, service.version (your git SHA), and deployment.environment. Without them, your APM groups every deploy together.

Don't import @opentelemetry/* in domain code

The exporter lives at the wire boundary on purpose. Your modules (src/<bc>/...) should never import from @opentelemetry/api or any SDK package directly — that couples your domain to a transport concern and breaks the multi-transport invariant (same module, monolith or split, HTTP or queue). If you need custom span attributes for a specific action, declare them as Studio metadata on defineAction (persona, journeyStep, slo) — the exporter will surface them automatically.

See also ​

  • Observability guide — the full reference for @nwire/telemetry-otel, including the complete telemetry-kind → OTel mapping table and per-backend collector configs (Datadog, Honeycomb, Tempo).
  • Telemetry → GreptimeDB — Vector + GreptimeDB pipeline on top of the OTel bridge, with SQL examples for persona / journey-step dashboards.
  • Trace topology diagram — the canonical action → events → reactions → projections tree this recipe wires into your APM.
  • Debugging with trace — the local correlationId loop (CLI + Studio) before you ever need an OTLP backend.
  • Telemetry kinds reference — every record the canonical stream emits.

MIT licensed.