Production observability with OpenTelemetry

This recipe takes a Nwire app from "logs only" to a live trace tree in Tempo, Honeycomb, or Jaeger — without writing any OTel instrumentation in your domain code. The bridge package @nwire/telemetry-otel does the translation for you.

What you'll wire up

Nwire's canonical telemetry events (action.dispatched, event.published, reaction.fired, …) get translated into OTLP spans and span events, then shipped out via the standard OTel SDK to your backend of choice — Tempo, Honeycomb, Jaeger, Datadog, or a Vector + GreptimeDB stack.

Install

@nwire/telemetry-otel is duck-typed against the OTel Tracer interface, so the only hard runtime dep on Nwire's side is @nwire/forge. You bring the OTel SDK and the exporter for the protocol your collector speaks.

bash

pnpm add @nwire/telemetry-otel \
         @opentelemetry/api \
         @opentelemetry/sdk-trace-node \
         @opentelemetry/exporter-trace-otlp-http \
         @opentelemetry/resources \
         @opentelemetry/semantic-conventions

Pick exporter-trace-otlp-http for HTTP/protobuf collectors (Tempo, Honeycomb, the OTel Collector default). Swap for exporter-trace-otlp-grpc if your collector listens on 4317 gRPC.

Wire it in

One file — call it apps/<wire>/otel.ts — boots the SDK and attaches the exporter to app.runtime. Everything else stays untouched.

// apps/learnflow-api/otel.ts
import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node";
import { BatchSpanProcessor } from "@opentelemetry/sdk-trace-base";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
import { Resource } from "@opentelemetry/resources";
import { SemanticResourceAttributes } from "@opentelemetry/semantic-conventions";
import { trace } from "@opentelemetry/api";
import { attachOtelExporter } from "@nwire/telemetry-otel";
import type { App } from "@nwire/forge";

export function attachTracing(app: App, serviceName: string): () => void {
  const provider = new NodeTracerProvider({
    resource: new Resource({
      [SemanticResourceAttributes.SERVICE_NAME]: serviceName,
      [SemanticResourceAttributes.SERVICE_VERSION]: process.env.GIT_SHA ?? "dev",
      [SemanticResourceAttributes.DEPLOYMENT_ENVIRONMENT]: process.env.NODE_ENV ?? "dev",
    }),
  });
  provider.addSpanProcessor(
    new BatchSpanProcessor(
      new OTLPTraceExporter({
        url: process.env.OTLP_URL ?? "http://localhost:4318/v1/traces",
      }),
    ),
  );
  provider.register();
  return attachOtelExporter(app.runtime, { tracer: trace.getTracer(serviceName) });
}

Call it from your wire's entry point:

// apps/learnflow-api/main.ts
import { learnflowApp } from "./learnflow.app";
import { attachTracing } from "./otel";

const app = learnflowApp.create({ /* config */ });
const detachTracing = attachTracing(app, "learnflow-api");

await app.start();

process.on("SIGTERM", async () => {
  detachTracing();
  await app.stop();
});

attachOtelExporter accepts any object that satisfies the duck-typed OtelTracer shape — a single startSpan(name, options) method returning a span with setAttribute(s), addEvent, recordException, setStatus, and end. The real @opentelemetry/api Tracer matches structurally, so you can pass it straight through.

Causation chains

Nwire envelopes carry correlationId (the trace) and causationId (the immediate parent message). The exporter uses them to stitch spans into a tree:

action.dispatched opens a span and keys it by envelope.messageId.
Every downstream record (event.published, actor.transitioned, projection.folded, reaction.failed) carries an envelope whose causationId points back at the originating action's messageId. The exporter does a Map.get(causationId) and attaches the record as a span event on that open action span.
reaction.fired and external.call.* open their own spans for sub-operations that have meaningful duration.
action.completed / dlq.recorded look the parent up by messageId and close it with OK or ERROR.

If a close record arrives without a matching open span (listener attached mid-flight, or messageId collision after a process restart), the exporter emits an orphan span tagged nwire.orphan = true so you don't silently lose data. The full mapping table lives in the OpenTelemetry guide.

What you'll see

Open a single request in Tempo or Jaeger and you'll get a tree like this:

nwire.action submissions.submit-answer            [124 ms]
├── event submissions.answer-submitted             (span event)
├── actor.transitioned submission/abc → submitted  (span event)
├── projection.folded submissions-by-student       (span event)
├── nwire.reaction submissions.answer-submitted   [38 ms]
│   └── nwire.action mastery.recompute-from-event  [22 ms]
│       └── projection.folded mastery-by-student   (span event)
└── nwire.external stripe.create-payment-intent   [311 ms]

Every node carries the full envelope as attributes — nwire.correlation_id, nwire.message_id, nwire.tenant, nwire.user_id, plus any Studio-aware metadata (persona, journeyStep, slo) you declared on defineAction. Click a failing span and you get the error name, message, stack, and attempt count without leaving the trace UI.

Production checklist

Sampling at the SDK. Wrap your NodeTracerProvider with ParentBasedSampler({ root: new TraceIdRatioBasedSampler(0.1) }) for 10% head sampling. Configure tail sampling at the collector to retain spans containing action.failed, reaction.failed, or dlq.recorded events at 100%.
Batch interval. BatchSpanProcessor's defaults (5s schedule delay, 30s export timeout, 2048 max queue) are fine for most workloads. Tune scheduledDelayMillis down to 1s if your end-to-end latency SLO is tight, up to 30s if you're paying per-request to the backend.
PII redaction. Add a SpanProcessor that strips sensitive attribute keys before export — emails, tokens, free-text inputs. Nwire never puts the action payload into span attributes by default, but custom event metadata can leak.
High-volume kinds. Use the kinds: [...] filter on attachOtelExporter to keep only action.*, external.call.*, and dlq.recorded in production. Studio's local stream stays untouched; only the OTLP export is filtered.
Detach before shutdown. Always call the returned detach() before provider.shutdown() so still-open spans are closed with a nwire.unsubscribed event rather than truncated mid-export.
Resource attributes. Always set service.name, service.version (your git SHA), and deployment.environment. Without them, your APM groups every deploy together.

Don't import @opentelemetry/* in domain code

The exporter lives at the wire boundary on purpose. Your modules (src/<bc>/...) should never import from @opentelemetry/api or any SDK package directly — that couples your domain to a transport concern and breaks the multi-transport invariant (same module, monolith or split, HTTP or queue). If you need custom span attributes for a specific action, declare them as Studio metadata on defineAction (persona, journeyStep, slo) — the exporter will surface them automatically.

Production observability with OpenTelemetry ​

What you'll wire up ​

Install ​

Wire it in ​

Causation chains ​

What you'll see ​

Production checklist ​

See also ​