Telemetry and usage

Ask a verifiable question

A slow answer can come from the model, a second tool turn, the network or the workflow. Final prose cannot distinguish them. Instrumentation observes these steps with identifiers and durations, then asks a bounded question: where was time spent and which step failed? The answer establishes neither prose correctness nor a persistent business effect.

• Business journal and effect evidence

Build and read correlation

ActivitySource starts the execution root. Descendants share TraceId; each span has SpanId and ParentSpanId. Read the root first, find children by parent, then compare status and duration. Nested durations overlap: adding them would count the same time more than once. A failed root does not mean an already committed transaction was undone.

public static TracerProvider CreateProvider(Action<SafeSpan> write) => Sdk.CreateTracerProviderBuilder()
    .AddSource("Maf.Renovation", "Maf.NativeAgent", "Microsoft.Agents.AI*", "Microsoft.Extensions.AI*")
    .AddHttpClientInstrumentation()
    .AddProcessor(new SimpleActivityExportProcessor(new SafeExporter(write))).Build();

/// <summary>Adds the native MAF decorator with sensitive data explicitly disabled.</summary>
public static AIAgent Instrument(AIAgent agent) => agent.AsBuilder()
    .UseOpenTelemetry(sourceName: "Maf.NativeAgent", configure: options => options.EnableSensitiveData = false).Build();

shared/Maf.Observability/Telemetry.cs · L20–L27 · 9a895c94534c

Source code · Maf.Observability · Folder

• shared/Maf.Observability/Maf.Observability.csproj

The native MAF UseOpenTelemetry decorator explicitly disables sensitive data. The workflow uses WithOpenTelemetry and HTTP uses outgoing instrumentation. The host adds execution, model, workflow and tool. Demo420 deterministic mode exercises the workflow and SQLite without a model request: absent HTTP or agent spans in that mode are expected. Live mode requires an available provider and an actual get_review_policy call before the workflow.

Preserve unknown usage

ObservedChatClient observes below the native tool loop. Each transport request produces a report; a second call after a tool result must remain visible. For streaming, usage deltas are aggregated once at completion. Failure, cancellation or abandonment leaves final usage unknown, including work the remote provider may have continued. The observer does not retry requests.

var input = usage?.InputTokenCount;
var output = usage?.OutputTokenCount;
long? derived = input.HasValue && output.HasValue ? checked(input.Value + output.Value) : null;
decimal? cost = rate is not null && input.HasValue && output.HasValue
    ? (input.Value * rate.InputPerMillion + output.Value * rate.OutputPerMillion) / 1_000_000m : null;
return new(input, output, usage?.TotalTokenCount, derived,
    contextCharacters.HasValue ? ((long)contextCharacters.Value + 3) / 4 : null,
    cost, rate?.Currency, rate?.Version);

shared/Maf.Observability/Consumption.cs · L16–L23 · 9a895c94534c

Source code · Maf.Observability · Folder

• shared/Maf.Observability/Maf.Observability.csproj

These values come from Maf.Observability.Tests fixtures MissingUsageIsNotZero and TotalsAndCostsAreAttributed; they illustrate the calculator contract. Provider total and derived sum remain separate fields. The integer (characters + 3) / 4 formula is a heuristic over the supplied context; it excludes instructions and the question when they are outside that text.

Display coverage before a total

int inputs = requests.Count(r => r.Input.HasValue);
int outputs = requests.Count(r => r.Output.HasValue);
long knownInput = requests.Where(r => r.Input.HasValue).Sum(r => r.Input!.Value);
long knownOutput = requests.Where(r => r.Output.HasValue).Sum(r => r.Output!.Value);
return new(requests.Count, inputs, outputs, knownInput, knownOutput,
    inputs == requests.Count && inputs > 0 ? knownInput : null,
    outputs == requests.Count && outputs > 0 ? knownOutput : null);

shared/Maf.Observability/UsageLedger.cs · L23–L29 · 9a895c94534c

Source code · Maf.Observability · Folder

• shared/Maf.Observability/Maf.Observability.csproj

UsageLedger shows how many requests have known counters, known subtotals and nullable complete totals. In fixture LedgerSeparatesKnownSubtotalsAndCompleteTotals, requests (Input=0, Output=2) and (Input=5, Output=null) yield KnownOutputSubtotal=2 but CompleteOutputTotal=null. With no request, the complete total is also null: no free invocation was measured.

No rate is configured by default. TokenRate carries two nonnegative per-million rates, currency and a version/source supplied by the caller. EstimatedCost requires actual Input and Output counters; it applies that rate assumption without becoming an invoice. Cache/reasoning categories, GPU, electricity and hosting are not priced. Low latency or short context alone cannot establish savings.

Purely fictional TotalsAndCostsAreAttributed example: Input=10 and Output=20 with TokenRate(1, 2, "TEST", "fixture-only-2026-10-24") yield (10×1 + 20×2)/1,000,000 = 0.00005 TEST. These fixture rates belong to no provider. Without both actual counters this calculation stays null even when a character estimate is available.

Reduce data before export

public static SafeSpan Project(Activity activity) => new(
    activity.Source.Name == Source.Name ? "host" : activity.Source.Name.StartsWith("System.Net.Http", StringComparison.Ordinal)
        ? "http" : activity.Source.Name.StartsWith("Microsoft.Agents.AI.Workflows", StringComparison.Ordinal)
            ? "maf.workflow" : activity.Source.Name.StartsWith("Microsoft.Extensions.AI", StringComparison.Ordinal)
                ? "maf.model" : "maf.agent",
    activity.Source.Name == Source.Name && activity.OperationName is "execution" or "tool" or "workflow" or "model"
        ? activity.OperationName : "operation",
    activity.TraceId.ToString(), activity.SpanId.ToString(), activity.ParentSpanId.ToString(),
    activity.Duration.TotalMilliseconds, activity.Status.ToString());

shared/Maf.Observability/Telemetry.cs · L30–L38 · 9a895c94534c

Source code · Maf.Observability · Folder

• shared/Maf.Observability/Maf.Observability.csproj

SafeSpan is a closed projection: source and operation categories, identifiers, duration and status. It drops arbitrary names, tags, baggage, events, URLs, resource attributes, exception messages and stacks. A tool or endpoint can carry a secret in its name, so filtering prompts alone would be insufficient. Do not replace this sole installed exporter with raw ConsoleExporter or unfiltered OTLP.

Bound what was observed

The guarantee concerns this exporter; other in-process listeners, application logs or console transcripts have their own policy. Business answers shown to the learner do not become safe traces when the entire console is redirected. Sampled export can omit steps. No collector, Docker or cloud account is required; production needs its own coverage, access and retention qualification.

Practice reading

dotnet test tests/Maf.Observability.Tests/Maf.Observability.Tests.csproj -m:1 -c Release --no-build --no-restore

• Demo420 — Safe Telemetry

• Demo440 — Context Budget

• Nullable usage and journal