telemetry-instrumentation

작성자: microsoft

vscode-documentdb 확장에서 텔레메트리를 위한 계측 패턴입니다. 기능에 텔레메트리를 추가하거나 기존 텔레메트리를 속성으로 보강할 때 사용합니다.

npx skills add https://github.com/microsoft/vscode-documentdb --skill telemetry-instrumentation

Telemetry Instrumentation

How to instrument code in vscode-documentdb to produce actionable telemetry.

What the Framework Gives You for Free

The @microsoft/vscode-azext-utils library automatically captures these — never add them manually:

Auto-capturedDetails
Durationmeasurements.duration — wall-clock ms from callback start to end
Resultproperties.result — "Succeeded", "Failed", or "Canceled"
Error infoproperties.error (name), properties.errorMessage (message)
Cancel detectionUserCancelledError sets result to "Canceled" automatically
Machine/session IDsVSCodeMachineId, VSCodeSessionId — always present
Extension metadataExtension name, version, VS Code version

Do not re-measure duration, set result, or catch errors just for telemetry — the wrapper handles it.

Core API

callWithTelemetryAndErrorHandling(eventId, callback)

The primary wrapper. Creates a telemetry event, runs the callback, reports result + duration + errors. The callback's return value is passed through, so you can wrap any function call directly:

import { callWithTelemetryAndErrorHandling } from '@microsoft/vscode-azext-utils';

// Simple — wrap logic, telemetry is automatic
await callWithTelemetryAndErrorHandling('myFeature.doSomething', async (context) => {
  context.telemetry.properties.someProperty = 'value';
  context.telemetry.measurements.itemCount = items.length;
  // ... your logic — errors are caught and reported automatically
});

// With return value — no need for intermediate variables
const result = await callWithTelemetryAndErrorHandling('myFeature.query', async (context) => {
  context.telemetry.properties.queryType = 'find';
  return doWork(); // return value passes through to the caller
});

Event name convention: use dot-separated hierarchy: connect, connect.getmetadata, connect.promptForCredentials.

Command Registration (auto-telemetry)

Every registered command gets a telemetry event automatically. Prefer the correlation-wrapped variants:

import { registerCommandWithTreeNodeUnwrapping } from '@microsoft/vscode-azext-utils';
import { withTreeNodeCommandCorrelation, withCommandCorrelation } from '../../utils/commandTelemetry';

// Tree node commands — unwraps tree item + tracks journeyCorrelationId
registerCommandWithTreeNodeUnwrapping(
  'vscode-documentdb.command.connectionsView.myCommand',
  withTreeNodeCommandCorrelation(myCommandHandler),
);

// Plain commands — tracks journeyCorrelationId
registerCommand('vscode-documentdb.command.myCommand', withCommandCorrelation(myCommandHandler));

For critical commands where errors need modal dialogs:

import { registerCommandWithTreeNodeUnwrappingAndModalErrors } from '../../utils/commandErrorHandling';

tRPC Procedures (auto-telemetry)

Webview tRPC calls get telemetry automatically via publicProcedureWithTelemetry. Each call emits documentDB.rpc.{type}.{path} with result, error, abort tracking. The DocumentDB TelemetryRunner contributes the full IActionContext to ctx.actionContext (narrow ctx to WithTelemetry<RouterContext> to read it):

myProcedure: publicProcedureWithTelemetry.input(schema).query(async ({ ctx, input }) => {
    const myCtx = ctx as WithTelemetry<RouterContext>;
    myCtx.actionContext.telemetry.properties.someDetail = 'value';
    myCtx.actionContext.telemetry.measurements.resultCount = results.length;
    return results;
}),

Naming Conventions

Property & measurement names

  • Default: use camelCase — authMethod, connectionMode, discoveryProviderId
  • Namespaced metadata: when spreading a flat metadata object into telemetry (e.g., cluster metadata collected once and reused), use prefix_camelCase — domainInfo_isAzure, serverInfo_version. The prefix acts as a namespace. This pattern exists in getClusterMetadata.ts and is matched by dashboard queries.
  • Never mix within one prefix: task_initializationCompleted (mixed snake + camelCase) is wrong — use task_initializationCompleted → taskInitializationCompleted or task_initialization_completed consistently.
  • Correlation IDs: always use camelCase — journeyCorrelationId, connectionCorrelationId.

Boolean properties

Always use the ternary pattern — it makes the intent explicit:

// ✅ Correct — clear intent, consistent
context.telemetry.properties.hasFolders = totalFolders > 0 ? 'true' : 'false';
context.telemetry.properties.isFirstConnection = isFirst ? 'true' : 'false';

// ❌ Avoid — works but less readable, can accidentally stringify non-booleans
context.telemetry.properties.isWindows = isWindows.toString();

Numbers: property or measurement?

Rule: if you'd ever want to compute P50, average, sum, or max over it, use a measurement. Numbers stored as string properties can't be aggregated in analytics without casting.

// ✅ Measurement — enables P50(pageNumber), avg(itemCount)
context.telemetry.measurements.pageNumber = input.pageNumber;
context.telemetry.measurements.itemCount = items.length;

// ❌ Wrong — forces dashboard to todouble(Properties.pageNumber)
context.telemetry.properties.pageNumber = input.pageNumber.toString();

Exception: numeric values used purely as categories (e.g., an error code used for grouping, not aggregation) can be properties.

Properties vs Measurements

UseTypeExample
Properties (string)Categorical data, flags, identifiersserverType, view, authMethod, hasFolders
Measurements (number)Counts, sizes, durations, ratesitemCount, loadTimeMs, retryAttempts

Properties — what to capture

// Feature context — what variant/mode is being used
context.telemetry.properties.view = 'connectionsView';
context.telemetry.properties.experience = node.experience.api; // 'vCore' | 'RU'
context.telemetry.properties.authMethod = 'connectionString';

// Boolean flags — always use 'true'/'false' strings
context.telemetry.properties.hasFolders = totalFolders > 0 ? 'true' : 'false';
context.telemetry.properties.isFirstConnection = isFirst ? 'true' : 'false';

// Classification of inputs or results
context.telemetry.properties.connectionMode = 'documentdb'; // what was selected
context.telemetry.properties.terminalType = 'PowerShell'; // what environment

// Error classification beyond the auto-captured error
context.telemetry.properties.connectionErrorType = 'auth'; // error category
context.telemetry.properties.parseError = 'SyntaxError'; // specific error type

// View identification — set for user-initiated or view-specific operations
// Skip for background tasks, activation events, or cross-cutting concerns
context.telemetry.properties.view = Views.ConnectionsView;

Measurements — what to capture

// Counts — how many items were involved
context.telemetry.measurements.itemCount = results.length;
context.telemetry.measurements.totalConnections = connections.length;
context.telemetry.measurements.savedConnections = rootItems.length;

// Requested vs returned — reveals pagination, limits, empty results
context.telemetry.measurements.pageSize = input.pageSize;
context.telemetry.measurements.documentCount = docs.length;

// Sub-durations — where time is spent (the total is auto-captured)
context.telemetry.measurements.loadTimeMs = Date.now() - startTime;
context.telemetry.measurements.mainFileLoad = (loadEnd - loadStart) / 1000;

// Attempts/retries
context.telemetry.measurements.connectionViewActivationAttempts = attempt + 1;
context.telemetry.measurements.cleanupIterations = iteration;

// Breakdown counts — for multi-category results
context.telemetry.measurements.createRecommendationCount = createCount;
context.telemetry.measurements.dropRecommendationCount = dropCount;

// Structural complexity — depth, nesting
context.telemetry.measurements.maxFolderDepth = maxDepth;

Dynamic property/measurement names

When the set of categories is data-driven, use a prefix:

context.telemetry.measurements[`${zonePrefix}_Connections`] = connectionsInZone;
context.telemetry.measurements[`${zonePrefix}_Folders`] = foldersInZone;

Sub-step Telemetry

Use nested callWithTelemetryAndErrorHandling inside a parent event to get fine-grained timing and success/failure per step:

await callWithTelemetryAndErrorHandling('connect', async (context) => {
  context.telemetry.properties.view = 'discoveryView';

  // Sub-step: prompting — gets its own duration, result, errors
  await callWithTelemetryAndErrorHandling('connect.promptForCredentials', async (subCtx) => {
    subCtx.errorHandling.rethrow = true; // let parent see failures
    await wizard.prompt();
  });

  // Sub-step: metadata collection (fire-and-forget, non-blocking)
  void callWithTelemetryAndErrorHandling('connect.getmetadata', async (metaCtx) => {
    const metadata = await collectMetadata();
    metaCtx.telemetry.properties = { ...metaCtx.telemetry.properties, ...metadata };
  });
});

When to use sub-steps:

  • Operation has distinct phases (prompt → connect → collect metadata)
  • You need per-phase duration/success data
  • A sub-step can fail independently without failing the parent

Naming: use parent event as prefix: connect.promptForCredentials, connect.getmetadata, connect.staticmetadata.

Correlation IDs — Linking Related Events

journeyCorrelationId — user journey across commands

Links a chain of user actions that form a logical journey (e.g., discover → select → connect → browse). Generated once at the start and propagated through tree items:

// Generate at the journey start (e.g., when a discovery tree root is created)
this.journeyCorrelationId = randomUUID();

// Tree items carry it to children
new MongoRUResourceItem(this, account, this.journeyCorrelationId);

// Commands pick it up automatically via withTreeNodeCommandCorrelation/withCommandCorrelation
// OR manually in a callWithTelemetryAndErrorHandling:
if (this.journeyCorrelationId) {
  context.telemetry.properties.journeyCorrelationId = this.journeyCorrelationId;
}

Use journeyCorrelationId when: a user flow spans multiple command invocations that should be analyzed together (discovery → connect → browse, folder operations in sequence).

connectionCorrelationId — linking connection sub-events

Links the connect, connect.staticmetadata, and connect.getmetadata events for a single connection attempt:

this.connectionCorrelationId = randomUUID();

void callWithTelemetryAndErrorHandling('connect.staticmetadata', async (context) => {
  context.telemetry.properties.connectionCorrelationId = this.connectionCorrelationId;
  // ...domain metadata
});

void callWithTelemetryAndErrorHandling('connect.getmetadata', async (context) => {
  context.telemetry.properties.connectionCorrelationId = this.connectionCorrelationId;
  // ...server metadata
});

Use connectionCorrelationId when: an operation emits multiple independent events that must be joined for analysis (especially fire-and-forget sub-events).

Session-scoped correlation — ongoing sessions

For features where a user session produces many events (e.g., shell, REPL, interactive query editing), generate a session-scoped correlation ID and attach it to every event in that session. This enables aggregation (total commands per session, session duration as max-min timestamps):

const sessionId = randomUUID();

// Each action within the session carries the same ID
context.telemetry.properties.shellSessionId = sessionId;
context.telemetry.measurements.commandIndex = commandCount++;

Controlling Telemetry Behavior

Suppressing noisy events

// High-frequency events (keystroke handlers, document change listeners)
context.telemetry.suppressIfSuccessful = true; // only emit on error/cancel

// Fully silent (internal bookkeeping)
context.telemetry.suppressAll = true;

Error handling control

// Don't show error notification to user (background/internal operations)
context.errorHandling.suppressDisplay = true;

// Re-throw so the parent caller sees the error
context.errorHandling.rethrow = true;

// Don't offer "Report Issue" button
context.errorHandling.suppressReportIssue = true;

Activation events

Mark startup/initialization telemetry so it can be filtered out of active-usage analysis:

context.telemetry.properties.isActivationEvent = 'true';

Common Mistakes to Avoid

❌ Don't measure duration manually — the framework does it

The framework auto-captures measurements.duration for every callWithTelemetryAndErrorHandling event. Never add Date.now() tracking for the same purpose:

// ❌ Wrong — redundant manual timing
const startTime = Date.now();
await callWithTelemetryAndErrorHandling('myEvent', async (context) => {
  await doWork();
  context.telemetry.measurements.durationMs = Date.now() - startTime; // duplicates framework
});

// ✅ Correct — framework captures duration automatically
await callWithTelemetryAndErrorHandling('myEvent', async (context) => {
  await doWork();
  // duration is auto-captured — no manual timing needed
});

Exception: sub-durations within a single event (e.g., "time spent on init vs. time on eval") are valid because they measure a phase, not the total.

❌ Don't create separate success/failure telemetry blocks

Wrap the operation in a single callWithTelemetryAndErrorHandling. The framework sets result=Succeeded or result=Failed automatically:

// ❌ Wrong — two fire-and-forget blocks, error rethrow inside void is swallowed
try {
  const result = await doWork();
  void callWithTelemetryAndErrorHandling('myEvent', async (ctx) => {
    ctx.telemetry.properties.isError = 'false';
  });
} catch (error) {
  void callWithTelemetryAndErrorHandling('myEvent', async (ctx) => {
    ctx.telemetry.properties.isError = 'true';
    throw error; // ⚠️ swallowed — void discards the promise
  });
}

// ✅ Correct — single wrapper, framework handles success/failure
await callWithTelemetryAndErrorHandling('myEvent', async (context) => {
  context.errorHandling.suppressDisplay = true;
  context.errorHandling.rethrow = true; // let caller handle display
  context.telemetry.properties.someProperty = 'value';
  const result = await doWork(); // errors auto-set result=Failed
  context.telemetry.properties.resultType = result.type;
});

❌ Don't use suppressIfSuccessful on events you want to measure

suppressIfSuccessful = true means the event is only emitted on error. Never use it on events where you need to count successful occurrences:

// ❌ Wrong — successful completions are silently dropped
registerCommand('completionAccepted', (context) => {
  context.telemetry.properties.category = category;
  context.telemetry.suppressIfSuccessful = true; // BUG: won't emit on success
});

// ✅ Correct — every acceptance is tracked
registerCommand('completionAccepted', (context) => {
  context.telemetry.properties.category = category;
  // no suppress — we want to count every acceptance
});

When suppressIfSuccessful IS correct: high-frequency background operations where you only care about failures (e.g., keystroke handlers, document change listeners, periodic health checks).

❌ Don't throw errors inside void callWithTelemetryAndErrorHandling

When using fire-and-forget (void) telemetry, thrown errors are swallowed because nobody awaits the promise. The framework captures the error in telemetry properties but the throw has no effect on the calling code:

// ❌ Wrong — throw inside void is swallowed, no effect on caller
void callWithTelemetryAndErrorHandling('myEvent', async (context) => {
  throw error; // captured in telemetry but NOT re-thrown to caller
});
// code continues here regardless

// ✅ Correct for fire-and-forget — just set properties, don't throw
void callWithTelemetryAndErrorHandling('myEvent', async (context) => {
  context.telemetry.properties.shellSessionId = sessionId;
  // fire-and-forget: no throw, no await needed
});

// ✅ Correct when you need error propagation — use await
await callWithTelemetryAndErrorHandling('myEvent', async (context) => {
  context.errorHandling.rethrow = true;
  await riskyOperation(); // error propagates to caller via rethrow
});

Masking Sensitive Data

Never let connection strings, passwords, tokens, or hostnames appear in telemetry.

import { maskSensitiveValuesInTelemetry } from '../../documentdb/utils/connectionStringHelpers';

// Mask all sensitive parts of a parsed connection string
maskSensitiveValuesInTelemetry(context, parsedConnectionString);

// Or mask individual values
context.valuesToMask.push(password, token);

Tree View Events

Tree getChildren calls emit telemetry. Always set context so analytics can distinguish navigation levels:

return callWithTelemetryAndErrorHandling('getChildren', async (context) => {
  context.telemetry.properties.parentNodeContext = element ? (await element.getTreeItem()).contextValue : 'root';
  context.telemetry.measurements.childrenCount = children.length;
  return children;
});

Note: Root-level getChildren fires automatically when panels become visible (passive rendering). Analytics filters these with parentNodeContext != "root" — always set this property.

What Data Points to Capture — Decision Guide

When instrumenting a feature, consider these categories:

For any operation

  • What variant/mode was used (property) — enables segmentation
  • How many items were involved (measurement) — reveals usage scale
  • Server/environment type when relevant (property) — different backends behave differently

For slow or async operations

  • Sub-step durations (measurements) — total duration is auto-captured, but breakdowns show where time goes
  • Retry/attempt counts (measurement) — reveals flakiness
  • Timeout vs success (property) — important for polling patterns

For operations that return results

  • Requested vs returned count — reveals empty results, pagination patterns, limit hits
  • Result classification — category/type breakdown of what was returned

For multi-step flows

  • Correlation ID — link events into a journey
  • Step identification — which step succeeded/failed (sub-step events or lastStep property)
  • Cancellation point — where the user abandoned the flow

For session-based features (shell, REPL, interactive editors)

  • Session correlation ID — group all events from one session
  • Command/action index — sequence number within the session (enables count-per-session aggregation since there's typically no "session end" event)
  • Server type — what kind of server the session targets

For error recovery flows

  • Recovery success flag — did the user successfully recover (e.g., reconnected = 'true')
  • Error-then-recovery correlation — link the original error to the recovery action

When to Override result Manually

The wrapper auto-sets result to "Succeeded" / "Failed" / "Canceled". Manual override is required in two cases:

  1. Early return on invalid state — function returns without throwing, but the operation logically failed:
if (!isValidFormat) {
  context.telemetry.properties.result = 'Failed';
  context.telemetry.properties.errorReason = 'invalidNodeIdFormat';
  return; // no throw → wrapper would report "Succeeded" without the override
}
  1. Non-throwing error signals — e.g., tRPC returns { ok: false } instead of throwing:
if (!result.ok) {
  context.telemetry.properties.result = 'Failed';
  context.telemetry.properties.error = result.error.name;
}

Do NOT override result when you also throw — the wrapper handles that automatically.

Common Pitfalls

  • Don't measure duration — callWithTelemetryAndErrorHandling does this already
  • Don't set result when throwing — the wrapper sets "Succeeded" / "Failed" / "Canceled" automatically from the throw. Only override result on early-return failures or non-throwing error paths (see above)
  • Don't catch errors just for telemetry — the wrapper catches and records them
  • Don't log sensitive data — use maskSensitiveValuesInTelemetry or context.valuesToMask
  • Don't create a new callWithTelemetryAndErrorHandling for every minor step — only when you need separate duration/result tracking for a step
  • Don't forget context.errorHandling.rethrow = true in sub-steps that should propagate failures to the parent
  • Don't use suppressIfSuccessful on events you need to count — it drops successful events entirely
  • Don't store numbers as string properties — use measurements for anything you'd want to aggregate (P50, sum, avg). Use .toString() only for numbers used as categorical grouping keys
  • Don't use .toString() for booleans — use the explicit ternary: value ? 'true' : 'false'

microsoft의 다른 스킬

oss-growth
microsoft
OSS 성장 해커 페르소나
agent-framework-azure-ai-py
microsoft
Microsoft Agent Framework Python SDK(agent-framework-azure-ai)를 사용하여 Azure AI Foundry 에이전트를 구축합니다. AzureAIAgentsProvider로 지속적 에이전트를 만들 때, 호스팅 도구(코드 인터프리터, 파일 검색, 웹 검색)를 사용할 때, MCP 서버를 통합할 때, 대화 스레드를 관리할 때, 또는 스트리밍 응답을 구현할 때 사용합니다. 함수 도구, 구조화된 출력, 다중 도구 에이전트를 다룹니다.
development
airunway-aks-setup
microsoft
AKS에서 AI Runway 설정 — 빈 클러스터에서 실행 중인 모델까지. 클러스터 검증, 컨트롤러 설치, GPU 평가, 공급자 설정, 첫 배포를 다룹니다. 시기: "AI Runway 설정", "AKS 클러스터 온보딩", "AI Runway 설치", "airunway 설정", "AKS에 모델 배포", "AKS에서 GPU 추론", "AKS에서 KAITO 설정", "AKS에서 LLM 실행", "AKS에서 vLLM", "AKS에서 모델 서빙 설정", "AI Runway 컨트롤러".
devops
appinsights-instrumentation
microsoft
Azure Application Insights로 웹앱을 계측하기 위한 지침입니다. 원격 분석 패턴, SDK 설정, 구성 참조를 제공합니다. WHEN: 앱 계측 방법, App Insights SDK, 원격 분석 패턴, App Insights란 무엇인가, Application Insights 지침, 계측 예시, APM 모범 사례.
devops
applicationinsights-web-ts
microsoft
브라우저/웹 앱을 Application Insights JavaScript SDK(@microsoft/applicationinsights-web)로 계측합니다. Real User Monitoring(RUM) — 페이지 뷰, 클릭, AJAX/fetch 종속성, 예외, 사용자 지정 이벤트, 백엔드 OpenTelemetry 트레이스와 상관관계가 있는 브라우저 측 GenAI 에이전트 트레이스에 사용합니다. SDK Loader Script 및 npm 설정, 프레임워크 확장(React, React Native, Angular), Click Analytics, 텔레메트리 이니셜라이저, 브라우저에서 생성된 에이전트/도구/모델 스팬에 대한 OTel GenAI 의미론적 규칙을 다룹니다.
devops
azure-ai-anomalydetector-java
microsoft
Azure AI Anomaly Detector SDK for Java로 이상 탐지 애플리케이션을 구축하세요. 단변량/다변량 이상 탐지, 시계열 분석 또는 AI 기반 모니터링을 구현할 때 사용하세요.
development
azure-ai-language-conversations-py
microsoft
azure-ai-language-conversations Python SDK를 사용하여 대화형 언어 이해(CLU)를 구현합니다. ConversationAnalysisClient로 대화 의도와 엔터티를 분석하거나, NLP 기능을 구축하거나, 애플리케이션에 언어 이해를 통합할 때 사용합니다.
development
azure-ai-ml-py
microsoft
Azure Machine Learning SDK v2 for Python. ML 작업 영역, 작업, 모델, 데이터 세트, 컴퓨팅 및 파이프라인에 사용합니다. 트리거: "azure-ai-ml", "MLClient", "workspace", "model registry", "training jobs", "datasets".
development