Ensuring Application Insights telemetry works for Teams bots

by

Summary

Copilot Studio agents send telemetry to Application Insights only when the instrumentation is enabled in the runtime environment. The testing channel (pva-studio) logs automatically, but agents deployed to Microsoft Teams run in a different host that requires explicit activation of Application Insights via the Production Settings in the bot’s Azure resources.

Root Cause

  • Telemetry flag disabled in the production deployment configuration.
  • The connection string was set in the authoring (studio) environment, but the runtime app service / Azure Bot Service does not inherit it.
  • Default logging level for production is set to Error/Warning, so custom events are filtered out.

Why This Happens in Real Systems

  • Separation of environments: Studio uses a sandbox that auto‑injects the AI SDK, while Teams bots run on Azure resources that need explicit configuration.
  • Security by default: Microsoft prevents accidental data leakage from production workloads, so telemetry must be opt‑in.
  • Configuration drift: Deploy pipelines often overwrite settings, removing the connection string unless it is locked in the Azure portal or ARM template.

Real-World Impact

  • Missing observability: No insight into user conversations, performance bottlenecks, or error rates in production.
  • Delayed incident response: Engineers cannot trigger alerts or dashboards for real users.
  • Compliance risk: Lack of telemetry might violate internal monitoring policies or SLAs.

Example or Code (if necessary and relevant)

{
  "APPLICATIONINSIGHTS_CONNECTION_STRING": "InstrumentationKey=xxxx;IngestionEndpoint=https://xxxx.api.applicationinsights.azure.com/",
  "APPINSIGHTS_INSTRUMENTATIONKEY": "",
  "TelemetryEnabled": true
}

How Senior Engineers Fix It

  • Verify runtime settings in Azure Portal → Configuration → add the APPLICATIONINSIGHTS_CONNECTION_STRING.
  • Enable telemetry in the Bot Framework Composer/Power Virtual Agents Production Settings: toggle “Enable Application Insights”.
  • Update deployment scripts (ARM, Bicep, Terraform) to include the connection string and TelemetryEnabled=true.
  • Set appropriate sampling and log level to capture customEvents.
  • Redeploy the bot and confirm data appears in the customEvents table for channelId values like msteams.

Why Juniors Miss It

  • Assume studio settings propagate automatically to production.
  • Overlook the separate Azure Bot Service configuration and focus only on the authoring UI.
  • Skip validation of environment variables after deployment, trusting the CI/CD pipeline.
  • Misinterpret the lack of data as a bug in the SDK rather than a missing configuration step.

Leave a Comment