Skip to content

Conversation

@kvenkatrajan
Copy link

Summary

This PR adds comprehensive telemetry instrumentation to the VS Code tool picker to track MCP server and individual tool enablement/disablement interactions. The telemetry includes detection of "virtual mode" when users exceed the configured tool threshold.

New Telemetry Events:

1. chat.toolPicker.mcpServerToggle

  • Triggered when: MCP servers are enabled/disabled in the tool picker
  • Data: serverId (hashed), serverLabel, enabled, virtualMode

2. chat.toolPicker.toolToggle

  • Triggered when: Individual tools are enabled/disabled in the tool picker
  • Data: toolId (hashed), toolName, toolSource, enabled, virtualMode

3. chat.toolPicker.action

  • Triggered when: Action buttons are clicked in the tool picker
  • Data: action (addMcpServer, configureMcpServer, installExtension, etc.)

Key Features:

  • Virtual mode detection when tool count exceeds threshold
  • Privacy compliant with hashed IDs
  • State change tracking to prevent duplicate events
  • Comprehensive coverage of all picker interactions

Files Changed:

  • src/vs/workbench/contrib/chat/browser/actions/chatToolPicker.ts

Testing:

✅ Passes TypeScript compilation
✅ Follows VS Code telemetry patterns
✅ Includes proper privacy classifications

- Add chat.toolPicker.mcpServerToggle event for server enable/disable
- Add chat.toolPicker.toolToggle event for individual tool changes
- Add chat.toolPicker.action event for action button usage
- Include virtualMode detection when tool count exceeds threshold
- Follow VS Code telemetry privacy guidelines with hashed IDs
- Add state change tracking to prevent duplicate events
Copilot AI review requested due to automatic review settings January 13, 2026 00:17
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds comprehensive telemetry instrumentation to the VS Code tool picker to track MCP (Model Context Protocol) server and tool enablement/disablement interactions. The implementation includes detection of "virtual mode" when users exceed the configured tool threshold.

Changes:

  • Added three new telemetry events: chat.toolPicker.mcpServerToggle, chat.toolPicker.toolToggle, and chat.toolPicker.action
  • Implemented virtual mode detection based on tool count threshold comparison
  • Added state tracking to prevent duplicate telemetry events by comparing previous and current states
  • Instrumented all action buttons (add MCP server, configure, install extension, etc.) with telemetry
Comments suppressed due to low confidence (1)

src/vs/workbench/contrib/chat/browser/actions/chatToolPicker.ts:73

  • The comment states "Hashed tool ID" but the code at line 687 sends item.id directly without any hashing. Tool IDs contain identifying information about the tool and its source. Either the IDs should actually be hashed before being sent to telemetry, or the classification comment should be updated to accurately reflect that raw IDs are being sent.
	toolId: { classification: 'SystemMetaData'; purpose: 'FeatureInsight'; comment: 'Hashed tool ID' };

type McpServerToggleClassification = {
owner: 'digitarald';
comment: 'Tracks when MCP servers are enabled/disabled in tool picker';
serverId: { classification: 'SystemMetaData'; purpose: 'FeatureInsight'; comment: 'Hashed MCP server ID' };
Copy link

Copilot AI Jan 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment states "Hashed MCP server ID" but the code at line 677 sends item.id directly without any hashing. MCP server IDs can be user-provided strings (e.g., "my-mcp-server-xyz") which may contain sensitive or identifiable information. Either the IDs should actually be hashed before being sent to telemetry, or the classification comment should be updated to accurately reflect that raw IDs are being sent.

This issue also appears in the following locations of the same file:

  • line 73

Copilot uses AI. Check for mistakes.
@pierceboggan pierceboggan assigned digitarald and unassigned dbaeumer Jan 13, 2026
@digitarald digitarald requested a review from aeschli January 13, 2026 04:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants