-
Notifications
You must be signed in to change notification settings - Fork 37.3k
Add telemetry instrumentation for MCP server and tool picker interactions #287192
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Add telemetry instrumentation for MCP server and tool picker interactions #287192
Conversation
- Add chat.toolPicker.mcpServerToggle event for server enable/disable - Add chat.toolPicker.toolToggle event for individual tool changes - Add chat.toolPicker.action event for action button usage - Include virtualMode detection when tool count exceeds threshold - Follow VS Code telemetry privacy guidelines with hashed IDs - Add state change tracking to prevent duplicate events
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds comprehensive telemetry instrumentation to the VS Code tool picker to track MCP (Model Context Protocol) server and tool enablement/disablement interactions. The implementation includes detection of "virtual mode" when users exceed the configured tool threshold.
Changes:
- Added three new telemetry events:
chat.toolPicker.mcpServerToggle,chat.toolPicker.toolToggle, andchat.toolPicker.action - Implemented virtual mode detection based on tool count threshold comparison
- Added state tracking to prevent duplicate telemetry events by comparing previous and current states
- Instrumented all action buttons (add MCP server, configure, install extension, etc.) with telemetry
Comments suppressed due to low confidence (1)
src/vs/workbench/contrib/chat/browser/actions/chatToolPicker.ts:73
- The comment states "Hashed tool ID" but the code at line 687 sends
item.iddirectly without any hashing. Tool IDs contain identifying information about the tool and its source. Either the IDs should actually be hashed before being sent to telemetry, or the classification comment should be updated to accurately reflect that raw IDs are being sent.
toolId: { classification: 'SystemMetaData'; purpose: 'FeatureInsight'; comment: 'Hashed tool ID' };
| type McpServerToggleClassification = { | ||
| owner: 'digitarald'; | ||
| comment: 'Tracks when MCP servers are enabled/disabled in tool picker'; | ||
| serverId: { classification: 'SystemMetaData'; purpose: 'FeatureInsight'; comment: 'Hashed MCP server ID' }; |
Copilot
AI
Jan 13, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The comment states "Hashed MCP server ID" but the code at line 677 sends item.id directly without any hashing. MCP server IDs can be user-provided strings (e.g., "my-mcp-server-xyz") which may contain sensitive or identifiable information. Either the IDs should actually be hashed before being sent to telemetry, or the classification comment should be updated to accurately reflect that raw IDs are being sent.
This issue also appears in the following locations of the same file:
- line 73
src/vs/workbench/contrib/chat/browser/actions/chatToolPicker.ts
Outdated
Show resolved
Hide resolved
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Summary
This PR adds comprehensive telemetry instrumentation to the VS Code tool picker to track MCP server and individual tool enablement/disablement interactions. The telemetry includes detection of "virtual mode" when users exceed the configured tool threshold.
New Telemetry Events:
1.
chat.toolPicker.mcpServerToggle2.
chat.toolPicker.toolToggle3.
chat.toolPicker.actionKey Features:
Files Changed:
src/vs/workbench/contrib/chat/browser/actions/chatToolPicker.tsTesting:
✅ Passes TypeScript compilation
✅ Follows VS Code telemetry patterns
✅ Includes proper privacy classifications