ETW for AI

MSO-Scripts Goals for AI

• Create a versatile WPR Recording Profile: (BETA\WPRP\AI.wprp)
• Create a regions-of-interest file for key ETW Providers: (BETA\WPAP\AI.regions.xlm)
• Create a WPA View Profile to effectively represent the ETW data: (BETA\WPAP\AI.wpaProfile)
• Test on a variety of representative systems.
• Create public documentation on this wiki.

Key ETW Providers

• ONNXRuntimeTraceLoggingProvider - 929DD115-1ECB-4CB5-B060-EBD4983C421D

• Microsoft.ML.ONNXRuntime - 3a26b1ff-7484-7484-7484-15261f42614d

• Microsoft.Windows.WinML - d766d9ff-112c-4dac-9247-241cf99d123f

• Microsoft.Windows.AI.MachineLearning - BCAD6AEE-C08D-4F66-828C-4C43461A033D

See: Tracing-Windows

Also run: WPR -ProfileDetails NeuralProcessing

Commentary

Currently the NPU area is fragmented and far from maturity. WPA is working only for devices using MCDM protocol. If MCDM is not supported, you should not see anything even collecting traces with the NeuralProcessing profile. MCDM is currently supported on Qualcomm and Intel NPUs. In most scenarios, AI models are running on top of the ONNX library instead of going directly to NPU drivers. This layer covers the difference between machines with and w/o NPU. ONNX runs computations on regular CPU if NPU is not available. Also, ONNX can use GPUs and some NPUs w/o MCDM support. The ONNX layer has its own set of ETW providers.

Reference

WPR / WPA: Neural Processing Support

How to Measure the Performance of AI Models Running on the Device

ONNX Runtime Performance Tuning

Microsoft Compute Driver Model overview

How to Read the NPU Utilization