Create automation to keep Azure Pipelines variables in sync

[lbussell]

Problem

Azure Pipelines variables configured in the ADO GUI can drift from expected values over time due to manual changes, accidental modifications, or incomplete setup.
Currently there's no automated way to detect when pipeline variables are out of sync with expected defaults.

Example: The MCR status portal has a temporary outage causing us to set waitForIngestionEnabled to false instead of true. It is easy for this change to be forgotten and not reverted back to the expected value.

Proposed Solution

Create a new tool and pipeline that:

1. Reads expected pipeline variable configurations from JSON files in the repo
2. Queries Azure DevOps (via Azure SDK/.NET) to get the current pipeline variables
3. Compares expected vs actual values to detect drift
4. Files a GitHub issue when drift is detected

Configuration

Imagine a JSON metadata file co-located with each the pipeline definition:

• Pipeline: eng/pipelines/dotnet-buildtools-image-builder-official.yml
• Metadata: eng/pipelines/dotnet-buildtools-image-builder-official.metadata.json

{
  "appliesTo": [
    {
      "name": "dotnet-buildtools-image-builder-official",
      "organization": "dnceng",
      "project": "internal",
      "pipelineId": 367
    },
    // Could optionally apply to multiple pipelines
  ],
  "variables": [
    {
      "name": "imageBuilder.pathArgs",
      "value": "",
      "allowOverride": true
    },
    {
      "name": "someOtherVariable",
      "allowOverride": true
      // Absence of value means the variable should exist but the value can be anything
    },
    {
      "name": "system.debug",
      "value": "false",
      "allowOverride": true
    },
    // etc...
  ]
}

Out of scope

• Auto-remediation of drift (setting variables back to expected values)
• Secret variables