Skip to content

[Change] CLI Validation CI Integration #283

@djm81

Description

@djm81

Why

Validation only has value if it runs automatically. Today, pr-orchestrator.yml runs a cli-validation job that only checks specfact --help. The new validation layers (snapshots, anti-patterns, acceptance tests) need to slot into CI with clear gating rules so broken CLI behavior never reaches PyPI.

What Changes

  • EXTEND: pr-orchestrator.yml with snapshot validation, black-box acceptance, and anti-pattern safety steps
  • NEW: Snapshot update CI workflow (manual trigger)
  • EXTEND: contract-test system with CLI behavior contracts as a new tier
  • EXTEND: pyproject.toml hatch scripts with combined CLI validation command

Acceptance Criteria

  • Snapshot mismatches block PR merge (hard gate)
  • Black-box acceptance failures block PR merge to main (hard gate)
  • Hypothesis edge cases are advisory (warn, don't block)
  • Snapshot update workflow available via workflow_dispatch
  • hatch run contract-test includes CLI validation tier
  • Documentation describes CI gates and snapshot update workflow

Blocked by: #280 (cli-val-02), #282 (cli-val-04)

OpenSpec Change Proposal: cli-val-05-ci-integration

Metadata

Metadata

Assignees

No one assigned

    Labels

    QAQuality Assurancechange-proposalProposal for a new changeenhancementNew feature or request

    Projects

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions