Skip to content

[Change] Output Snapshot Stability #280

@djm81

Description

@djm81

Why

Users who script SpecFact in CI/CD pipelines rely on stable output: help text format, error message wording, JSON/YAML structured output shapes. Today, a refactoring PR can silently change help text or error phrasing with no test catching it. Snapshot testing makes output changes explicit, reviewable, and intentional — treating help text and structured output as versioned contracts.

What Changes

  • NEW: syrupy dev dependency for pytest snapshot testing
  • NEW: Snapshot tests for all command --help outputs
  • NEW: Snapshot tests for structured output shapes (JSON/YAML) and key error messages
  • EXTEND: CI pipeline to reject unreviewed snapshot changes
  • EXTEND: pyproject.toml with syrupy configuration and hatch scripts

Acceptance Criteria

  • Syrupy integrated and configured
  • Snapshot tests exist for all command group help text
  • Snapshot tests exist for key structured outputs and error messages
  • hatch run snapshot-update and hatch run snapshot-check scripts work
  • Documentation describes snapshot update workflow

OpenSpec Change Proposal: cli-val-02-output-snapshot-stability

Metadata

Metadata

Assignees

No one assigned

    Labels

    QAQuality Assurancechange-proposalProposal for a new changeenhancementNew feature or request

    Projects

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions