Consolidate infrastructure scripts and eliminate CI code duplication

[eberrigan]

Problem

Code duplication between infrastructure scripts and CI workflows reduces testability, reproducibility, and maintainability:

• init-terraform.sh logic duplicated in CI workflow (terraform-deploy.yml:109-130)
• verify-deployment.sh exists but not used by CI - verification logic duplicated inline (terraform-deploy.yml:232-350)
• Scripts in inconsistent locations (scripts/ vs lablink-infrastructure/)

Goals

1. Consolidate scripts in scripts/ folder
2. Make scripts config-aware - read from config.yaml instead of requiring manual parameters
3. Use scripts in CI - eliminate inline duplication
4. Improve testability - scripts can be tested locally with same logic CI uses

Implementation Plan

Task 1: Update CI to use init-terraform.sh

• Script already in scripts/ folder
• Update CI workflow to call script instead of inline code (replace lines 109-130)
• Verify script works from CI environment
• Update documentation

Task 2: Move and Enhance verify-deployment.sh

Location:

• Move lablink-infrastructure/verify-deployment.sh → scripts/verify-deployment.sh

Make Config-Aware:

• Read all config values: dns.*, ssl.*, app.region
• Auto-construct FQDN from DNS pattern + environment
• Read IP from Terraform outputs instead of manual parameter
• New usage: ./scripts/verify-deployment.sh <environment> (auto-detects values)
• Support backwards compatibility: ./scripts/verify-deployment.sh <domain> <ip>

Add CI Mode:

• Add --ci flag for CI-friendly output (exit codes, JSON, no colors)
• Optimize timeouts for CI environment
• Ensure proper exit codes for workflow status

Replace CI Inline Code:

• Replace "Verify DNS Resolution" step (lines 232-267) with script call
• Replace "Verify Service Health" step (lines 268-350) with script call
• Simplify workflow to single script invocation

Task 3: Update Documentation

• Update lablink-infrastructure/README.md - new paths and usage
• Update root README.md - updated file tree
• Update example config files - new script paths in comments
• Add troubleshooting guide for running scripts locally

Task 4: Testing

• Test scripts locally for all environments (dev, test, prod, ci-test)
• Test CI workflow with updated script calls
• Verify backwards compatibility with manual arguments

Benefits

• DRY: Single source of truth for init and verification logic
• Testability: Test locally with exact same code CI uses
• Reproducibility: Consistent behavior between local and CI
• Maintainability: Changes in one place, not scattered across files
• Consistency: All infrastructure scripts in scripts/ folder

Files to Change

• scripts/verify-deployment.sh (moved and enhanced)
• .github/workflows/terraform-deploy.yml (simplified)
• lablink-infrastructure/README.md (updated paths)
• README.md (updated file tree)
• Config example files (updated script path comments)

[Solix45456767]

{
"comment": "We've seen massive wins consolidating our infrastructure scripts into shared libraries—cut our CI pipeline maintenance time by ~40% and made onboarding new services trivial. The key was treating infrastructure code like product code: versioning, testing, and code review from day one. What's been your biggest pain point when managing scripts across multiple repos, and have you found a pattern that actually stuck in your org?"
}