Logger Package Implementation Requirements

[alfredosa]

Logger Package Implementation Requirements

Overview

Implement a flexible, extensible logging package for the Mage project that wraps Go's slog library with enhanced features including colored output, artifact logging, GitHub Actions integration, and per-devtool configurability.

Core Requirements

1. Colored Output (Label-Only Coloring)

• Requirement: Color only the log level labels (DEBUG, INFO, WARN, ERROR), not the entire message
• Rationale: Maintains readability while providing visual distinction between log levels
• Implementation: Use slog.HandlerOptions.ReplaceAttr to intercept and colorize level labels
• Color Scheme:
  • DEBUG: Gray + Bold
  • INFO: Blue + Bold
  • WARN: Yellow + Bold
  • ERROR: Red + Bold
• Auto-detection: Automatically detect TTY and disable colors for non-terminal outputs
• Override: Support explicit enable/disable via configuration

2. Artifact Logging

• Requirement: Write logs to both console and artifact file simultaneously
• Use Case: CI/CD pipelines need persistent log files for debugging and archival
• Behavior:
  • Multi-writer support (console + file)
  • Disable colors in artifact files (plain text only)
  • Append mode for artifact files
  • Optional path configuration via LOG_ARTIFACT_PATH environment variable
• File Management: Handle file creation, permissions (0644), and proper opening

3. GitHub Actions Integration

• Requirement: Support GitHub Actions-specific logging formats
• Features Needed:
  • Workflow Commands: Emit ::error::, ::warning::, ::notice::, ::debug:: commands
  • Step Summary: Write markdown summaries to $GITHUB_STEP_SUMMARY
  • Log Grouping: Support collapsible log groups with ::group:: and ::endgroup::
  • Annotations: File/line annotations for errors and warnings
• Detection: Auto-detect GitHub Actions environment via GITHUB_ACTIONS=true
• Format Example:

  ::error file=app.go,line=10,col=5::Connection failed
  ::group::Building Go modules
  ... nested logs ...
  ::endgroup::
  

4. Per-Devtool Configurability

• Requirement: Different devtools need different logging configurations
• Configuration Dimensions:
  • Log Level: Each devtool may have different verbosity needs
  • Output Format: Some tools output JSON, others text
  • Artifact Paths: Tool-specific artifact files (e.g., golangci-lint.log, prettier.log)
  • Source Location: File/line information (expensive, only when needed)
  • Output Parsing: Some tools need their output parsed and reformatted
• Configuration Sources (priority order):
  1. Explicit code configuration (highest priority)
  2. Environment variables (e.g., LOG_LEVEL_GOLANG=debug)
  3. Configuration file (future: .mage/logging.yaml)
  4. Global defaults (lowest priority)
• Example Configuration:

  // Global default
  logger.SetGlobalLevel(slog.LevelInfo)
  
  // Per-devtool override
  golangLogger := logger.New(
      logger.WithDevTool("golang"),
      logger.WithLevel(slog.LevelDebug),
      logger.WithArtifact("artifacts/golang.log"),
      logger.WithSource(true),
  )

5. Structured Logging with Context

• Requirement: Support key-value pairs and context propagation
• Features:
  • Structured key-value logging: log.Info("message", "key", value)
  • Context variants: InfoContext(ctx, "message", "key", value)
  • Logger chaining with attributes: log.With("package", "golang").Info("message")
  • Attribute inheritance through context
• Use Cases for CLI Dev Tooling:
  • Track which devtool is running (e.g., tool=golangci-lint)
  • Track operation/command being executed (e.g., operation=lint, command=check)
  • Track file paths or directories being processed (e.g., directory=/repo/cmd)
  • Track GitHub Actions job/step context (e.g., job=build, step=3)
  • Correlate logs across multiple tool invocations in a pipeline
  • Track performance metrics (e.g., duration=2.3s, files_processed=42)

6. Environment Variable Configuration

• Required Variables:
  • LOG_LEVEL: Global log level (debug, info, warn, error)
  • LOG_ARTIFACT_PATH: Path to artifact file
  • LOG_FORMAT: Output format (text, json, github) [future]
  • LOG_COLOR: Force enable/disable colors (true, false, auto)
  • LOG_LEVEL_<DEVTOOL>: Per-devtool level overrides (e.g., LOG_LEVEL_GOLANG=debug)
• Behavior: Environment variables should be read once at initialization, not on every log call

7. Output Format Support

• Current: Text format via slog.TextHandler
• Future Requirements:
  • JSON Format: Machine-parseable structured logs
  • GitHub Format: GitHub Actions workflow commands
  • Custom Formats: Pluggable handler interface
• Format Selection: Via LOG_FORMAT env var or WithFormat() option

Technical Constraints

1. Use Standard Library slog

• Rationale: Go 1.21+ standard library, no external dependencies
• Approach: Wrap slog rather than replace it
• Benefits: Familiar API, battle-tested, forward-compatible

2. Zero Breaking Changes

• Requirement: Logger API must be stable from day one
• Strategy: Use options pattern for all configuration
• Versioning: Consider this v1.0 API - no breaking changes allowed

3. Performance Considerations

• Requirement: Minimal overhead, especially when logs are disabled
• Constraints:
  • Level checks should be fast (slog handles this)
  • Color detection done once at initialization
  • No string formatting when log level disabled
  • File I/O should not block logging calls

Integration Points

1. Global vs Local Loggers

• Global Default: logger.Default() for simple cases
• Package Loggers: logger.Default().With("package", "golang") for package-level
• Operation Loggers: logger.New(opts...) for specific operations
• Context Loggers: Pass loggers through context when needed

File Structure

internal/logger/
├── logger.go       
├── options.go     
├── colors.go     
├── github.go      
├── formats.go     
└── logger_test.go  

Open Questions & Trade-offs

1. File Lifecycle Management

Question: How should artifact files be closed?

• Option A: Keep file handle open for duration of program (simpler)
• Option B: Add Close() method, require manual cleanup (cleaner)
• Option C: Use sync.Pool for file handles (complex)
• Recommendation: Start with Option A, add Option B if needed

2. Log Level Granularity

Question: Should we support more granular levels than debug/info/warn/error?

• Option A: Stick with standard slog levels (simpler)
• Option B: Add custom levels like TRACE, FATAL (more flexible)
• Recommendation: Start with Option A, slog's four levels are sufficient

3. Configuration File Support

Question: Should we support a configuration file (e.g., .mage/logging.yaml)?

• Pros: Centralized configuration, version controlled
• Cons: Additional complexity, file parsing overhead
• Recommendation: Environment variables

4. Performance vs. Features

Note: Synchronous logging features might block or introduce small perf impact.
I don't think this is a big deal now. Just putting it down as a note.

5. GitHub Actions Auto-Detection

Question: Should GitHub format be auto-enabled when GITHUB_ACTIONS=true?

• Option A: Auto-enable (convenience, less configuration)
• Option B: Require explicit opt-in (explicit, user control)
• Recommendation: Auto-enable with ability to override

[AtzeDeVries]

This would be great to have.

I guess we can take two approaches.

1. Feed some structured format from each devtool into the logger
2. Every devtool is responsible for it own logging.

For approach 1:

I think requirement 1 and 3 require a wrapper around the output for every devtool because the requirements require

for req 1:

• Loglevel

for req 3:

• Loglevel
• filename
• linenumber/column.

Did i miss anything requirement?

Once we have set this we should try to get the logs structured from the devtool (with req 5 in mind).

For approach 2:

This might give a bit more flexibility, you need to commit to req 1,3 and 5 but you dont necessarily need to implement a structured output to feed into the centralized logger.