Skip to content

Implement audit archiver service#19

Merged
maxmalkin merged 12 commits intomainfrom
implement-audit-archiver
Mar 2, 2026
Merged

Implement audit archiver service#19
maxmalkin merged 12 commits intomainfrom
implement-audit-archiver

Conversation

@maxmalkin
Copy link
Owner

@maxmalkin maxmalkin commented Mar 2, 2026

Summary

  • Implement the audit archiver one-shot pipeline for audit log partition lifecycle management
  • Create future partitions (7 days in advance), archive expired partitions to Parquet in S3-compatible cold storage, and drop archived partitions from PostgreSQL
  • Add S3 and local filesystem cold storage backends with Hive-style key format
  • Add PostgreSQL advisory lock leader election for single-instance guarantee
  • Add Arrow schema (18 fields) and ZSTD-compressed Parquet writer
  • Add MinIO to docker-compose for local S3-compatible development
  • 18 unit tests covering partition logic, Parquet roundtrip, storage keys, config defaults, and local FS operations

@maxmalkin maxmalkin merged commit f8fb2b9 into main Mar 2, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant