Policies

Atomic, portable rules for processing telemetry. From the creator of Vector.

The problem

Every telemetry tool has the same configuration problem. Datadog Agent, OTel Collector, Vector, Fluent Bit—they all work the same way: you write a config that defines processing rules, and data flows through sequentially.

This breaks down:

• Configs grow into thousands of lines that nobody fully understands
• You need to understand the whole config to safely change any part
• Performance degrades as you add rules—a hundred regex patterns might work, a thousand won't
• AI can't help because the config is too interconnected

We saw this happen across thousands of Vector deployments. Configs that started clean became unmaintainable. Everyone hits the same wall.

A different model

id: drop-checkout-debug-logs
name: Drop checkout debug logs
log:
  match:
    - resource_attribute: ["service.name"]
      exact: checkout-api
    - log_field: LOG_FIELD_SEVERITY_TEXT
      exact: DEBUG
  keep: none

A policy does one thing. You read it and know exactly what it does. No context needed.

Attributes use path arrays to support nested access:

# Flat attribute
log_attribute: ["user_id"]

# Nested attribute (e.g., http.request.method)
log_attribute: ["http", "request", "method"]

How it works

┌─────────────────────────────────────────────────────────────────┐
│                          Traditional                            │
│                                                                 │
│   Log ──▶ Component A ──▶ Component B ──▶ Component C ──▶ Out   │
│               │                │               │                │
│           bottleneck      bottleneck      bottleneck            │
└─────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────┐
│                           Policies                              │
│                                                                 │
│                        ┌─▶ Policy 1 ─┐                          │
│                        ├─▶ Policy 2 ─┤                          │
│        Log ──▶ Match ──┼─▶ Policy 3 ─┼──▶ Merge ──▶ Out         │
│                        ├─▶ Policy 4 ─┤                          │
│                        └─▶ Policy N ─┘                          │
│                                                                 │
│                      parallel execution                         │
└─────────────────────────────────────────────────────────────────┘

Telemetry arrives. The runtime matches against all policies in parallel. No sequential bottleneck. Matching policies contribute to fixed stages—keep first, then transform. If any policy drops the telemetry, it's gone. Otherwise transforms apply and it flows out.

Policies are independent by design. Ten thousand policies execute as fast as ten.

What this unlocks

Scale. Tens of thousands of policies without performance degradation. Drop exactly the log patterns you want, not broad categories.

AI generation. Each policy is a bounded problem. No DAG to reason about. AI can generate and manage policies at scale.

Portability. Policies use OpenTelemetry's data model. Same policy works across any runtime that implements the spec.

Simplicity. Add a policy without understanding the others. Remove one without fear. Each one stands alone.

Get started

• Edge — A minimal, lightweight proxy that enforces policies. Deploy it anywhere in your infrastructure—before your tools, after them, as a sidecar. Takes minutes to set up.

• policy-rs — Rust SDK for implementing the policy spec in your own tools.

• policy-zig — Zig SDK for implementing the policy spec in your own tools.

• policy-go — Go SDK for implementing the policy spec in your own tools.

• otel policyprocessor — OpenTelemetry Collector Policy processor for running policies natively in the collector.