HOWTO: Interpret Structured Probe Output¶
Purpose: Read and evaluate structured probe output from HybridOps verification steps to confirm environment health or identify degradation.
Difficulty: Intermediate
Track: Verification-Driven Operations
Overview¶
Probes are the verification primitives that HybridOps uses to confirm environment state. Unlike raw CLI output, probe results are typed, structured, and threshold-bound: making them machine-readable and audit-suitable. This HOWTO teaches you to navigate probe output as a first-class records, not just as a pass/fail indicator.
1. Where Probe Output Lives¶
- Probe records paths under
<runtime-root>/logs/. - Probe output in the run record run-record output.
- Locating probes from a specific step.
2. Probe Output Schema¶
- Signal types: latency, reachability, count, diff, boolean.
- Value, unit, threshold, and evaluation fields.
- Timing: collected_at, duration_ms.
3. Evaluating Threshold Results¶
- Pass, warn, and fail thresholds.
- Absolute vs relative thresholds.
- Reading the evaluation message.
4. Correlating Across Steps¶
- Before/after probe pairs and how to compare them.
- Cross-step correlation using the shared run ID.
- Multi-target probe aggregation.
5. Distinguishing Real Failures from Transient Noise¶
- Retry behaviour and its effect on probe output.
- Single-sample vs windowed probes.
- When to escalate vs re-probe.
References¶
- ADR-0401 – Unified Observability with Prometheus
- HOWTO: Read a Run Record
- HOWTO: Package a Run Record
License: MIT-0 for code, CC-BY-4.0 for documentation unless otherwise stated.