Observability Signals and Alert Design
Operational signals, useful questions, and failure modes for service monitoring and incident diagnosis.
Operational signals, useful questions, and failure modes for service monitoring and incident diagnosis. Each row includes a practical application and a failure to check before using it.
Use it when
- Add useful diagnostics to a small API.
- Choose alerts tied to a real user-visible failure.
Watch out for
- Names and conventions vary by instrumentation stack.
- Collecting telemetry does not by itself provide an actionable alert.
The data
10 rows
| Signal | Question | Example | Pitfall |
|---|---|---|---|
| Metric | How much or how often? | Request count per service and status class. | Unbounded labels create excessive cardinality. |
| Log | What happened at this event? | A structured failure record with an operation ID. | Logging credentials or full private payloads. |
| Trace | Where did a request spend time? | Spans covering a request and its database query. | Missing propagation breaks the end-to-end path. |
| Latency histogram | What does the duration distribution look like? | Bucket counts for request duration. | A mean hides slow-tail behavior. |
| Error ratio | What fraction of eligible requests failed? | Failed requests divided by all eligible requests. | A raw error count changes with traffic. |
| Saturation | How close is a resource to its capacity? | Queue depth relative to workers. | A high utilization number alone may not indicate user harm. |
| SLI | What measured behavior represents service quality? | Fraction of requests completed within the latency target. | The denominator must match user-relevant work. |
| SLO | What reliability target do we intend to meet? | A target success fraction over a defined window. | A target without a window is underspecified. |
| Burn rate | How quickly is the error budget being consumed? | Observed error fraction divided by the allowed fraction. | Short windows can be noisy at low volume. |
| Correlation ID | Which records describe one operation? | A stable request or job identifier across signals. | It is not a user identity and should not contain secrets. |
Fields
The shape of this dataset, read from the data itself: every column, its type and a real value. Agents can use this to plan a query without fetching the whole document.
| Section | Field | Type | Example |
|---|---|---|---|
| data | signal | string | Metric |
| data | question | string | How much or how often? |
| data | example | string | Request count per service and status class. |
| data | pitfall | string | Unbounded labels create excessive cardinality. |
Fetch the same bytes
The static files contain the same reference rows as the API, with no API rate limit. Live metadata, such as view counts, may differ. Use the API for content negotiation; use the static files for a cacheable snapshot.
curl "https://yjtoon.com/api/dataset/observability-signals?format=toon"
const res = await fetch( "https://yjtoon.com/static-data/dataset/observability-signals.toon" ); const toon = await res.text();
Rate limit: 120 requests per minute per IP, no key and no signup. API reference →
Sources and editorial notes
Original editorial reference notes by YJTOON, informed by the linked primary documentation. External specifications retain their own licenses.
Editorial review: . Source documents keep their own licenses.