api: YAML JSON TOON Database
version: 1.0.0
format: toon
dataset:
  id: 1633
  slug: observability-signals
  title: Observability Signals and Alert Design
  description: "Operational signals, useful questions, and failure modes for service monitoring and incident diagnosis."
  summary: "Operational signals, useful questions, and failure modes for service monitoring and incident diagnosis. Each row includes a practical application and a failure to check before using it."
  use_when[2]: Add useful diagnostics to a small API.,Choose alerts tied to a real user-visible failure.
  caveats[2]: Names and conventions vary by instrumentation stack.,Collecting telemetry does not by itself provide an actionable alert.
  related[2]: site-reliability-practices,reliable-background-jobs
  reviewed_at: 2026-10-12
  sources[2]{title,url}:
    OpenTelemetry signals,"https://opentelemetry.io/docs/concepts/signals/"
    Google SRE monitoring guidance,"https://sre.google/sre-book/monitoring-distributed-systems/"
  reading[1]: logs-metrics-and-traces-for-a-small-api
  provenance: "Original editorial reference notes by YJTOON, informed by the linked primary documentation. External specifications retain their own licenses."
  entry_count: 10
  fields[4]{section,field,type,example}:
    data,signal,string,Metric
    data,question,string,How much or how often?
    data,example,string,Request count per service and status class.
    data,pitfall,string,Unbounded labels create excessive cardinality.
  category: DevOps
  category_slug: devops
  tags: "devops,observability signals,reference,engineering"
  view_count: 2
  created_at: 1791798640
  updated_at: 1791798640
data[10]{signal,question,example,pitfall}:
  Metric,How much or how often?,Request count per service and status class.,Unbounded labels create excessive cardinality.
  Log,What happened at this event?,A structured failure record with an operation ID.,Logging credentials or full private payloads.
  Trace,Where did a request spend time?,Spans covering a request and its database query.,Missing propagation breaks the end-to-end path.
  Latency histogram,What does the duration distribution look like?,Bucket counts for request duration.,A mean hides slow-tail behavior.
  Error ratio,What fraction of eligible requests failed?,Failed requests divided by all eligible requests.,A raw error count changes with traffic.
  Saturation,How close is a resource to its capacity?,Queue depth relative to workers.,A high utilization number alone may not indicate user harm.
  SLI,What measured behavior represents service quality?,Fraction of requests completed within the latency target.,The denominator must match user-relevant work.
  SLO,What reliability target do we intend to meet?,A target success fraction over a defined window.,A target without a window is underspecified.
  Burn rate,How quickly is the error budget being consumed?,Observed error fraction divided by the allowed fraction.,Short windows can be noisy at low volume.
  Correlation ID,Which records describe one operation?,A stable request or job identifier across signals.,It is not a user identity and should not contain secrets.