api: YAML JSON TOON Database
version: 1.0.0
format: toon
dataset:
  id: 1653
  slug: reliable-background-jobs
  title: Reliable Background Job Patterns
  description: "Queue and worker design patterns for duplicate deliveries, failure recovery, concurrency, and bounded work."
  summary: "Queue and worker design patterns for duplicate deliveries, failure recovery, concurrency, and bounded work. Each row includes a practical application and a failure to check before using it."
  use_when[2]: Design a worker that can safely retry after a crash.,Separate delivery guarantees from business effects.
  caveats[2]: Exactly-once delivery claims do not establish exactly-once external effects.,Patterns must be implemented with the database and queue’s real atomicity guarantees.
  related[2]: observability-signals,concurrency-design-patterns
  reviewed_at: 2026-10-12
  sources[2]{title,url}:
    AWS SQS delivery semantics,"https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/standard-queues-at-least-once-delivery.html"
    AWS transactional outbox pattern,"https://docs.aws.amazon.com/prescriptive-guidance/latest/cloud-design-patterns/transactional-outbox.html"
  reading[1]: background-jobs-idempotency-and-the-outbox-pattern
  provenance: "Original editorial reference notes by YJTOON, informed by the linked primary documentation. External specifications retain their own licenses."
  entry_count: 10
  fields[4]{section,field,type,example}:
    data,pattern,string,Idempotent handler
    data,mechanism,string,Make repeated application produce the intended …
    data,use,string,Handle at-least-once message delivery.
    data,failure_to_test,string,Deliver the same job twice.
  category: Design Patterns
  category_slug: design-patterns
  tags: "design-patterns,reliable background jobs,reference,engineering"
  view_count: 10
  created_at: 1791798640
  updated_at: 1791798640
data[10]{pattern,mechanism,use,failure_to_test}:
  Idempotent handler,Make repeated application produce the intended single result.,Handle at-least-once message delivery.,Deliver the same job twice.
  Deduplication key,Persist a stable key with the committed effect.,Prevent repeating one business operation.,Crash between recording the key and committing the effect.
  Transactional outbox,Commit the domain change and outgoing record together.,Keep a database write aligned with event publication.,Crash after commit but before publication.
  Visibility timeout,Lease a job for a bounded processing interval.,Recover work from a failed worker.,Run a job longer than the lease.
  Lease heartbeat,Extend a lease while an owned job is progressing.,Support long-running bounded tasks.,Lose connectivity during an extension.
  Exponential backoff,Increase delay between repeated failures with jitter.,Reduce pressure on a failing dependency.,Make many workers fail simultaneously.
  Dead-letter queue,Move repeatedly failing jobs aside with evidence.,Keep a poison job from blocking useful work.,Replay a corrected job without duplicating effects.
  Concurrency limit,Bound active jobs per resource or tenant.,Protect downstream capacity.,Enqueue a burst above steady-state throughput.
  Cancellation,Check a cooperative cancellation signal at safe points.,Stop obsolete or user-cancelled work.,Cancel while an external side effect is in flight.
  Checkpoint,Persist completed progress under a versioned contract.,Resume a long task after a crash.,Resume with a changed code or data version.