Alerting basics

Topic: Monitoring basics

Define alerts on metrics or log patterns; route to on-call or ticketing. Use clear thresholds and runbooks. Use when you need to be notified of failures or anomalies.

Intent: How-to

Quick answer

  • Alert when metric exceeds threshold or is missing. Example: CPU over 90 percent for 5m, or no heartbeat for 2m.
  • Route alerts to people or channels. Avoid alert fatigue; tune thresholds and group similar alerts.
  • Attach runbook or link. Document what to do when alert fires. Review and tune regularly.

Prerequisites

Steps

  1. Define condition

    Choose metric and threshold. Add duration to reduce noise. Add missing-data alert for critical checks.

  2. Route and notify

    Send to PagerDuty, Slack, or email. Escalate if not acknowledged. Group by service or team.

  3. Runbook and tune

    Link runbook or doc. Review false positives; adjust threshold or duration. Regular alert review.

Summary

Define alerts on metrics with thresholds; route to on-call; attach runbooks; tune to reduce fatigue.

Prerequisites

Steps

Step 1: Define condition

Metric, threshold, duration; missing-data for critical.

Step 2: Route and notify

Route to PagerDuty, Slack, or email; set escalation.

Step 3: Runbook and tune

Link runbook; review and tune thresholds.

Verification

  • Test alert fires and is received; runbook is available.

Troubleshooting

Too many alerts — Raise threshold or duration; group. No alert — Check routing and condition.

Next steps

Continue to