6.5
Monitoring
Why this exists
Everything so far has been reactive. Something breaks, you find out, you fix it.
The job you're actually being hired for is the other one: finding out before the person who depends on it does. That's the difference between a tech assistant and someone people trust with a system — not how fast you fix things, but whether the news comes from you or from them.
The idea
Monitoring is one question, asked on a timer, that tells you when the answer changes.
Not dashboards. Not graphs. A dashboard only helps if someone's looking at it, and at 7am nobody is.
The shape of a useful check:
| Part | Example |
|---|---|
| A question with a right answer | "Did yesterday's sync run?" |
| A schedule | Every morning at 6 |
| A threshold | Zero rows is wrong. Under 50 is suspicious. |
| Somewhere it shouts | Slack, email — somewhere you actually look |
That third part is what separates a check from noise. "How many bookings yesterday" is a number. "Fewer than 20 bookings yesterday" is an alarm. A check that reports normal every day gets ignored within a week — you want one that's silent when things are fine and loud when they aren't.
Start with the thing that would embarrass you most. Not the most likely failure, the most expensive one. If a payment sync silently stops, how long until someone notices? If the answer is "days", that's your first check, regardless of how rarely it happens.
Three questions that generate good checks:
- What breaks quietly? (Loud failures announce themselves. Quiet ones don't.)
- What would I be embarrassed to hear about from someone else?
- What did I check manually this week? (Anything you checked twice is a check.)
That third one is the honest source of most monitoring. You already check things — you just do it by remembering, which works right up until the week you're busy.
One good check beats ten mediocre ones. Ten checks means ten alerts, most of them fine, and within a fortnight you're ignoring all of them. That's worse than having none, because now you believe you're covered.
Do
- Write down three things you checked manually this week. Anything you checked twice is a candidate.
- For each, ask: if this broke quietly, how long until someone noticed? Pick the one with the longest answer.
- Turn it into a question with a right answer. Not "how many" — "fewer than N" or "hasn't run since".
- Ask Claude to build it as a scheduled check. Be explicit that it must be read-only — it's answering a question, not fixing anything.
- Make it alert somewhere you actually look. Slack, email. Not a log file.
- Make it fire on purpose. Change the threshold so it triggers, confirm the alert arrives, then set it back.
- Set the schedule and let it run.
- Write
monitoring.md: what it checks · what "wrong" means · where it shouts · what you'd do when it fires.
Step 6 is the one people skip. An untested alert is not an alert — it's a belief. The day it matters is a terrible day to find out the webhook was wrong.
Done when
Never do
Never build a check that reports normal. Daily "everything's fine" messages get filtered within a week, and then the one that isn't fine gets filtered too.
Never ship an alert you haven't triggered on purpose. Untested alerting is the most common failure in monitoring — the check works, the notification doesn't, and nobody finds out until the morning it mattered.
Never let a monitoring check write anything. It's answering a question. The moment it can also fix things, an unattended process is changing your system at 6am with nobody watching.