k8s-node-not-ready-drain

Drain a node stuck NotReady and reschedule its pods, after establishing whether the kubelet is gone or merely unreachable.

Procedure

  1. start start. Goes next to s1
  2. check Confirm the node is NotReady. Goes on_fail to s9; next to s2
  3. decision Is the kubelet reachable?. Goes if unreachable to s3; if responding to s6; if default to s9
  4. action Cordon the node, RW (reversible-write). Goes next to s4
  5. wait Wait for the node to recover. Goes retry to s6; next to __end_success
  6. escalate Escalate to the cluster owner. Terminal
  7. action Drain the node, DEL (destructive), requires approval. Goes rollback to s5; next to __end_success
  8. action Uncordon the node, RW (reversible-write). Goes next to __end_aborted
  9. end success. Terminal
  10. end aborted. Terminal

Open the graph on its own page — larger, printable, and an address you can send.

Walk it without running it — you answer what your environment would answer, and the path, the capabilities, the approvals and the distance back to an undo come out of the document. Nothing executes; there is no Run button here and there will not be one.

Trust

T2 · attested 2026-10-11 · runbooks.directory

Nobody has attested this record. That is the ordinary case: T2 is what the catalog can check on its own, and T3 needs somebody to run the procedure and sign for it.

  • Link liveness has not been checked: it needs a network and the catalog builds without one. Run pnpm gates --online where there is one.

Steps

  1. s1 check — Confirm the node is NotReady
  2. s2 decision — Is the kubelet reachable?
  3. s3 action — Cordon the node (reversible-write)
  4. s4 action — Drain the node (destructive) — requires approval
  5. s5 action — Uncordon the node (reversible-write)
  6. s6 wait — Wait for the node to recover
  7. s9 escalate — Escalate to the cluster owner

Capabilities

This is a privilege request. Check it against your own allowlist before running.

  • cli:kubectl — used by s1, s3, s4, s5

Provenance

Source
https://runbooks.directory/r/std/k8s-node-not-ready-drain/source.md
Fetched
2026-09-03
Source last checked
not since it was captured on 2026-09-03 — recorded as current
Licence
CC-BY-4.0
How it got here
Written by this catalog's maintainers and reviewed in full.
Highest risk in this procedure
destructive

Take it with you

Run it under supervision

This procedure contains destructive steps. Run it here, where the supervisor is the thing invoking the commands and therefore sees every one of them: each step is proposed before it happens and a gate stops for you. Nothing about the run leaves this machine.

# destructive steps: every one stops for you before it runs.
npx @runbooks/cli run std/k8s-node-not-ready-drain@1.0.0 --as "$(whoami)" \
  --input node=<node name>