fw: a dead firewall leaves the network down, and can be restarted into it

Two things can happen when a firewall dies: the traffic it was
filtering carries on unfiltered, or it stops.  Only the second is
defensible.  A machine briefly off the network is a machine somebody
notices and fixes; a machine briefly on the network with no rules is
the thing the firewall was installed to prevent, and nobody notices it
at all.

fw already does the second, in all three modes, and by mechanism rather
than by care.  Measured rather than assumed:

	before fw   device: /net/ether1  addr: 10.9.9.1  route: 10.9.9.254
	fw running  device: pkt0         addr: 10.9.9.1  route: 10.9.9.254
	fw killed   device:              addr:           route:

pktmedium is unbindonclose, so the interface and the address go when
fw's fds close, and the card is left bound to nothing with nothing
reading it.  In a namespace it is harder still: /net answers "i/o on
hungup channel" and bind -a '#I' /net answers "mount/attach disallowed",
because the device mask was dropped before the program started.

So todo item 1 -- "a dead fw takes the network with it", open since the
first commit -- was the requirement written down as a defect.  It is
now design.md and a FAILURE section in fw(8), and the tests assert it,
which is the point: this is exactly the property a later helpful change
reverses without meaning to.  putback() was that change, written and
never run; deleting it removed a fail-open path, not just dead code.

What was actually broken is recovery, and in a way nobody had reached:
a fw that dies leaves its control filesystem mounted, and a corpse of a
mount fails everything asked of it -- including the access() check fw
makes before touching a card, which then refuses the restart:

	fw: /tmp/rdbg/ctl: clone failed; not touching /net/ether1 until it exists

That check exists so fw does not take a card it cannot then serve, and
it was keeping fw from ever coming back.  Now the dead mount is cleared
first, the same way reclaim() clears the pkt interface the same dead fw
left behind: both are its own wreckage.

With that and -a/-g in the service file -- so a restart does not need
the address it just lost -- the whole cycle works and never passes
through open: crash, network down, restart, network up and filtered.
Ten checks, three of which fail against the previous fw.c.  87 pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

1 files changed, 29 insertions, 24 deletions

diff --git a/fw/doc/todo.md b/fw/doc/todo.md
index d101989..6c39e9b 100644
--- a/fw/doc/todo.md
+++ b/fw/doc/todo.md
@@ -4,8 +4,9 @@ Most of this came from code review. Where a finding has been fixed it
 says so; where it has not, it says what is actually true. An item never
 appears in both halves.
 
-`test/fwtest.rc` is a check for every bug that has shipped here — 77 of
-them now. Run it twice in a row after touching anything.
+`test/fwtest.rc` is a check for every bug that has shipped here, and
+for the properties that must not quietly stop being true — 87 of them
+now. Run it twice in a row after touching anything.
 
 ## Fixed, first round
 
@@ -84,7 +85,22 @@ them now. Run it twice in a row after touching anything.
 10. **Promiscuous mode had no filter behind it.** The card must be
     promiscuous for multicast, but every neighbour's unicast was then
     judged, counted and flow-tracked as if it were ours.
-11. **"fw daemonizes, so svc cannot supervise it" was wrong.** `svc`
+11. **"A dead fw takes the network with it" was the requirement, not
+    the bug.** It had been item 1 since the beginning, on the grounds
+    that a dead `fw` leaves the machine with no network. That is the
+    only defensible thing for a firewall to do: the alternative is
+    traffic flowing unfiltered, which is the moment the firewall exists
+    for. All three modes do it, by mechanism rather than by care —
+    `unbindonclose` on the card and the gateway, a hung-up mount plus a
+    dropped `#I` in a namespace — and all three are now measured.
+    `putback`, deleted earlier as dead code, was the fail-open version
+    of this; it never ran, so it was never wrong, but repairing it
+    would have been. What remained was that recovery did not work:
+    `-a` and `-g` are now in the service file so a restart does not
+    need the address it just lost, and `fw` clears the dead mount its
+    own corpse left on the control mountpoint, which was refusing the
+    restart by way of the check meant to protect the card.
+12. **"fw daemonizes, so svc cannot supervise it" was wrong.** `svc`
     has had the detaching shape from the start: `ready=srv:` watches
     the `/srv` name, and for those services the process exiting "is
     normal and never causes a restart". Nothing needed a foreground
@@ -97,29 +113,11 @@ them now. Run it twice in a row after touching anything.
 
 ## Still open
 
-### 1. Nothing puts the card back
-
-Taking a card is destructive and is not undone. The pkt interface is
-`unbindonclose`, so when `fw` stops the interface and the address both
-go, leaving the card bound to nothing and the machine with no network.
-`fw` cannot restart unaided either: the address it would read off the
-card is the address that just vanished.
-
-The cleanup that claimed to handle this was dead code and is gone. The
-answer is not more note handling — it has to be something that outlives
-`fw`, which means the supervisor, with the address stored where `fw`
-does not own it.
-
-This is what makes `restart=always` a half-measure rather than an
-answer: `svc` will start `fw` again, and the new one has no address to
-read off a card that no longer has one. It needs `-a`, or the card
-configured again first. Supervision works; recovery does not.
-
-### 2. One card per fw, and rules cannot name a card
+### 1. One card per fw, and rules cannot name a card
 
 Wants repeatable `-e` and an `ifc=` attribute, together.
 
-### 3. Tflush is not implemented
+### 2. Tflush is not implemented
 
 A request `fw` is blocked on cannot be abandoned. Only affects
 namespace mode, where waiting for an inbound connection is the one
@@ -127,11 +125,18 @@ operation that blocks indefinitely. ~80 lines, with a race that cannot
 be fully closed: between the syscall returning and the handler clearing
 its entry, a note may land on a worker that has moved on.
 
-### 4. Positional `delete n` renumbers
+### 3. Positional `delete n` renumbers
 
 Inherent to positional deletion; iptables has it too. Said out loud
 here rather than fixed.
 
+## Decided, and now written down
+
+**A dead `fw` leaves the network down, not open.** See `design.md` and
+FAILURE in `fw(8)`. The tests assert it in card and namespace mode,
+because it is exactly the kind of property a later helpful change would
+reverse without meaning to.
+
 ## Never tested
 
 - **The wire side of card mode.** The suite can take a spare card — it