From dee31d66b705734be68c1bb21ff5dfc7bdaae23b Mon Sep 17 00:00:00 2001 From: Calvin Morrison Date: Wed, 19 Aug 2026 10:00:34 -0400 Subject: fw: make the /srv name worth watching, and say how to supervise it todo.md said fw daemonizing meant svc could not supervise it, and that it needed a foreground mode. Both wrong. svc has had the shape from the start. ready=srv: watches the /srv name rather than the pid, and its own manual says why: "Use this for a service that posts to /srv and lets the process you started exit, which many Plan 9 file servers do on purpose. For these the /srv file is watched and the process is not, so the process exiting is normal and never causes a restart." init.c agrees -- reap() returns early for Ksrv with that comment on it. Nor is detaching unusual. 42 commands under /sys/src/cmd use postmountsrv or threadpostmountsrv; five call srv() directly, and every one of those is a stdio server (ramfs -i, ext4srv -s, skelfs, hjfs, wacom) speaking 9P on file descriptors it was handed. That is not a foreground service, it is a pipe server: no /srv, no mount, nothing to supervise. A foreground mode for fw would buy a pid to watch, and the /srv name is the better signal -- it survives the process that made it. What was true underneath the wrong diagnosis: the name was not honest. Taking the control filesystem away ends the server proc, and the relays carried on filtering: procs: 3 ... take the ctl filesystem away ... procs after: 2 still filtering? pkt interface: pkt0 A firewall nobody can reach, stop, or notice, and the /srv name gone while it runs. Srv.end now takes the whole thing down, which is the answer the relays already gave when their wire failed. Same test after: three procs become none and the interface goes with them. So: no flag, an example service file in lib/svc, and a SUPERVISION section in fw(8) that says what init should watch and what restarting will and will not fix. Restart=always brings fw back; it does not undo taking the card, so the new fw has no address to read. Supervision works, recovery does not, and that stays open as item 1. Two checks. Against the previous fw.c they fail with two orphaned procs and a pkt interface still bound -- and so do the leak checks at the end of the run, which is what they were built for. 77 pass. Co-Authored-By: Claude Opus 5 --- fw/doc/design.md | 9 +++++++ fw/doc/todo.md | 29 +++++++++++++++-------- fw/lib/svc/fw.ether0 | 27 +++++++++++++++++++++ fw/man/fw | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++++ fw/src/fw.c | 19 +++++++++++++++ fw/test/fwtest.rc | 34 ++++++++++++++++++++++++++- 6 files changed, 173 insertions(+), 11 deletions(-) create mode 100644 fw/lib/svc/fw.ether0 diff --git a/fw/doc/design.md b/fw/doc/design.md index c14787c..1060ba6 100644 --- a/fw/doc/design.md +++ b/fw/doc/design.md @@ -143,6 +143,15 @@ forbid are dropped. pf and iptables leave them running until they time out; that is a wart everyone has to learn, and "I blocked it, why is the transfer still going" is the wrong thing to discover at 3am. +**A file server, so it detaches.** `fw` posts to `/srv`, forks the +server and lets the process you started exit — what 42 of the 47 file +servers in `/sys/src/cmd` do, and the five that do not are stdio +servers speaking 9P on their own standard input, which is a different +thing altogether. So a supervisor watches the `/srv` name and not the +pid, which `svc` already provides for. The name is only worth watching +if it is honest, which is why anything that stops one part of `fw` now +stops all of it. + **Per-rule logging to `/sys/log/fw`.** Global logging either floods a disk or tells you nothing. `syslog(2)` does not create its file, so `fw` says so at startup rather than dropping the lines silently. diff --git a/fw/doc/todo.md b/fw/doc/todo.md index de5eb8e..d101989 100644 --- a/fw/doc/todo.md +++ b/fw/doc/todo.md @@ -4,7 +4,7 @@ Most of this came from code review. Where a finding has been fixed it says so; where it has not, it says what is actually true. An item never appears in both halves. -`test/fwtest.rc` is a check for every bug that has shipped here — 72 of +`test/fwtest.rc` is a check for every bug that has shipped here — 77 of them now. Run it twice in a row after touching anything. ## Fixed, first round @@ -84,6 +84,16 @@ them now. Run it twice in a row after touching anything. 10. **Promiscuous mode had no filter behind it.** The card must be promiscuous for multicast, but every neighbour's unicast was then judged, counted and flow-tracked as if it were ours. +11. **"fw daemonizes, so svc cannot supervise it" was wrong.** `svc` + has had the detaching shape from the start: `ready=srv:` watches + the `/srv` name, and for those services the process exiting "is + normal and never causes a restart". Nothing needed a foreground + mode. What was true underneath it: the control filesystem going + away ended the server proc and left the relays filtering, so the + `/srv` name could be gone while the firewall was still running. + Now any part stopping stops all of them, which is what makes the + name worth watching. `fwstart` is still the wrong shape; a service + file per card is in `lib/svc`. ## Still open @@ -98,19 +108,18 @@ card is the address that just vanished. The cleanup that claimed to handle this was dead code and is gone. The answer is not more note handling — it has to be something that outlives `fw`, which means the supervisor, with the address stored where `fw` -does not own it. This also blocks `svc` supervision. +does not own it. -### 2. fw daemonizes, so svc cannot supervise it +This is what makes `restart=always` a half-measure rather than an +answer: `svc` will start `fw` again, and the new one has no address to +read off a card that no longer has one. It needs `-a`, or the card +configured again first. Supervision works; recovery does not. -Needs a foreground mode; `ready=srv:name` fits, since `-s` already -posts to `/srv`. `fwstart` is the wrong shape and should probably go — -one service per card is the right one. - -### 3. One card per fw, and rules cannot name a card +### 2. One card per fw, and rules cannot name a card Wants repeatable `-e` and an `ifc=` attribute, together. -### 4. Tflush is not implemented +### 3. Tflush is not implemented A request `fw` is blocked on cannot be abandoned. Only affects namespace mode, where waiting for an inbound connection is the one @@ -118,7 +127,7 @@ operation that blocks indefinitely. ~80 lines, with a race that cannot be fully closed: between the syscall returning and the handler clearing its entry, a note may land on a worker that has moved on. -### 5. Positional `delete n` renumbers +### 4. Positional `delete n` renumbers Inherent to positional deletion; iptables has it too. Said out loud here rather than fixed. diff --git a/fw/lib/svc/fw.ether0 b/fw/lib/svc/fw.ether0 new file mode 100644 index 0000000..93de412 --- /dev/null +++ b/fw/lib/svc/fw.ether0 @@ -0,0 +1,27 @@ +# +# One firewall, one card. Copy this per card that should be filtered: +# the name, the srv name and the -e argument change together. +# +# fw daemonizes, as Plan 9 file servers do, so what init watches is the +# /srv name and not the pid. The name is there exactly while the +# firewall is filtering: it appears when the server posts it and goes +# when the last of fw exits, whether that is the wire failing, the +# control filesystem going away, or a note. +# +# It has to start after the card has an address, because that is where +# fw reads the address, mask and gateway from - and before anything +# dials, because a connection made before fw is up was never filtered. +# +svc=fw.ether0 + exec=/bin/fw + args=-s + args=fw.ether0 + args=-m + args=/mnt/fw/ether0 + args=-e + args=/net/ether0 + args=/lib/fw/host.ndb + needs=ipconfig + ready=srv:fw.ether0 + restart=always + enable=no diff --git a/fw/man/fw b/fw/man/fw index 99ec41f..9279c60 100644 --- a/fw/man/fw +++ b/fw/man/fw @@ -348,6 +348,67 @@ everything else instead. .PP .B /lib/fw/fw.rc does this in the right order: private namespace, mount, drop, exec. +.SH SUPERVISION +.I Fw +posts its filesystem on +.BI /srv/ name +with +.BR -s , +forks the server, and the process you started exits - the ordinary +Plan 9 file server shape. +A supervisor must therefore watch the +.B /srv +name rather than the pid. +Under +.IR init (8) +that is +.BR ready=srv: : +.IP +.EX +svc=fw.ether0 + exec=/bin/fw + args=-s + args=fw.ether0 + args=-m + args=/mnt/fw/ether0 + args=-e + args=/net/ether0 + args=/lib/fw/host.ndb + needs=ipconfig + ready=srv:fw.ether0 + restart=always + enable=yes +.EE +.PP +The name means what it says: it appears when the server posts it and +goes when the last of +.I fw +exits. +Anything that stops one part stops all of them - a wire that fails, a +control filesystem that is unmounted, a note - so a +.B /srv +name that is present is a firewall that is filtering. +.PP +There is no foreground mode and none is needed. +What such a mode would buy is a pid to watch, and the +.B /srv +name is the better signal: it survives the process that made it and +answers the question that matters. +.PP +Restarting is not free. +.B Restart=always +will bring +.I fw +back, but taking a card is destructive and is not undone, so the card +must be configured again before the new +.I fw +can read an address off it \- see +.B BUGS +and +.IR ipconfig (8). +A copy of this file per card is the shape to use; one +.I fw +per card is the only shape there is. .SH BOOT .B /lib/fw/fwstart starts a firewall for each card named in @@ -442,6 +503,10 @@ starts one firewall per card at boot .B /lib/ndb/fw which cards are filtered, and with what .TP +.B /lib/svc/fw.ether0 +an example service file for +.IR init (8) +.TP .B /mnt/fw control files .TP @@ -459,6 +524,7 @@ are recorded .IR dial (2), .IR syslog (2), .IR fork (2), +.IR init (8), .IR ipconfig (8) .SH BUGS There is no address translation. diff --git a/fw/src/fw.c b/fw/src/fw.c index 6ae38df..bb97960 100644 --- a/fw/src/fw.c +++ b/fw/src/fw.c @@ -1022,11 +1022,30 @@ Done: respond(r, nil); } +/* + * The control filesystem going away takes the firewall with it. + * + * Without this the server proc ends - its mount gone, its /srv name + * removed - and the relays carry on filtering with nothing left to + * talk to them: no ctl, no rules, no flows, and no way for a + * supervisor to see that anything happened. Measured: three procs + * become two and the traffic keeps flowing. A firewall nobody can + * reach or stop is not a state worth staying in, and it is the same + * answer the relays already give when their wire fails. + */ +static void +endsrv(Srv*) +{ + syslog(0, "fw", "control filesystem gone; stopping"); + threadexitsall("ctl"); +} + static Srv fs = { .read= fsread, .write= fswrite, .destroyfid= fsdestroyfid, +.end= endsrv, }; /* a rule change must not leave traffic running that the rules now forbid */ diff --git a/fw/test/fwtest.rc b/fw/test/fwtest.rc index 0325bfd..3f80242 100755 --- a/fw/test/fwtest.rc +++ b/fw/test/fwtest.rc @@ -46,7 +46,7 @@ mkdir -p $mtpt # The stacks the packet checks build for themselves. Fixed numbers are # fine because they are unbound again at the end of each block, and the # last check proves it. -stacks=(20 21 22 23 24 25 26) +stacks=(20 21 22 23 24 25 26 27 28) # A port of our own. Conversations outlive the run that made them - a # devip Conv is never freed - so a fixed port makes one run's leftovers @@ -551,6 +551,38 @@ echo '== a card' } } +echo '== supervision' +# fw daemonizes, like every other Plan 9 file server: the process you +# exec posts to /srv, forks the server and exits. A supervisor watches +# the /srv name instead of the pid, so the name has to mean what it +# says - present exactly while the firewall is filtering. +@{ + rfork n + mkdir -p $tmp/nD $tmp/nE $tmp/sctl + bind -a '#I27' $tmp/nD + bind -a '#I28' $tmp/nE + $fw -s fw.test.$pid -m $tmp/sctl $tmp/empty.ndb \ + $tmp/nD^'!'^10.9.9.1^'!'^/24 $tmp/nE^'!'^10.9.9.2^'!'^/24 >[2]/dev/null + check 'the process that starts it exits, as a file server does' '' $"status + sleep 2 + + r=`{if(test -e /srv/fw.test.^$pid) echo yes; if not echo no} + check 'and leaves its name in /srv' yes $"r + r=`{ls $tmp/sctl | wc -l} + check 'with the control files mounted' 4 $"r + + # what a supervisor would restart on. Taking the control + # filesystem away used to end the server proc and leave the relays + # filtering: a firewall nobody could reach, stop, or notice. + unmount $tmp/sctl + rm /srv/fw.test.$pid + sleep 3 + r=`{ps | awk '$NF == "fw"' | wc -l} + check 'losing the control filesystem stops the whole firewall' 0 $"r + r=`{sed 1q $tmp/nD/ipifc/0/status | awk '{print $2}'} + check 'and the interface it made goes with it' maxtu $"r +} + echo '== fwstart' if(test -f $lib/fwstart) @{ -- cgit v1.2.3