diff options
| -rw-r--r-- | fw/doc/design.md | 42 | ||||
| -rw-r--r-- | fw/doc/todo.md | 53 | ||||
| -rw-r--r-- | fw/lib/svc/fw.ether0 | 21 | ||||
| -rw-r--r-- | fw/man/fw | 89 | ||||
| -rw-r--r-- | fw/src/fw.c | 23 | ||||
| -rwxr-xr-x | fw/test/fwtest.rc | 61 |
6 files changed, 229 insertions, 60 deletions
diff --git a/fw/doc/design.md b/fw/doc/design.md index 1060ba6..98eca29 100644 --- a/fw/doc/design.md +++ b/fw/doc/design.md @@ -3,8 +3,11 @@ **A firewall for 9front. One program, filtering at a network card, between two networks, or in front of one namespace.** -Status: working and tested on the init-test VM, in all three modes. Not -yet fit to run on a machine you care about — see `todo.md`. +Status: working on the init-test VM in all three modes, with 87 checks +in `test/fwtest.rc` covering every bug that has shipped. What is left +open is in `todo.md`, and none of it is a reason not to run this any +more; what has never been tested is the wire — a second machine on the +same segment, and IPv6 anywhere. ## Why @@ -143,6 +146,41 @@ forbid are dropped. pf and iptables leave them running until they time out; that is a wart everyone has to learn, and "I blocked it, why is the transfer still going" is the wrong thing to discover at 3am. +**A dead firewall takes the network with it.** There are two things +that can happen when a firewall dies: the traffic it was filtering +flows unfiltered, or it stops. Only the second is defensible. A machine +that is briefly off the network is a machine an operator notices and +fixes; a machine that is briefly on the network with no rules is a +machine nobody notices, and it is exactly the moment you built the +firewall for. + +`fw` does the second, in all three modes, and it is the mechanism +rather than anything careful: + + card: pktmedium is unbindonclose, so the interface and the + address go when fw's fds close. The card is left + bound to nothing and nothing is reading it. + gateway: the same, on both sides at once. + namespace: the mount is hung up, and #I was dropped from the + process group's device mask, so the program cannot + bind the real stack back in its place. + +Measured, not assumed — a card-mode fw killed leaves `device` with +nothing after it, no address in `ipselftab` and no default route; a +sandboxed program gets `i/o on hungup channel` from `/net` and +`mount/attach disallowed` from `bind -a '#I' /net`. + +This is why `putback` was deleted rather than repaired. It meant to +restore the card to the stack as fw exited, which is precisely the +first option: the machine back on the network with no firewall on it. +It never ran, so nothing was ever wrong; had it run, it would have been +wrong every time. + +The cost is that recovery needs an address `fw` no longer knows, and +that is a real cost, paid deliberately. `-a` and `-g` in the service +file buy it back: crash, network down, restart, network up and +filtered, and no moment in between where it is up and unfiltered. + **A file server, so it detaches.** `fw` posts to `/srv`, forks the server and lets the process you started exit — what 42 of the 47 file servers in `/sys/src/cmd` do, and the five that do not are stdio diff --git a/fw/doc/todo.md b/fw/doc/todo.md index d101989..6c39e9b 100644 --- a/fw/doc/todo.md +++ b/fw/doc/todo.md @@ -4,8 +4,9 @@ Most of this came from code review. Where a finding has been fixed it says so; where it has not, it says what is actually true. An item never appears in both halves. -`test/fwtest.rc` is a check for every bug that has shipped here — 77 of -them now. Run it twice in a row after touching anything. +`test/fwtest.rc` is a check for every bug that has shipped here, and +for the properties that must not quietly stop being true — 87 of them +now. Run it twice in a row after touching anything. ## Fixed, first round @@ -84,7 +85,22 @@ them now. Run it twice in a row after touching anything. 10. **Promiscuous mode had no filter behind it.** The card must be promiscuous for multicast, but every neighbour's unicast was then judged, counted and flow-tracked as if it were ours. -11. **"fw daemonizes, so svc cannot supervise it" was wrong.** `svc` +11. **"A dead fw takes the network with it" was the requirement, not + the bug.** It had been item 1 since the beginning, on the grounds + that a dead `fw` leaves the machine with no network. That is the + only defensible thing for a firewall to do: the alternative is + traffic flowing unfiltered, which is the moment the firewall exists + for. All three modes do it, by mechanism rather than by care — + `unbindonclose` on the card and the gateway, a hung-up mount plus a + dropped `#I` in a namespace — and all three are now measured. + `putback`, deleted earlier as dead code, was the fail-open version + of this; it never ran, so it was never wrong, but repairing it + would have been. What remained was that recovery did not work: + `-a` and `-g` are now in the service file so a restart does not + need the address it just lost, and `fw` clears the dead mount its + own corpse left on the control mountpoint, which was refusing the + restart by way of the check meant to protect the card. +12. **"fw daemonizes, so svc cannot supervise it" was wrong.** `svc` has had the detaching shape from the start: `ready=srv:` watches the `/srv` name, and for those services the process exiting "is normal and never causes a restart". Nothing needed a foreground @@ -97,29 +113,11 @@ them now. Run it twice in a row after touching anything. ## Still open -### 1. Nothing puts the card back - -Taking a card is destructive and is not undone. The pkt interface is -`unbindonclose`, so when `fw` stops the interface and the address both -go, leaving the card bound to nothing and the machine with no network. -`fw` cannot restart unaided either: the address it would read off the -card is the address that just vanished. - -The cleanup that claimed to handle this was dead code and is gone. The -answer is not more note handling — it has to be something that outlives -`fw`, which means the supervisor, with the address stored where `fw` -does not own it. - -This is what makes `restart=always` a half-measure rather than an -answer: `svc` will start `fw` again, and the new one has no address to -read off a card that no longer has one. It needs `-a`, or the card -configured again first. Supervision works; recovery does not. - -### 2. One card per fw, and rules cannot name a card +### 1. One card per fw, and rules cannot name a card Wants repeatable `-e` and an `ifc=` attribute, together. -### 3. Tflush is not implemented +### 2. Tflush is not implemented A request `fw` is blocked on cannot be abandoned. Only affects namespace mode, where waiting for an inbound connection is the one @@ -127,11 +125,18 @@ operation that blocks indefinitely. ~80 lines, with a race that cannot be fully closed: between the syscall returning and the handler clearing its entry, a note may land on a worker that has moved on. -### 4. Positional `delete n` renumbers +### 3. Positional `delete n` renumbers Inherent to positional deletion; iptables has it too. Said out loud here rather than fixed. +## Decided, and now written down + +**A dead `fw` leaves the network down, not open.** See `design.md` and +FAILURE in `fw(8)`. The tests assert it in card and namespace mode, +because it is exactly the kind of property a later helpful change would +reverse without meaning to. + ## Never tested - **The wire side of card mode.** The suite can take a spare card — it diff --git a/fw/lib/svc/fw.ether0 b/fw/lib/svc/fw.ether0 index 93de412..627b1d3 100644 --- a/fw/lib/svc/fw.ether0 +++ b/fw/lib/svc/fw.ether0 @@ -8,9 +8,12 @@ # when the last of fw exits, whether that is the wire failing, the # control filesystem going away, or a note. # -# It has to start after the card has an address, because that is where -# fw reads the address, mask and gateway from - and before anything -# dials, because a connection made before fw is up was never filtered. +# It has to start before anything dials: a connection made before fw is +# up was never filtered. +# +# If fw dies the machine has no network until it is restarted. That is +# the intended behaviour and not a fault - see FAILURE in fw(8) - so +# restart=always is what you want, not a supervisor that gives up. # svc=fw.ether0 exec=/bin/fw @@ -20,6 +23,18 @@ svc=fw.ether0 args=/mnt/fw/ether0 args=-e args=/net/ether0 +# +# The address and gateway, rather than letting fw read them off the +# card. A fw that has died has taken the address with it, so a +# restarted fw has nothing to read; told them, it can restart into the +# bare card it left behind. Without them restart=always brings fw back +# only to fail, and the machine stays off the network - which is the +# safe direction, but not a working one. +# + args=-a + args=10.0.2.15/24 + args=-g + args=10.0.2.2 args=/lib/fw/host.ndb needs=ipconfig ready=srv:fw.ether0 @@ -348,6 +348,44 @@ everything else instead. .PP .B /lib/fw/fw.rc does this in the right order: private namespace, mount, drop, exec. +.SH FAILURE +When +.I fw +dies, the traffic it was filtering stops. +It does not carry on unfiltered. +.PP +Filtering a card, the interface that replaced the card is unbound when +.I fw +stops and the address goes with it, so the machine has no network at +all: the card is bound to nothing and nothing is reading it. +Between two networks, the same happens on both sides and nothing +crosses. +Filtering a namespace, the mount is hung up and every path into +.B /net +fails; the program cannot put the real one back either, because it was +denied +.B #I +before it started \- see +.BR CONTAINMENT . +.PP +This is deliberate, and it is the reason +.I fw +does not try to give the card back as it exits. +A machine that is briefly off the network is a machine somebody +notices; a machine that is briefly on the network with no rules is the +thing the firewall was installed to prevent. +.PP +The cost is that +.I fw +cannot restart unaided: the address it would read off the card is the +address that has just gone. +Give +.B -a +and +.B -g +so that it does not have to read them, and it can be restarted into a +machine it has itself left bare \- see +.BR SUPERVISION . .SH SUPERVISION .I Fw posts its filesystem on @@ -373,6 +411,10 @@ svc=fw.ether0 args=/mnt/fw/ether0 args=-e args=/net/ether0 + args=-a + args=10.0.2.15/24 + args=-g + args=10.0.2.2 args=/lib/fw/host.ndb needs=ipconfig ready=srv:fw.ether0 @@ -395,17 +437,20 @@ What such a mode would buy is a pid to watch, and the name is the better signal: it survives the process that made it and answers the question that matters. .PP -Restarting is not free. -.B Restart=always -will bring +The address and gateway are given here rather than read off the card, +which is the one place that is worth doing. +A .I fw -back, but taking a card is destructive and is not undone, so the card -must be configured again before the new +that has died has taken the address with it, so a restarted .I fw -can read an address off it \- see -.B BUGS -and -.IR ipconfig (8). +has nothing to read; told them, it can restart into the bare card it +left behind, and does \- including clearing away its own control mount, +which a dead +.I fw +leaves behind as a mount that fails everything asked of it. +Without them the restart fails and the machine stays off the network, +which is the safe direction but not a working one. +.PP A copy of this file per card is the shape to use; one .I fw per card is the only shape there is. @@ -549,26 +594,14 @@ so a fragmented IPv6 datagram does not cross. IPv4 fragments do: the first one carries the transport header and decides, and the rest of the datagram inherits what it decided. .PP -Taking a card is destructive and is not undone. -The interface that replaces it is unbound when -.I fw -stops, and the address goes with it, so a -.I fw -that is killed leaves the card bound to nothing and the machine with no -network. -It cannot be restarted unaided either: the address it would have read -off the card is the address that has just been lost, so it must be told -one with -.BR -a , -or the card configured again with -.IR ipconfig (8) -first. -Nothing puts the card back. -Whatever restarts -.I fw -must configure the card first, with the address it had, which +Taking a card is destructive and is not undone, on purpose; see +.BR FAILURE . +A .I fw -no longer knows. +that is killed therefore leaves the machine with no network, and must +be told an address with +.B -a +to start again. .PP The first packet to an unresolved next hop is dropped while .I fw diff --git a/fw/src/fw.c b/fw/src/fw.c index bb97960..800e004 100644 --- a/fw/src/fw.c +++ b/fw/src/fw.c @@ -1463,9 +1463,26 @@ threadmain(int argc, char **argv) * that has no card behind it and no network at all, which * is a bad way to discover a typo in -m. */ - if(access(mtpt, AEXIST) < 0) - sysfatal("%s: %r; not touching %s until it exists", - mtpt, etherdev); + if(access(mtpt, AEXIST) < 0){ + /* + * A fw that died left its control filesystem mounted + * here and what remains is a corpse: the name is + * there and every operation on it fails, including + * the check above. So the restart after a crash was + * refused by the very test meant to keep fw from + * taking a card it could not then serve. Clear it, + * the same way reclaim() clears the pkt interface the + * same dead fw left behind; both are its wreckage, + * and a firewall that cannot restart into its own + * wreckage stays down for good. + */ + unmount(nil, mtpt); + if(access(mtpt, AEXIST) < 0) + sysfatal("%s: %r; not touching %s until it exists", + mtpt, etherdev); + fprint(2, "fw: cleared %s, left mounted by an earlier fw\n", + mtpt); + } fprint(2, "fw: filtering packets on %s\n", etherdev); syslog(0, "fw", "started, filtering %s for %s", etherdev, etheraddr); diff --git a/fw/test/fwtest.rc b/fw/test/fwtest.rc index 3f80242..bd7423d 100755 --- a/fw/test/fwtest.rc +++ b/fw/test/fwtest.rc @@ -547,10 +547,71 @@ echo '== a card' r=`{awk '$1 == "0.0.0.0" {print $3}' $tmp/nC/iproute | sed 1q} check 'and the route that went with the card' 10.9.9.254 $"r + # The question a firewall has to answer about its own death: + # does the network come back open, or not at all. A note is + # a fair stand-in for a crash - what matters is the kernel + # closing what fw held, which happens either way. No other + # fw is running here; the leak checks at the end say so. + kill fw | rc + sleep 3 + r=`{ps | awk '$NF == "fw"' | wc -l} + check 'killing one part kills the firewall' 0 $"r + r=`{awk '/4u$/ {print $1}' $tmp/nC/ipselftab} + check 'a dead fw leaves the stack no address' '' $"r + r=`{awk '$1 == "0.0.0.0" {print $3}' $tmp/nC/iproute | sed 1q} + check 'and no route: the network is down, not open' '' $"r + # "device" with nothing after it: the pkt interface is gone + # and the card was never given back + r=`{sed 1q $tmp/nC/ipifc/0/status | awk '{print $2}'} + check 'and nothing bound to hand the card back' maxtu $"r + + # recovery, without ever passing through open. The dead fw + # also left its control filesystem mounted, and a corpse of a + # mount fails every operation on it - including the check fw + # makes before touching a card. So a restart has that to get + # past too. + @{ $fw -n $tmp/nC -e /net/ether1 -m $mtpt $tmp/empty.ndb } >[2]/dev/null + if(~ $status '') + r=started + if not + r=refused + check 'it will not restart on a guess' refused $"r + + $fw -n $tmp/nC -e /net/ether1 -a 10.9.9.1/24 -g 10.9.9.254 \ + -m $mtpt $tmp/empty.ndb >[2] $tmp/rerr + sleep 2 + r=`{grep -c 'cleared '^$mtpt $tmp/rerr} + check 'the control mount the dead one left is cleared away' 1 $"r + r=`{awk '/4u$/ {print $1}' $tmp/nC/ipselftab} + check 'but does when told the address the card no longer has' 10.9.9.1 $"r + r=`{awk '$1 == "0.0.0.0" {print $3}' $tmp/nC/iproute | sed 1q} + check 'route and all' 10.9.9.254 $"r + stopfw $tmp/nC } } +echo '== a dead firewall leaves a sandbox with no network either' +@{ + rfork n + $fw $tmp/empty.ndb >[2]/dev/null + # the drop is per process group and this block has its own, so it + # reaches this shell and its children and nothing else + echo 'chdev' '&~' 'Iluσ' > /dev/drivers + kill fw | rc + sleep 3 + if(@{ ls /net/tcp/clone } >/dev/null >[2]/dev/null) + r=there + if not + r=gone + check 'the filtered /net dies with it' gone $"r + if(@{ bind -a '#I' /net } >[2]/dev/null) + r=allowed + if not + r=refused + check 'and the real one cannot be bound back in its place' refused $"r +} + echo '== supervision' # fw daemonizes, like every other Plan 9 file server: the process you # exec posts to /srv, forks the server and exits. A supervisor watches |