|
The suite passed the first time and failed the second, always on "a
permitted connection crosses, and is tracked". A comment blamed timing
and told the reader that a failure of that check alone was not evidence
of a fault. It was.
Each run left six fw processes alive with their pkt interfaces still
bound. After four runs #I22 looked like this:
0: device pkt0 maxtu 1500 ... pktout 1826 | 10.9.9.1 /120
1: device pkt1 maxtu 1500 ... pktout 0 | 10.9.9.1 /120
2: device pkt2 maxtu 1500 ... pktout 0 | 10.9.9.1 /120
3: device pkt3 maxtu 1500 ... pktout 0 | 10.9.9.1 /120
Four interfaces, one address, and the stack routes out the first, so the
current fw sees nothing and its flows file is empty. pktout 1826 into a
wire whose far end died two runs ago is the trap design.md already
records costing an afternoon. Measured, not guessed: kill every fw,
run once, 22 passed; run again immediately, 21 passed with that check
failing.
fw cannot be stopped by pid -- it daemonizes, so the shell's $apid is
gone before the server exists, and ps shows it no arguments -- and "kill
fw" would be wrong on a machine running a real one. So stopfw takes the
interfaces away instead and fw follows: the relay's read fails and
threadexitsall takes the rest down. That doubles as a live test of the
fail-closed path, since a relay that goes back to dying quietly now
shows up in the two new checks at the end, which count fw processes and
bound interfaces and would have caught this on the day.
Two other ways a check could pass without meaning anything. A "refused"
result was returned for any failure at all, so a check for a hole went
green on a kernel that never had the hole -- gre raw is refused says
nothing if there is no /net/gre. Each such check is now paired with one
asking, outside the sandbox, whether the thing being refused exists.
And the diagnostic could not be read: a failed > is reported by rc
itself and escapes any >[2] around it, so wr does the same create(2)
with cp, whose error lands on its own standard error and is printed
when a check fails.
Finally the port is derived from the pid. A devip Conv is never freed,
so a fixed port made one run's leftovers into the next run's "address in
use".
29 checks, and 29 pass twice in a row with no cleanup between. With the
stopfw calls disabled the two new ones report 6 processes and 4
interfaces left behind.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
|
Every check is a bug that once shipped, which is the only reason to have
any of them. Two would have caught real ones early: a rule set
containing ip= edited through ctl (the %M bug, where both paths were
tested but never together), and a rule set written in two writes (each
Twrite replaced the whole set).
Runs against two IP stacks it makes for itself, so it needs no network
and does not disturb the machine's. Card mode is deliberately not
covered: it takes the card away, and a test that can leave you with no
network is a test nobody runs.
The harness had two bugs of its own worth recording. Counters kept in
variables reported one pass out of seventeen, because every check runs
inside an @{} that needs its own namespace and an assignment there never
reaches the parent; results go to a file now. And a failed redirect is
reported by the outer shell rather than the block, so the message cannot
be captured from inside - the checks test whether a write was refused,
not what it said.
One check is timing-sensitive and marked as such: it passes standalone
and fails here intermittently.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
