Requirements are written in natural language, which can be ambiguous. A sentence can carry multiple meanings while remaining grammatically correct. This article examines one such sentence. It explores the design that replaced the literal reading and details four traps encountered during development. None of these issues reached a vessel, but all of them would have compiled.

The sentence

My project segments ship networks into zones and controls their communication. Each zone operates in specific states. In “observe” mode, the gateway accepts all traffic while logging what a policy would have dropped. In “enforce” mode, the gateway drops denied traffic.

New zones start in “observe.” This is logical, as no one wants an untested engine room policy to be enforced immediately. The specification also requires new policy versions to run in “observe” for 24 hours. This makes sense for new deployments. However, the requirement is problematic for established zones.

If an operator adds a single rule to a long-running, enforced zone, the current logic forces it into 24 hours of “observe.” This effectively disables the firewall’s security for a full day after every minor change. This is an unacceptable security risk.

Two potential fixes were rejected. First, some suggested skipping the 24-hour period if a rule only tightens policy. While this doesn’t open new traffic, tightening is exactly what causes operational breaks. We need that “observe” time to identify these issues. Second, some suggested accepting the 24-hour gap as the price of caution. This is also flawed because it sacrifices security to gather information that could be obtained through other means.

The design: run the new version beside the old one

A shadow stage has replaced the literal reading of the design. The current version continues to enforce rules in active zones. Meanwhile, the candidate version is compiled into a secondary set of chains. These chains monitor the same packets as the active ones. They do not make decisions, but they count what the candidate would allow or deny.

After 24 hours, the staging logic reviews these counts. If the candidate would deny a necessary flow, it remains in the staging area until an engineer releases it. Only then does the candidate replace the active version, zone by zone, in a fixed order.

Zones that have never enforced rules have no existing protection. As originally specified, these zones run in observation mode for the full 24 hours. The core requirement remains intact. It is now understood as a period for the zone to learn, rather than a state it must occupy.

Nftables has a compact structure. In the forward base chain, the system accepts replies to existing connections. Before checking the zone’s enforcing chain, the first packet of each new flow from the zone jumps to the zone’s shadow chain:

chain forward {
    type filter hook forward priority filter; policy drop;
    ct state untracked jump admit
    ct direction reply ct state established,related accept
    iifname "vlan10" ip saddr 10.10.10.0/24 ct state new jump shadow-office
    iifname "vlan10" ip saddr 10.10.10.0/24 jump from_office
    ...
}

The shadow chain dispatches traffic by destination, just like the enforcing chain. It sends packets to chains that count “shadow-allow-office” or “shadow-deny-office.” These chains record potential denials in a set and then return. The packet then reaches the “jump from_office” enforcing chain. It proceeds as if the shadow chain never existed. That is the design. However, the phrase “as if the shadow had not been there” was the trap.

Trap one: a verdict inside a jumped chain is final

It is tempting to view a jumped chain as a subroutine with a return value. In nftables, this is incorrect. A “jump” pushes a return point, and a “return” or the end of a chain pops it. However, a verdict is not a return value.

An “accept” verdict in a jumped chain stops packet evaluation immediately. A “drop” verdict ends it everywhere. The calling chain cannot see or override these outcomes. Consequently, if a shadow chain uses “accept,” it approves the packet before the enforcing chain is ever reached. This turns the shadow into an unintended union of the old and new policies. A shadow “drop” is even worse, as it enforces denials prematurely.

Therefore, shadow chains must follow strict rules. They cannot contain “accept,” “drop,” “reject,” or “queue.” They cannot jump to an enforcing chain. Every shadow chain must end in “return.”

Inside the shadow, dispatching to per-destination chains must use “goto” instead of “jump.” This replaces the current chain without pushing a return point. As a result, the final “return” leads back to the forward chain, just before the enforcing jump. A test validates these requirements by scanning every rule in every shadow chain. We identified this necessity during the design phase. Articulating these rules clearly makes them easy to follow, proving the value of explaining complex designs aloud.

Trap two: the candidate leaked into the enforcing chains

Several policy rules use “internet” as a destination. The gateway converts this word into specific addresses by using exclusion. It defines the internet as anything that is not a private range, a special range, or one of the vessel’s own networks. The vessel’s own networks include the zone segments and any segment named as a source in a policy rule. This final part created a trap.

The compiler builds enforcing chains from the active version. It builds shadow chains from the candidate version. To find the vessel’s networks, the first implementation combined every version in use. This included the active version and all candidates. This approach seemed like the cautious choice. However, it had a specific effect. If a candidate named a new segment, that segment was added to the exception list of every enforcing chain’s “internet.” Previously, the enforcing version treated flows to that range as internet traffic. Now, those flows would no longer match the internet rule. Instead, they would fall through to later entries in the chain. Since the default for an enforcing zone is to deny, these flows would be blocked.

Private ranges are always excluded, so this issue only impacts segments with public addresses. However, the principle remains important. A version intended only for monitoring was already influencing the active version. A code review identified this problem when the per-zone compiler was written. The fix ensures that each version’s chains only exclude the networks that version explicitly names. Address translation toward the internet now follows only the version in force. Consequently, a candidate version no longer affects anything outside its own shadow chains.

This trap is caused by shared state between two systems that should be independent. In this case, a single list of networks was computed once and used everywhere. While correct for one version, this approach was subtly flawed for two.

Trap three: admissions needed shadows too

Not every packet the gateway sees belongs to an established flow. On some vessels, traffic leaves through a router, and the gateway only sees the incoming replies. Because these replies lack connection-tracking state, the gateway uses “admissions” to handle them. Admissions are rules derived from policy that accept untracked replies for flows opened by an outside peer. In the forward chain, these appear as the “ct state untracked jump admit” line, which takes precedence.

The initial shadow design only covered the zone’s chains. If a new version changed only the admissions, it was given a 24-hour stage, but its shadow remained empty. Additionally, the dependency check only inspected the chains to ensure a candidate did not deny a critical flow. Consequently, a candidate that removed a necessary admission could pass the check and be enforced.

To fix this, we gave admissions a shadow form. The candidate’s admissions are now compiled into count-only entries. The shadow chain is accessed from the “admit” chain for every untracked packet. A matching entry counts the packet as a “shadow-allow” and returns.

The dependency check now evaluates inbound flows—those opened by a peer—against the admissions of either the current or candidate version. If a zone’s candidate changes only its admissions, the shadow chain correctly reflects those entries. The lesson is that we must properly enumerate all inputs. We previously defined the shadow as “the candidate’s chains,” but we needed to shadow “everything the candidate decides.” The two were equivalent until they were not.

Trap four: making the wrong output unrepresentable

The first trap tests rendered text for forbidden words. This test is necessary, but it only addresses a symptom. The root cause is a renderer that contains instructions to accept forbidden text and a code path that emits it.

The initial implementation reused the enforcing rule type for shadow rules. It used a generic type to determine the outcome: an “action” (allow or deny) for enforcing rules and a “ShadowVerdict” (count-only) for shadow rules. While this was a reasonable start, it was incomplete. The shared rule type still contained the original “action” field. As a result, a shadow rule still held a real allow or deny command, which could be accidentally triggered by one careless line of code.

The fix separated these types. A shadow rule is now its own type and no longer contains a field for an action:

pub enum ShadowVerdict {
    /// The candidate would let the flow pass.
    WouldAllow,
    /// The candidate would deny the flow.
    WouldDeny,
}

pub struct ShadowRule {
    pub src: Match,
    pub dst: Match,
    pub proto: Option<Proto>,
    pub ports: Vec<PortRange>,
    pub verdict: ShadowVerdict,
    pub rule: Option<usize>,
}

A shadow admission contains no verdict. The renderer’s code for shadow chains only receives these specific types. There is no function to convert a “ShadowVerdict” into an nftables verdict. “WouldAllow” acts as a counter and a return. “WouldDeny” records the event in a denied set, then acts as a counter and return. To render a shadow “accept” by mistake, a future update would need to add an action back to the type. This would be a visible decision in a code review rather than an invisible one in a template.

The text test remains. The type makes the mistake difficult to write. The test makes it impossible to ship if someone writes the mistake anyway. Neither approach covers the other fully. A type cannot detect a hand-written rule string. A test cannot detect a mistake in a policy shape that it does not render.

What it costs, and what it does not cover

  • A staged zone maintains two sets of chains while its candidate runs: one for decision-making and one for counting. We have not measured the performance cost.
  • The shadow process only triggers for a flow’s first packet or for untracked packets. Therefore, established traffic does not incur this cost. Each shadow tracks only the traffic from the previous 24 hours. Consequently, flows that occur weekly or infrequently may not appear.
  • For critical flows, the candidate is also checked against data from the past week, rather than just the current shadow. The system informs the operator which traffic the candidate would deny. It does not determine if that denial is correct. That decision remains with the operator, and the stage persists until they act.

On 2026-10-24, we tested this in the lab using a compressed 24-hour timing file. A staged version ran its shadow and transitioned zones into enforcement. The system correctly paused when the candidate blocked a navigation data flow that had appeared over six days. The operator then released the hold from the gateway page, and the system rolled back the zone after repeated flow denials. The run successfully passed all 21 checks.

Lessons

The original requirement was technically correct. It identified the right goal: providing a day of evidence before enforcement. However, the author chose a specific, flawed mechanism to reach it. The team kept the goal but changed the mechanism. When a requirement seems surprising, follow this pattern: identify its underlying purpose. Then, determine if a different mechanism can achieve that purpose without the surprise.

The four traps shared a common feature. In each case, a passive element gained an active path. A verdict became final, a list was shared, an input was forgotten, or a field was left behind. A shadow only remains a shadow if all these paths are closed. Each path had to be identified and secured individually.

References