Sharing a Secret Is Easy; Sharing State Is Not: Six Traps in Cross-Machine SSO

Extending a login gate from one machine to single sign-on across several subdomains looks like a matter of getting the other two machines to trust the same key. The key part genuinely is simple — same code, same signing key, each site verifying independently.

The hard half is the other one: a secret can be copied; state cannot. This records the problems encountered running one gate across three machines. Almost all of them live where I assumed something was shared and it was not, or assumed it was not and it was.

What the setup looks like

Three machines each run an identical gate in front of their own application. A user logs in once on any of them, the browser receives a cookie scoped to the parent domain, and subsequent visits to other subdomains carry it automatically for each site to verify on its own.

cookie = b64u(payload) . b64u(HMAC-SHA256(payload, cookie_key))
payload = {"exp": expiry, "u": username, "v": version}

All three share one cookie_key, so a cookie minted by any of them verifies on the other two. Nothing clever here, and it did work first time.

Replay protection is per-site, not global

The first problem is in the design, and only became apparent after it worked.

One-time-code replay protection depends on a spent counter: record the most recently used time step and require subsequent codes to strictly advance. The three machines share the code seed — necessarily, or a single authenticator app’s codes would only work on one of them — but each maintains its own state file.

The consequence: a code from one 30-second window, once used on machine A, can be used again on machine B, and again on C. Replay protection became per-site rather than global.

This is not an implementation error. It follows directly from sharing a secret without sharing state. Making it genuinely single-use would require the three machines to share one counter, which means shared storage and cross-machine locking — introducing a new single point of failure to close a replay window bounded at two extra uses within 30 seconds. Not a good trade.

So the correct action here is to write it down as a known and accepted property rather than leave it unstated. The real attack surface is: an attacker needs a valid, just-used code within 30 seconds and the correct password. Known-and-accepted and unknown are different states.

The key has a second copy, and the rotation script does not know

This one was the hardest to diagnose, because its symptom points at the wrong place.

The application on one machine needed to verify the cookie a second time itself, beyond the gate, for its own session logic — so its configuration file also held a copy of cookie_key. When the key rotated, the sync script updated all three gates and missed that second copy.

The symptom: nginx’s auth subrequest succeeded (the gate verified with the new key), the request proxied through, the application verified with the old key and returned 401. And the gate’s log showed nothing wrong — its side was entirely healthy.

It is easy to go looking at the gate, because “401” matches the intuition of “authentication failed.” The useful discriminator is: did the gate’s access log record this authentication as successful? If it did, the problem is downstream of the gate.

What generalises: every secret needs a maintained list of who holds it, and rotation follows the list rather than memory. Any secret with a second consumer will be missed during some rotation — the only question is whether it is this one or the next.

A too-narrow preservation rule in the deploy script switched login off

The same class of problem in another form.

That machine’s deploy script rewrites the application’s environment file, and to avoid clobbering runtime configuration it preserves variables matching a prefix. The original pattern was ^APP_PANEL_.

The SSO key variable added later is named APP_HB_COOKIE_KEY — which does not match. The next deploy erased it and login stopped working outright.

# before: only PANEL-prefixed vars survive, so new ones get erased
grep -E '^APP_PANEL_' "$ENV" > "$KEEP"

# after
grep -E '^APP_(PANEL|HB)_' "$ENV" > "$KEEP"

This genuinely happened once. The lesson is not “the regex was too narrow” but that allowlist-style preservation rules fail silently as configuration grows — a new variable outside the allowlist raises no error, it simply disappears.

The more robust arrangement inverts it: preserve everything by default and explicitly overwrite only the entries the deploy needs to update. Then the default behaviour for a new variable is to survive, not to vanish.

Clearing the application’s own session is not logging out

After adopting SSO, the application’s existing logout button becomes a trap.

It clears the session credential the application issued. But on the next request the gate sees a still-valid SSO cookie and passes it through as usual — a refresh puts the user right back in a logged-in state. The “sign out” button in the interface looks perfectly normal; it simply does nothing.

Correct logout must reach the gate’s logout endpoint and clear the parent-domain cookie. And because that cookie is domain-wide, doing so logs the user out of all three sites at once — which is right. The dual of single sign-on is single sign-out.

The common thread: after adopting unified authentication, every identity-related piece of legacy logic in the application needs re-auditing. Login, logout, session expiry, permission changes — for each of them, the source of truth has moved. Left unaudited, none of them error; they just behave incorrectly in some particular scenario.

Compare the stored material, not what it generates

One further trap is purely about measurement method, and deserves its own note because it manufactures phantom failures.

After syncing credentials you want to verify all three machines match. The intuitive approach has each generate a current code and compares them:

# produces false mismatches
for h in host-a host-b host-c; do
  ssh "$h" 'gate.py now'
done

The problem is that codes are generated from a 30-second time step. Three SSH calls run in sequence with hundreds of milliseconds to seconds between them, and if that gap happens to straddle a step boundary, the later machine reports the next window’s code. It looks like the credentials disagree when they are identical.

That phantom failure readily sends someone re-running the sync and investigating the network, costing far more time than the check itself.

The right approach compares the stored material — hashes of the seed and key, which do not vary with time:

for h in host-a host-b host-c; do
  printf '%-14s ' "$h"
  ssh "$h" 'sudo sha256sum /etc/gate/secret.json | cut -c1-16'
done

Generalised: to verify two configurations match, compare the configurations — not artefacts derived from them that carry time or randomness. A mismatch in derived output can come from the configuration differing or from the derivation itself, and you cannot tell which.

As an aside, these three machines have no fully connected direct path — inbound from the cloud machine to the two at home does not work. So the sync script has to run from a host that can reach all three and act as the relay. That constraint does not affect correctness, but it determines which machine the script lives on, and it is worth confirming during design rather than discovering after the script is written.

Under proxy auth, exempt paths need more care than protected ones

Two of the three applications support proxy authentication — the application stops verifying credentials itself and trusts an identity header supplied by the reverse proxy. The mode makes SSO land cleanly, and it outsources the entire “who is this” decision to that header.

Which turns every authentication-exempt path into a potential back door. Public share links, health checks, machine-facing API endpoints — each bypasses the gate for its own valid reason, and unless each explicitly blanks the identity header, anyone who sends it themselves walks in as whoever they like.

location / {
    auth_request /__auth;
    auth_request_set $user $upstream_http_x_auth_user;
    proxy_set_header X-Auth-User $user;      # only ever from the gate's response
}

location ^~ /share/ {
    proxy_set_header X-Auth-User "";          # exempt paths must blank it
}

A concrete test applies here: wherever an exemption is written, ask whether that path passes through, unchanged, any input the authenticated path depends on. Headers, cookies and query parameters all count.

The three machines have different exemption lists — one exempts public downloads, another exempts Bearer-authenticated machine endpoints. Different lists mean correctness cannot be inherited by copying another machine’s configuration; each needs its own pass. That is what makes multi-machine deployment harder than single: the configuration looks identical and the exceptions are not.

Sync is one-directional, because the links themselves are asymmetric

The shape of the credential sync script is dictated by one physical constraint: reachability between these three machines is not symmetric.

The cloud machine has a public address; the two at home have no stable inbound. So cloud → home does not connect while home → cloud does. That makes sync a pull rather than a push: initiated from the side that can reach the cloud, fetching material and writing it locally.

It is easy to write the script assuming all three can reach each other, because during local testing the machine in front of you genuinely can reach all of them. The real topology is a directed graph, not a fully connected one. Before designing any multi-machine procedure, measure the reachability matrix — every ordered pair — rather than inferring “they can reach each other” from “my laptop can reach them.”

for a in host-a host-b host-c; do
  for b in host-a host-b host-c; do
    [ "$a" = "$b" ] && continue
    printf '%-10s -> %-10s ' "$a" "$b"
    ssh -o ConnectTimeout=8 "$a" "nc -z -w 5 $b 22 && echo ok || echo unreachable"
  done
done

That matrix has a second use: it determines which machine changes first during key rotation. Rotate the initiator first, while it cannot yet push to the others, and there is a window where the sites no longer authenticate each other’s cookies. The correct order has every verifier accept both old and new keys, then removes the old one once all are updated — one extra deploy in exchange for eliminating that window.

The cost this design already carries

One last thing that is not a bug but must be understood.

A cookie scoped to the parent domain is sent by the browser to every subdomain, including the one running WordPress. HttpOnly prevents page JavaScript from reading it; it does not prevent a compromised server from reading it straight out of the request headers.

Which means this SSO binds the security of every subdomain to whichever one is weakest. This is not something adding sites created — it held from the moment they shared a cookie domain. Another site makes it no worse, and no better.

What you can do is write it into the design document and use it to decide which services do not belong in that domain. Genuinely sensitive material deserves its own hostname and its own credentials rather than being attached to the same SSO for convenience.

Which is the point of all of this: single sign-on saves the user repeated typing, at the cost of merging several systems’ trust boundaries into one. Once merged, the boundary is only as strong as its weakest segment — a conclusion that belongs in the decision to adopt it, not after.