A row with no workspace stamp is invisible, not missing

Three words that are not synonyms#

Missing

The row does not exist. Something failed before it was written, or it was hard-deleted. This is the only one of the three where a backup is part of the answer.

Deleted

The row exists and a person put it out of sight — threads.deleted_at and its equivalents on calls. Reversible, deliberate, and the person's own decision to make.

Invisible

The row exists, holds live data, and is excluded by a predicate. Nothing failed and nothing raised. Restoring a backup here creates a second copy of a record that was never gone.

Choosing between them by instinct is how one hidden row becomes two live ones. The case that made this concrete: two connected mail identities vanished from a screen with no duplicate warning, no error and no failing request — and the uniqueness check that should have prevented a reconnection could not see them either, so connecting again would have written a second row for one real inbox, both holding credentials.

The triage, in about a minute#

1. Fetch the record by its identifier — a link, a permalink, an action that takes an id.

   Result If it answers, the row exists and this is a visibility problem. Nothing needs restoring, and the incident is now about a predicate rather than about data loss.

2. Ask, read-only, how many rows in that table carry an empty stamp rather than a NULL one.

   Result The two are different faults. A NULL usually means a column was added without a backfill; an empty string usually means something wrote a default it never intended. The counts tell you which story you are in.

3. Check the uniqueness path for the same key.

   Result If the key reads as free while a hidden row holds it, a duplicate is one user action away and the screen is actively inviting one. That is the urgent half of the incident.

4. Only then consider a backup, and only if step one found nothing.

   Result A restore against an invisible row is how a support ticket becomes a data problem.

What else hides a row that exists#

The last row is worth sitting with. The same class of oversight produces both invisibility and over-visibility, depending on which half is missing: a stamp that matches nothing hides a row from its owner, and a table that nothing filters shows rows to everybody.

Why the fix is not a quick update#

The instinct is to stamp the row as soon as something notices. That write is refused: the policy's WITH CHECK will not allow a row to be updated into a scope it is not already in, so an attempt from a request turns a quietly hidden row into a server error on somebody's screen. The permission that would allow it is not one a request should hold.

So the repair belongs to the schema owner and runs where schema changes already run — migrate.backfill_canonical_workspace, on every boot, on the connection that legitimately writes across scopes. The mechanics of that repair are written up in a row with an empty workspace id; this page is about the decision that comes before it.

• A NOT NULL constraint prevents the next one and finds none of the existing ones. Repair first, constrain afterwards, or the migration fails on the rows you were trying to fix.
• The read-only count proves how many rows are hidden now. UNKNOWN: it cannot say when each was written or by which path, because nothing recorded a fault at the time.
• Control-plane tables cannot hold this defect — platform identity, sessions, billing and website enquiries are outside every workspace deliberately.

Questions#

Related#

• Data that is invisible rather than missing
• Three independent layers of tenant isolation
• Headline numbers that stop at the page size
• Why 'connected' is not enough to prove mailbox health
• Troubleshooting
• Security and isolation