← Devlog
devonboardauthtestingprocess

A green test suite is not a working feature

Nineteen passing tests said the auth flow worked. A ten-step manual walkthrough found three bugs they couldn't see.

DevOnboard’s auth frontend merged this week — but not when the tests went green. It merged when a ten-step manual walkthrough cleared, and the gap between those two moments is the story.

The walkthrough is a checklist I wrote before touching a browser: bring the stack up, confirm the login redirect, complete the OAuth flow, verify the token and role, hit a bad URL for the 404 page, log in as a non-admin user, try to access admin routes as that user, and force a token refresh. Every step has an expected outcome. Merging is gated on all ten.

Step one failed immediately. Three bugs, none of which any unit test could have caught:

  1. The .env file was missing — lost to .gitignore on a fresh branch. Recreated from .env.example. Tests never noticed because they mock the environment.
  2. The login page was building the GitHub OAuth URL client-side using an env var the frontend never actually had. The fix was better than the bug: point the login button at the backend’s /auth/login endpoint and let the server own OAuth initiation entirely — client ID stays server-side, and the CSRF state cookie (which client-side construction had silently skipped) is now set correctly.
  3. The OAuth callback redirected to the wrong port — a FRONTEND_URL default pointing at the production port instead of Vite’s dev server.

The walkthrough also forced me to test as a user who isn’t me. Verifying the non-admin path meant creating a throwaway GitHub account — and that step exposed a subtlety: navigating to an admin route by typing the URL reloads the page, wipes the in-memory auth store, and makes an unauthorized user look unauthenticated. Client-side navigation shows the real behavior. Knowing the difference matters when your session state is deliberately memory-only.

One honest find went straight onto the backlog rather than getting a quick patch: a hard refresh currently logs you out, because nothing restores the session from the refresh cookie on app init. That’s a real gap — but it’s a scoped, understood gap with a plan, which beats a hidden one.

Tests verify the pieces. The walkthrough verifies the product. I’m keeping both, and the checklist gate stays for every feature that follows.