Application Test Sweep · Sample Report

Nodl — getnodl.app

Exploratory quality & security sweep of a multi-tenant SaaS (React 19 · WebGL canvas · Supabase)
Performed by
Erik Erelof, Senior Test Lead
Method
Human-led, AI-accelerated
Period
June – July 2026
Verdict
Ship-ready
Why this report is about our own product. Nodl is built by us. Before its public launch, we ran the same sweep we sell — and published everything it caught, including the issues that were our own fault. A sample report that finds nothing would sell nothing. Every finding below is real, was reproduced, and is now fixed or explicitly accepted.

1Verdict

Summary

Ship-ready. The sweep found 2 high-severity issues — both in the authentication and access-control layer, both fixed and re-verified before launch. Two medium-severity stability issues were fixed or mitigated. The product's multi-tenant isolation, bot protection, and load behaviour were verified directly against production. One low-risk item is consciously accepted and documented.

2
High · fixed
2
Medium · resolved
3
Low / accepted
4
Verifications passed

2Scope & method

A sweep is session-based exploratory testing performed by a senior human tester, accelerated by AI tooling — not a scripted test plan, and not an automated scan.

The human decides where to dig, judges severity, and signs every finding. AI is used to move faster: generating probe scripts, simulating concurrent load, drafting reproduction steps, and sweeping wide surfaces the human then verifies by hand. Every finding in this report was reproduced before it was written down.

AreaWhat was exercised
AuthenticationSign-up trigger behaviour, anonymous demo sessions, sign-out flows, session revocation timing, captcha enforcement
Access controlRow-level security (RLS) policy audit across all multi-tenant tables; direct API probing with real user and anonymous JWTs
Load & capacityConcurrent-visitor burst against production, per-visit payload cost, auth rate-limit configuration
Client stabilityWebGL renderer lifecycle across SPA navigation, race conditions in async UI flows, browser matrix

Environments: production (read-only probing) and an isolated staging environment for anything invasive. No customer data was accessed or created.

3Findings

High Public demo visitors silently created permanent accounts in the tenant database Fixed & re-verified

The product offers a no-signup public demo that uses anonymous authentication. The database trigger that provisions a new workspace on user signup also fired for these anonymous demo sessions — so every single demo visitor caused a permanent ghost company, workspace, and profile to be created in the production database.

Beyond uncontrolled data growth, ghost tenants polluted the billing entity table and would have skewed every customer count the business relies on. Launch-day traffic would have created thousands of them in hours.

Reproduction
  1. Open the public demo page in a private browser window
  2. Let the demo canvas load, then close the tab
  3. Inspect the tenant tables: a new company, workspace and profile row exist for a user that never registered
Resolution

Guard added to the signup trigger: anonymous sessions return immediately and provision nothing. Re-verified by repeating the reproduction — no rows created.

High Five row-level security gaps in multi-tenant access control Fixed & re-verified

A full audit of the database's row-level security policies — the layer that keeps one customer's data invisible to another — found five policies that granted broader access than intended, including definer-privilege functions that skipped membership checks. The gaps were found by reading every policy line by line and then probing the API directly with real JWTs from differently-privileged test users, not by trusting the policy names.

Reproduction

Withheld by design. We document access-control findings generically in public reports and provide full reproduction detail to the product owner only. (Here we are the product owner — the point stands for customer reports.)

Resolution

All five policies rewritten and hardened in two dedicated migrations; membership-check helper functions made consistent. Re-probed with the same JWT matrix after deployment — all cross-tenant requests now return empty.

Medium Sign-out race left the app hanging on a dead session Fixed

The demo page revokes its anonymous session when the visitor leaves. But the app's own navigation into the canvas also counts as "leaving" — so the server-side session revoke raced the canvas boot. When the revoke won, the canvas booted on a dead token and hung on an endless spinner. Timing-dependent: invisible in local development, frequent in production.

Reproduction
  1. Enter the demo on a production-latency connection
  2. Observe the transition into the canvas view intermittently hang on the loading state
  3. Network log shows the token revocation completing before the canvas's first authenticated fetch
Resolution

The leave-handler now recognises the app's own navigation and skips revocation for it; real departures still revoke.

Medium WebGL renderer crash on repeated SPA re-entry Mitigated · root cause tracked

Navigating demo → canvas → browser-back → demo again produced an empty canvas and an unrecoverable renderer exception (TypeError in the render-target system). Deterministic in both Firefox and Chrome. Root cause: module-level renderer state survives SPA navigation, so a second mount reuses destroyed GPU resources.

Reproduction
  1. Open the demo and enter the canvas
  2. Press browser back, then enter the demo again
  3. Canvas renders empty; console shows the render-target exception
Resolution

Shipped mitigation: re-entry after a previous mount forces a clean page load, wiping the stale renderer state; first-time visitors keep the fast path. Full root-cause cleanup (renderer singletons, texture caches, probe contexts) is tracked as follow-up work. We report mitigations as mitigations — not as fixes.

Low Ghost frame from a hidden view bleeding through a view switch Fixed

Switching between the two main views occasionally flashed a UI row from the outgoing view over the incoming one for ~150 ms. Cause: a CSS transition shorthand also animated the visibility property that the view switch relies on, keeping the hidden view painted after the switch. Fixed by transitioning only the intended properties. A one-line cause that took real reproduction discipline to pin down — glitch reports of this kind are verified with frame-by-frame screen capture, not dismissed.

Low Onboarding modal race on first load Fixed

The first-run onboarding modal raced initial data loading and could appear for users who already had content, or fail to appear for genuinely new users, depending on fetch timing. Fixed by gating the modal on data-ready state rather than mount timing.

Low Anonymous demo sessions accumulate user rows Accepted risk · documented

Each public demo visit leaves one row in the authentication user table. No security or cost impact at current scale; purely hygiene. Documented with a recommendation for periodic cleanup. We list accepted risks openly rather than padding reports with them as findings — or worse, hiding them.

4Verified under fire

Passes are findings too — these were probed directly against production, not assumed:

CheckResult
Bot protectionScripted anonymous sign-in without a captcha token rejected by the auth API (400)Pass
Demo isolationAnonymous JWTs are read-only and scoped to the demo tenant; write attempts rejected at the policy layerPass
Launch-day load50 concurrent simulated demo visitors: 500/500 API queries succeeded, worst-case visit ≈ 2 s under a sequential worst-case model (the real client fetches in parallel)Pass
Capacity headroomFull demo visit costs 46.8 KB of egress — ~5,000 visits ≈ 5% of the hosting plan's monthly budget; auth rate limit confirmed at a sane defaultPass

5What we'd test next

A sweep is honest about its edges. Given another session on this product, the next charters would be:

6What a customer report looks like

Exactly like this one — for your product instead of ours. Fixed price, 48-hour turnaround. You get every finding with reproduction steps, severity judged by a human who has done this for a living for years, a verified-passes section, and a next-charters list. Access-control reproduction details stay private to you.