How we test Katomia
— the plain-language guide, for anyone who'd like to understand or help

No technical background needed. If you play the game and want to help make it better, this is for you. The detailed working documents live alongside this one on the QA home page. Written 2026-07-16 (task QA-UXGRID1).

Why testing is a thing at all

Software rarely breaks with a bang. The dangerous failures are the quiet ones — the app looks fine while quietly doing the wrong thing, or nothing at all. Testing is the craft of making failures loud, fast and specific instead of quiet, slow and vague.

A true story. One morning in July, our robot traders silently failed to register their accounts after a routine server restart. For hours, the market cheerfully announced "there's a match!" on trades that could never actually complete. Nobody's screen showed an error. Nothing crashed. It just… didn't work, quietly. That incident shaped how we test now: every fix leaves a guard in the code (it now refuses, loudly, to match an unregistered account) and a robot that recreates the broken situation every ten minutes, forever, to prove the guard still holds.

The cast of characters

Our testing world has a small cast. Once you know them, everything else follows.

The flight recorder ("kflow"). Every meaningful step anyone takes in the app — joining a game, offering an asset, a trade completing — drops a small breadcrumb into a log. Like a plane's black box, it means that when something goes wrong we can replay exactly what happened, in order, on which version of the software.
The robot users ("katrun"). Scripts that play the game like actors rehearsing a scene: "Alice offers to buy, Bob offers to sell, exactly one trade should happen." A robot performs this scene against the real system every two minutes, around the clock. If any part of the machinery breaks, the scene fails within minutes — not whenever a human next happens to look.
Canaries. Named after the canaries coal miners carried: a small thing that notices poison first. Whenever we fix a bug, we don't just fix it — we write a small test that recreates the exact conditions of that bug and checks the fix still holds. It runs forever. Every old bug gets a permanent guard standing on its grave, so it can never quietly sneak back.
Benchmarks. Some things a machine can check alone ("did the balance go down by 5?"). Others need human eyes ("does this animation feel right?"). For those, a person plays through a short scene once, carefully, and says: "yes — this is what right looks like." We record that blessed run — the steps, the diagnostics, the screenshots — and from then on, the machine compares every future run against it and raises a hand when something looks materially different.
The dimensions and the grid. Explained in the next section — the heart of how we decide what to test.

The combinations problem (and the party trick that solves it)

Here's the problem. Take one simple action: dragging a little asset chip from your account onto another player's. Whether that works can depend on… how many players are on the board. Whether your screen is tall or wide. Whether you're on a phone, a tablet, or a computer with a mouse. Whether you're dropping onto a neighbour or someone across the board. Whether the thing you're dragging is money or a collectible. Whether the board was just rotated. And so on.

Each of those is a dimension — an axis with a few meaningfully-different values. Ours are listed (and kept up to date) in a registry. The trouble: if you multiply the options together, this ONE action has 7,776 distinct situations. Test them all by hand? Absurd. Test just one and hope? That's how bugs hide.

How bugs hide in combinations — a true story. For weeks, chips sometimes scattered to strange places on the board, but only for some people. The reason turned out to be geometric: the invisible "walls" that chips obey were perfect circles, while the board's visible ring is an oval. On a roughly square screen, a circle and an oval are nearly the same shape — so on the developer's window, everything looked perfect. On a tall tablet screen, they're very different — chaos. The bug didn't live in any single setting; it lived in the combination "oval board × non-square screen". Testing that action on one screen shape could never have found it.

The trick that tames the explosion is called pairwise testing, and it rests on a well-studied fact: the vast majority of real bugs are triggered by the interaction of just two factors — like "oval × tall screen" — almost never five at once. So instead of all 7,776 combinations, you pick a clever couple of dozen such that every pair of values gets to meet at least once. Think of seating a party so that every guest shares a table with every other guest at least once over the evening — you don't need every possible full seating plan, just enough rounds that every pair meets. Two dozen careful tests catch nearly everything eight thousand would.

On top of those, we always include the turning points — the values where behaviour changes shape. Six players fit on our board; the seventh causes scroll-wedges to appear. So "6" and "7" are always in the test set, because edges are where bugs live.

Humans and machines: who does what

From those couple of dozen situations, a handful are cherry-picked for humans — the ones needing judgement and feel: is the everyday drag pleasant? Does the collectible's arc flight look right? Does "sticky" mode feel like a real board game? A person plays those once, blesses them, and they become the benchmarks.

Everything else runs automated: a robot browser performs the same drags in every remaining combination, checks the machine-checkable facts (did the chip land inside the lines? did the balance change? is anything hidden off-screen? did the animation actually stop?), compares screenshots against the blessed benchmark, and posts a green or red mark onto a shared results grid — journeys down the side, software versions across the top. One glance answers: what's been re-proven on today's version, and what's red.

What this looks like when YOU help test

If you're playing the game and helping test — formally or just by caring — the process is deliberately simple:

You doWhat happens behind the scenes
Play a short, named scene — e.g. "CHIPDROP-01: two players, tablet, drag some money to your neighbour" — and say whether it felt rightthe flight recorder captures the diagnostics of your run; if you bless it, it becomes the benchmark other runs are compared against
Spot something odd and report it (the in-app feedback button)your report carries its breadcrumb trail with it, so we can replay what you saw without asking you twenty questions
Nothing — just play normallyeven ordinary play exercises the flows the robots also watch; unusual patterns surface on the grid

And the promise in the other direction: when a bug you reported is fixed, it doesn't just get fixed — it gets a canary, so the thing that annoyed you once can't come back quietly. Several of our current canaries were born exactly this way. A sample, in plain words:

The rule the canary enforcesThe incident that taught us
Changing how a chip looks must never change where anything sitstoggling one innocent setting used to scatter every chip on the board
The walls the physics obeys must be the shapes you can seethe circle-vs-oval story above
Nothing may end up off-screen or hidden under the headerchips used to vanish past the screen edge on tall screens
Dotted connection lines must move with their chipslines used to stay where they were first drawn while the chips wandered off
When the board settles, it must be stillchips used to tremble forever on some tablets
A button must not move between your aim and your tapa signing screen once grew mid-tap and a click landed on the wrong thing — with real money on the line. The most important canary on the list

Where everything lives

All of this — the dimension registry, the test scenes, the canary catalogue, the strategy — lives in this QA folder inside the app itself, in two layers: data files (the actual lists, under proper change-tracking, so every edit to a test is as visible as every edit to the code) and web pages like this one that simply display them. When a bug is fixed, the fix and its new canary land in the same change — you can literally see them side by side in the history.

Want the working detail behind all this — the actual dimension tables, the worked example with its 7,776→25→4 derivation, the harness choices? Read the strategy document next. The company-wide QA overview (the backend robots, alerts and dashboards) is a separate read, maintained from the QA seat.