Reading poster scan data without fooling yourself

Poster scan data has a specific way of misleading you: it looks like measurement, so it gets treated like proof. Once every printed copy has its own QR code you have real numbers per location — and real numbers with a small sample and an unfair denominator will send you confidently in the wrong direction. Here is how to read poster scan data so that it actually improves the next print run.

Totals mostly measure how many you printed

The first chart everyone looks at is scans per design or scans per batch, and it is almost always the wrong chart. If you printed 80 of design A and 20 of design B, A will win. That tells you nothing you did not already decide at the printer.

The comparison that means something is scans per registered print: total scans on that design or that group, divided by the number of copies you actually put up and recorded. It is a rate, not a count, and it is the only version that survives unequal print runs.

This is why the A/B view in Wildpost compares on scans per registered print and never on raw totals, and why it counts registered prints rather than generated codes — a code that was printed and left in the box has no chance to be scanned and would quietly deflate the design it belongs to.

Location dominates everything else

Before you conclude anything about your artwork, check where the copies went.

Dwell time is the single largest effect in this data by a wide margin. A poster at a bus stop will outscan the identical poster on a busy pavement several times over, and that gap is larger than any plausible difference between two designs. Which means:

If design A ended up mostly in waiting places and design B mostly on lampposts, you have not run an A/B test on the designs. You have measured bus stops.

Two ways to avoid it, in order of preference:

1. Randomise placement. Alternate designs as you work down a route, so both end up in the same mix of spot types. This is the one that actually works.
2. Compare within a spot type. If placement was not randomised, compare A and B only among the notice-board copies, then only among the window copies. Smaller samples, fair ones.

Know your sample floor and respect it

The uncomfortable arithmetic: with 12 scans on one design and 9 on another, you have learned nothing. That difference is well inside what coin-flipping produces.

Useful working floors:

Wildpost's A/B tab refuses to name a winner below its sample floor, which is a deliberately annoying feature. The alternative — showing a percentage lift computed from eleven scans — is the single most effective way to make someone confidently wrong.

Read the shape of the curve, not just the height

The daily series per spot tells you things the total cannot.

• A spike then nothing. Almost always one event: a market day, a match, a passing group. Real, but not repeatable, and not a reason to put more posters there next month.
• A slow steady trickle. The best kind of spot. Regular passers-by, week after week. These are the ones to repeat.
• A ramp that stops dead. The poster came down, got covered, or the weather got it. Worth a walk past to check — a spot that died at day nine may be a great spot with a paste problem.
• Weekend peaks. Tells you what kind of place it is. Useful for deciding when to put up the next round rather than where.

A spot with 60 scans in one afternoon and a spot with 60 scans spread over three weeks are not the same spot, and the ranked list shows them as equals.

Where scans do not come from

Three sources of noise worth knowing about, because each has a tell:

• You and your team. Everyone tests their own code. Test scans arrive before the placement is registered and are visibly clustered, so if a spot has scans dated before you put it up, that is what they are.
• Repeat scans by the same person. Suppressed within a short window using a one-way hash of the browser's user agent — no IP address is stored — which removes the obvious double-counting. A curious person scanning again the next day still counts twice, and nothing can distinguish that from a second person.
• Crawlers. Search engines that find a scan URL will fetch it. robots.txt disallows /s/, which handles the well-behaved ones. A sudden identical count across many spots on the same day is the signature of one that is not.

What to actually do with a fortnight of data

A short, honest routine:

1. Sort by scans per print, not by total. Ignore anything with fewer than five registered prints behind it.
2. Take the top five spots and the bottom five. Look at what the top five have in common. It is nearly always dwell time.
3. Check the curves of the top five. Drop any that are a single spike; those are events, not places.
4. Write the survivors down as a list of named spots. This list, not the chart, is the asset. It is what makes the next campaign cheaper.
5. Only then look at the design comparison, and only if placement was randomised.

That is the whole method. Poster scan data is genuinely useful, but almost all of its value is in one output: a short list of specific places, by name, that reliably work for your audience. Everything else on the dashboard is in service of producing that list.

Related: where to put up posters and how to put a QR code on a poster.