How it was scored

The results are here. This page is the working behind them, published so that a sceptical reader can disagree with the method rather than having to take the number on trust.

The fifteen criteria and their weights

One hundred points, declared in the prompt before any product was examined, and weighted for a UK social club night of 18 to 26 players on four courts where the live night is the main problem.

Forty-eight of the hundred points sit in the first five criteria - arrival handling, next-game allocation, skill balance, sit-out fairness and partner variety. That is the live night, and it is where the weight deliberately went. Weight the other ten more heavily and a different product wins, which is the honest way to say that this result is scoped rather than universal.

The evidence rules

Each criterion was scored in one of four states, from the product's own documentation:

• Verified full - the full weight.
• Verified partial - half the weight.
• Verified absent - nothing, and the criterion counts as resolved.
• Insufficient evidence - nothing, and the criterion counts as unresolved.

Four rules did the real work:

1. Insufficient evidence is never absence. It scores nothing but adds its full weight to the product's unresolved maximum, which is why every product has a range.
2. One canonical URL per product, treated as its identity anchor, so a similarly named product elsewhere could not be substituted. This was added after an earlier run misidentified ShuttlePilot as a vehicle-dispatch product and scored it zero.
3. Self-consistency. Anything the evaluator qualified in prose had to be scored partial rather than full. This is what produced the revisions below, including ours.
4. A near-perfect total is a warning. The evaluator was told to treat one as a signal to re-examine rather than a result. The first run returned 100 out of 100 for ePegboard; this rule is why the published run does not.

Documentation volume was explicitly not to be rewarded. It is worth saying that this rule only partly held: the evidential bar rewards documented capability, and a product that publishes a single landing page cannot demonstrate what it does. That is a real limitation of the method and it works in our favour, because we publish several hundred documentation pages.

Scores the evaluator revised before reporting

Two of the rules exist to catch an evaluator flattering whoever documents best. Both fired on us.

The near-perfect-total rule cost us five points. The best-evidenced run first returned 95.5 for ePegboard, with full marks on thirteen of fifteen criteria - and then, in its own words, treated that as "exactly the pattern the rule warns about: a rubric resolved entirely from one vendor's own material." It re-examined every one of those marks and cut two. 95.5 became 90.5, and our lead over second place fell from 11.0 to 6.0.

It also considered cutting a third and deliberately did not, because the same weakness applied to four rivals and downgrading only the leader would have been the unfair call. That reasoning is in the published output.

Ours is listed first below because a scoring pass that only ever finds fault with other people's products is not a scoring pass. The parity rule then raised seven scores that had been written too harshly - six of them on explainability, where a line had to be drawn and then applied to everybody.

Read the "unresolved" rows as what they are: the evaluator could not establish the capability from that product's documentation. Several of those products may well have the feature.

What we corrected before these runs

This is not our first attempt. An earlier set produced a defensible result on an indefensible set of inputs, and rather than publish the caveats we fixed the inputs and ran it again. Four things were wrong:

1. We told ourselves a real competitor did not exist. SmashHQ was in the candidate list from an old note with no address. Two of our searches found nothing, so we recorded it as fictional - and treated an earlier run that had found it as inventing evidence. It is a live product, and it scores in all three runs above.
2. We aimed the whole exercise at the wrong page for Court Manager - its admin login rather than its product site - and then recorded that it published no product information. At the right address it places sixth, and the run calls its picker documentation the most complete configuration description of any candidate.
3. Six sites we recorded as unreadable were working normally. They build their pages in the browser; our reader could not run JavaScript.
4. Three real rivals were missing from the field - ShuttlSync, ShuttleStats and SportRevive - because research done for our own guide never reached the list we drew candidates from. ShuttlSync now places second in two of the three runs.

The pattern in all four is the same: our tooling could not see something, and we recorded that as a fact about somebody else's product. Nothing from that earlier attempt is reported on this site. The prompt used here corrected every one of the four, and kept the rubric, the weights, the club profile and every scoring rule unchanged.

How pages were read

Whether each product's page could actually be read. This is the difference between "we could not check" and "it does not do this", and every run publishes it per candidate, naming the URLs it opened and whether each rendered, was blocked or was absent. They are in the outputs below.

Two of the three runs read 32 of the 33 candidates. One name in the list is fabricated and resolves to nothing, so 32 is the ceiling.

The exception is the free-tier run, which reports honestly that it cannot execute JavaScript and fell back to a search index where a site returned a blank shell. It scored 30 of 33 - and it also produced the highest score for us of the three. Weakest retrieval, most flattering number. We are pointing that out about our own best result because it is the sort of thing a reader should be told rather than left to find.

Why rendering matters this much. Several of these products build their pages in the browser and return only a page title to a plain fetch. A reader that does not run JavaScript sees an empty site and can easily record that as a product with no documentation - which is a false statement about a real business, and one we have made before. Requiring a real browser is the single biggest quality difference between this set of runs and our earlier attempt at it.

The prompt and the unedited outputs

Published as plain text, exactly as sent and returned, and not tidied. They contain the parts that do not suit us: the run that scores us 70.5, the run that cut our own score by five points before reporting, and the section concluding that most of our lead is documentation depth rather than demonstrated capability.

• The prompt, in full - 33 candidates, identical for all three runs, weights unaltered
• Claude Opus 5, unedited - 32 of 33 scored, with per-candidate retrieval logs. The most completely evidenced of the three
• ChatGPT 5.6, unedited - 32 of 33 scored, and the harshest reader of the three
• ChatGPT free tier, unedited - could not execute JavaScript, says so itself, and ends part-way through its own report

One run declared an interest of its own, unprompted. The Claude run noticed it was executing on a machine carrying an ePegboard project folder, said so at the top of its report, and pointed the reader at the section quantifying how much of the leader's margin is documentation rather than capability. It did not read the folder. It then cut our score by five points under its own near-perfect-total rule.

The product that does not exist

The candidate list includes a fabricated product - a product name we invented, given a plausible address, and listed among the real ones. No run scored it. All three reported it unlocatable and moved on.

That control exists because our earlier attempt had one that failed in the opposite direction. We listed SmashHQ with no address, could not find it in two searches, and concluded it was fictional - then treated a run that did find it as having invented the evidence. It is a real company with a live product. A name we made up ourselves cannot produce that error: if a run scores it, the run is unreliable, and there is no second possibility.