Verification

• Observation
• Interpretation
• Decision
• Adaptation
• Verification
• Learning

Definition

Verification is the test of whether an adaptation produced the effect its decision predicted, against evidence named before the change was made.

The claim

Evidence named in advance is categorically different from evidence selected afterwards. In any sufficiently instrumented organization some measure will have moved, so post-hoc selection can always produce a success.

A second claim, less often made

Verification must test the interpretation, not only the outcome. A change can produce the predicted effect for a reason nobody anticipated, and an organization recording only the outcome will apply the wrong lesson with full confidence.

Known limits

Markets do not hold still during a test. In live conditions the honest output is a confidence level rather than a proof, and any account presenting clean causation for a change made in a live market is claiming more than the method supports.

This page expands one stage of the cycle set out in Section 7. The commercial practice built on the model is described separately by Digilu.