The WSPR User Manual sets out the purpose of WSPR:
The WSPR software is designed for probing potential radio propagation paths using low power beacon-like transmissions.
Though that talks about measuring radio paths, it is often used to compare transmitters or receivers over radio paths.
WSPR SNR measurements include the end to end radio path, which on some bands is highly variable, so using WSPR reported SNR values to compare two transmitters can be quite challenging.
We are all familiar with ad-hoc tests where a station might switch between two antennas and ask for comparative reports from receiving stations. At time when the radio path characteristics change greatly, changes in transmitter are often masked or confused by path variation.
Of course some practitioners will conduct several so-called A/B changes, perhaps as many as five and someone (receiver or transmitter) makes an informal judgement of the central tendency of the observations. The observations might be given in quite subjective terms, or in quantitative terms, possibly from an S meter of unknown calibration.
Repeated measurements of the same thing, or same type of thing (eg 10 measurements of 1 new dry cell, or one measurement each of 10 new dry cells) tend to yield a set of slightly different observations.
For a lot of common physical things, the distribution of repeated measurements follows a bell shaped probability curve.
Most things that we repeatedly measure will return slightly different results from observation to observation due to various contributions in an imperfect world.
Above is a plot of the probability distribution of a normally distributed random variable with mean=1 and variance=1 (standard deviation=1). Continue reading WSPR for A/B tests – a discussion – part 1