Review of Dodd’s WSPR based antenna comparison

Dodd espoused the merit of WSPR for antenna comparison in his article (Dodd 2011).

He documented a series of WSPR spots of his transmitter on 20m in a table swapping between antennas during the test period, one side of the table for each antenna. (Don't be misled, the dipole is not half wave dipole but some non-descript multi band loaded dipole.)

He calculates the average for each data set and states:

The average from the dipole and the loop -16.74 and -17.0dB respectively meaning that the performances were very similar.

You might reasonably interpret this to mean that there was no significant difference between the antennas, one was as good as the other.

To review his method, I downloaded archived spot records for the period shown in the table for 14/07/2011.

The first thing that becomes obvious is that he does not use all observations, it seems he has selected stations greater than or equal to 5639km distance though that is not explained in the text. Why was one and only one North American station excluded?

Means (or averages) are a descriptive statistic for a Normal Distribution, the familiar bell shaped distribution that results from so many experiments. There are standard statistical tests for comparing the means of two sample sets to test whether they are significantly different (you cannot simply compare the means). See Student's t-test for more discussion.

So the first question is whether his use of means is valid. The first test of Normality is to plot a frequency histogram and compare it to the familiar bell shaped curve. I have plotted a frequency histogram for all observations in his set since his calculation gives almost equal means.

Above, the frequency distribution is nothing like the bell shaped curve of a Normal Distribution (it is severely skewed for one thing), we do not need to proceed to a Chi squared test to confirm Normality, it plainly is not Normal. A consequence is that no parametric inference can be made based on calculated mean and variance.

It is a poorly designed experiment with inadequate analysis and no inferences can be drawn. His stated findings are not substantiated by the experimental data or his analysis.

Designing and conducting a statistically significant experiment to compare antennas using WSPR is quite a challenge.

References

Dodd, P. Nov 2011. Antennas, tools for ascertaining  antenna performance in Radcom.