On 5/10/2016 we cut over to a new Internet access service, switching from Telstra 8Mb/s ADSL to iiNet NBN 12Mb/s.
Over some years, I have run an automated file transfer to measure the speed of our access service. The tests are done between 6:00 and 22:00, I am interested in performance during the times I want to use the service, and less interested in times when I would usually be sleeping.
Above is a graph of yesterday’s performance. Note that the service does not delivery anything like the 12Mb/s description of the service, and it collapses in the evenings when IP television demands exceed the network’s capacity.
Above is a chart of the tests over the previous week… the collapse in the evening is common.
So, how does one characterise the speed. People like to think of simple concepts like ‘average’, but let us look at the distribution of speeds.
Above is a chart of the frequency distribution of speed observations for last week. It is not in the form of the Normal Distribution, the classic bell-shaped curve for which common parametric statistics like Mean (average) and Standard Distribution are meaningful, but it is a skewed distribution. Without knowing the characteristics of the distribution, it is a misuse of parametric statistics to apply parametric statistics like Mean (average) and Standard Distribution.
So we must rely on non-parametric statistics.
A simple statistic is the median, the “middle” number of a set of ordered observations. 50% of the observations will be above the median and 50% below.
That concept can be applied to give the measure that was exceeded or not by any given percentage of observations. A percentile is a measure used in statistics indicating the value below which a given percentage of observations in a group of observations fall.
Above is a chart of the 90 percentile rank and 10 percentile rank of the services over some years.
A way to interpret the percentile is that the 10 percentile rank is the speed that is delivered 90% of the time, and it you think it is fair that the supplier should deliver the ‘rated’ speed most of the time, 90% of the time, the 10 percentile rank is a key measure.
At the start of the period, Telstra delivered a 90 percentile rank of about 70% of the service speed. That collapsed in the latter half of 2012 when they promoted T-box (IP TV) and closed the mail movie service. IPTV is a higher revenue earner. Service slowly degraded to the point that 10% of the time, transfers in a week ran below 0.5Mb/s (6% of the service speed).
Notwithstanding that Telstra said they could not improve the service, and why did they need to as ACCC had advised “that since speed was no longer stated on the bill, it was not an enforceable element of contract”, they did make some improvement about the time NBN fibre was being installed.
The interesting thing about the period from 05/2015 to 09/2016 was how close the 10 percentile rank is to the 90 percentile rank. The service now seems to have capacity to withstand peak usage by all users, albeit artificially capped at 2Mb/s.
You might ask the question, are they playing with us, turning service up and down at will, and imposing artificial constraints on speed at a level well below their original description of the service.
Above are the observations over 26 weeks. The first weeks are the Telstra service and from 5/10/2016 the new iiNet NBN based 12Mb/s service.
Note the clear stepwise drop in performance for the first two weeks of November. Service is restored to the October levels until almost three weeks in February runs 40% faster, and less variation from test to test. The October performance is again restored and that has continued until this day.
The two weeks of greatly improved performance in February questions whether the service at other times falls short of what an NBN AVC TC4 access service should deliver.
In answer to the question posed earlier, they are playing with us.
Since the ACCC took an interest in the matter, the industry (RSPs and NBN are pussy-footing around softening the meaning of speed.
Above is an extract from nbnCo’s price book of the services that RSPs buy from them to deliver a service like this. The AVC TC4 access service rates are indexed on “Downstream Mbps”. Note this is not the only component of charge levied by nbnCo for the service, but this speed rated component alone accounts for more than half of the consumer’s monthly charge on basic plans (eg dodo’s $39.90/month).
The speed of an access service is an important parameter of service level, not the only one, but if not the most important, a very important parameter. Services are usually sold with the mention of a speed rating, though with recent ACCC positioning, speed is couched in some terms to try to make it a nebulous, unmeasurable, unenforceable quantity. The fact is that everyone from Prime Ministers down has talked Mb/s since the inception of NBN, and nbnCo uses the figure as an index of chargeable access services.
That being the case, consumers might expect that when they buy a service in a competitive market, it should have a speed rating that can be compared with the tariff to assess value for their own needs, and they should expect that most of the time that they want to use the service, it delivers the rated speed.
“Most of the time that they want to use the service, it delivers the rated speed” could be defined to mean that the 90 percentile rank of speed measured between 6:00 and 22:00 over a week is the determined delivered speed. In that context, the determined delivered speed of iiNet’s 12Mb/s service was 2.9Mb/s, just 24% of the “rated speed”.
It is time for the ACCC to act to protect consumers from this sort of misleading and deceptive product promotion and delivery whilst encouraging an efficient competitive market.