The first 21 days of Exetel broadband Internet access

We have had wired broadband service delivered to these premises for almost ten years, supplied by three vendors: Telstra Bigpond, iiNet, Amaysim, Southern Phone and now Exetel.

At The first seven days of Exetel broadband Internet access I discussed the performance failure of the service observed over the first seven days.

Following that, a fault report was submitted to Exetel, and at their request, further observation of speed and ping test latency.

Service levels

I have an expectation that “I want it to deliver most of the rated speed, most of the time during the hours that I want to use it”.

As a result of behaviour of the industry, the ACCC gives some guidance on terms used to advertise a service, and a service expectation.

Essentially they say:

Standard Plus Evening Speed—plans using this label will deliver a minimum speed of 30Mbps during the busy period. This plan would be suitable for a higher usage profile (e.g. streaming an ultra-high definition movie and streaming music on one or more other device during the busy period)

So on that measure, how well did they perform?

Above are the results of file transfer tests conducted automatically, the above are filtered for those in the ACCC’s defined “evening hours” on which they base their service level enforcement. All x symbols in the pink area break the 30Mb/s minimum. Continue reading The first 21 days of Exetel broadband Internet access

The first seven days of Exetel broadband Internet access

We have had wired broadband service delivered to these premises for almost ten years, supplied by three vendors: Telstra Bigpond, iiNet, Amaysim, Southern Phone and now Exetel.

The change from Amaysim was at Amaysim’s choice, they decided to quit the wired broadband business stating that they could not make a profit, and they sold their customer base to Southern Phone.

Southern Phone was unable to deliver the VOIP service we have had for many years through all of those broadband suppliers, so we churned to an Exetel “Standard Plus” service.

Cutover

The cutover was badly coordinated, there was a 12 hour outage, and I had to phone Exetel (yes, wait in the interminable queue to try to get the modem PPP credentials, it had DSL service and was routed to Exetel, just needed the authorisation details for Exetel).

Telecommunications providers are all about strong procedures for handling usual business, and this part of the business is known as provisioning. The events with Exetel hint that their provisioning processes are not sensitive to churning an existing NBN connected service which should be done to minimise disruption, including advising the logon credentials, doing it when their support desk is open, holding if the customer needs a new modem etc. This is the bread and butter of the business, but a FAIL to Exetel on this occasion.

First impressions are lasting ones… so how do they come back from that start?

Service levels

I have an expectation that “I want it to deliver most of the rated speed, most of the time during the hours that I want to use it”.

As a result of behaviour of the industry, the ACCC gives some guidance on terms used to advertise a service, and a service expectation.

Essentially they say:

Standard Plus Evening Speed—plans using this label will deliver a minimum speed of 30Mbps during the busy period. This plan would be suitable for a higher usage profile (e.g. streaming an ultra-high definition movie and streaming music on one or more other device during the busy period)

 

So on that measure, how well did they perform?

Above are the results of file transfer tests conducted automatically, the above are filtered for those in the ACCC’s defined “evening hours” on which they base their service level enforcement. All + symbols in the pink area break the 30Mb/s minimum. Continue reading The first seven days of Exetel broadband Internet access

Yet another change of Internet broadband access

We have had wired broadband service delivered to these premises for almost ten years, supplied by three vendors: Telstra Bigpond, iiNet, Amaysim, Southern Phone and now Exetel.

Download speed

Download speed is an important performance characteristic, and fairly quantifying it is a challenge.

If I were to express my service expectation as “I want it to deliver most of the rated speed, most of the time during the hours that I want to use it” we can express that by better defining the three terms:

  • most of the rated speed;
  • most of the time; and
  • the hours that I want to use it.

Most of the rated speed

Let’s consider that most means 80%, so the critical value of speed is 80% of the rated speed, eg 80% of 12Mb/s is 9.6Mb/s.

Most of the time

Let’s consider that most means 80%, so the critical value is 80% of the time.

During the hours that I want to use it

I sleep at nights, and use the Internet variously during the day, the most important times for service to be adequate are from 06:00 to 20:00.

Critical speed requirement

Capturing the foregoing, I could write “Download speed must exceed 80% of the rated speed, 80% of the time between the hours of 06:00 and 20:00” each week.

This is known as a Service Level specification.

Statistically, this is stated as the 20 percentile speed measured between 06:00 and 22:00 over a week must be greater than 80% of the rated speed.

Test results

A realistic simple HTTP download is scheduled every half hour, and the effective download speed is recorded. This data has been gathered for many years.

Over the nearly 10 years of service, the rated speed has varied, 8, 12, and 50Mb/s at different times. To compare these, the speed in any one observation needs to be normalised to the applicable rated speed, so for this purpose speed is calculated as a percentage of the applicable rated speed.

Telstra

Above is a plot of the measured speed over a period of many years. Continue reading Yet another change of Internet broadband access

First full month of Amaysim broadband broadband Internet access

We have had wired broadband service delivered to these premises for almost ten years, supplied by three vendors: Telstra Bigpond, iiNet, and Amaysim.

Amaysim has supplied a 25Mb/s  broadband service for a month now and set an important baseline for service.

Above is a plot of the 90 percentile and 10 percentiles for each week so far. Continue reading First full month of Amaysim broadband broadband Internet access

First full week of Amaysim broadband Internet access

iiNet

Review of 10 months of iiNet broadband Internet access detailed a fairly disappointing performance record of our first NBN based service, a 12/1Mb/s FTTN service from iiNet.

With a perception that online performance had degraded and was unsatisfactory, the results of an automated half hourly file download were consulted to verify those perceptions.

Above is a plot of the 90 percentile and 10 percentiles for each week. Continue reading First full week of Amaysim broadband Internet access

Review of 10 months of iiNet broadband Internet access

On 5/10/2016 we cut over to a new broadband Internet access service, switching from Telstra 8Mb/s ADSL to iiNet NBN 12Mb/s.

Over some years, I have run an automated file transfer to measure the speed of our access service. The tests are done between 6:00 and 22:00, I am interested in performance during the times I want to use the service, and less interested in times when I would usually be sleeping.

One of the performance measures is speed, it seems simple enough especially when it is talked about by politicians and promoted by RSPs and NBN’s advertisements.

So, how does one characterise the speed. People like to think of simple concepts like ‘average’, but let us look at the distribution of speeds.

Above is a chart of the frequency distribution of speed observations for a week in March 2017. It is not in the form of the Normal Distribution, the classic bell-shaped curve for which common parametric statistics like Mean (average) and Standard Distribution are meaningful, but it is a skewed distribution. Without knowing the characteristics of the distribution, it is a misuse of parametric statistics to apply parametric statistics like Mean (average) and Standard Distribution. Continue reading Review of 10 months of iiNet broadband Internet access

iiNet broadband Internet access – speed observations

I have for many years measured broadband Internet access service performance by measuring the transfer rate for a single HTTP download which is scheduled regularly.

Since moving to iiNet / NBN about 6 months ago, I have had difficulty reconciling apparent workstation performance with the measured download speed.

During investigation of variation in download speed measurements, it became apparent that in the absences of other network traffic, the first download was faster than further downloads within a short space of time.

The above chart shows the download speed on 10 downloads, each download ran for 5s and there was a 2s pause after completion before the next download was started. The file downloaded is one nominated by iiNet for download speed tests and is internal to their network.

The first transfer in this cluster ran at 5.8Mb/s (about half of the ‘headline’ access service rate). The following nine were between 0.9 and 3.1Mb/s, averaging 1.9Mb/s.

This behaviour is reliably reproduced on subsequent tests. It may be a result of the MPLS network, it may be some clever management of what you see vs what you get.

To better capture network performance, I have changed by measurement strategy to perform a 2s download, then wait 2s after it is completed, then perform a measured 5s download to calculate a more realistic download speed.

Above is a plot of a day’s tests using the adjusted algorithm. The median speed is 3.9Mb/s, less than a third of the ‘headline’ rate.

Little wonder that we need intervention as announced by the competition regulator, the industry cannot regulate itself effectively.

Review of 5 months of iiNet broadband Internet access

On 5/10/2016 we cut over to a new broadband Internet access service, switching from Telstra 8Mb/s ADSL to iiNet NBN 12Mb/s.

Over some years, I have run an automated file transfer to measure the speed of our access service. The tests are done between 6:00 and 22:00, I am interested in performance during the times I want to use the service, and less interested in times when I would usually be sleeping.

Above is a graph of yesterday’s performance. Note that the service does not delivery anything like the 12Mb/s description of the service, and it collapses in the evenings when IP television demands exceed the network’s capacity.

Above is a chart of the tests over the previous week… the collapse in the evening is common. Continue reading Review of 5 months of iiNet broadband Internet access

Queueing in data networks

Modern data networks route or switch relatively small ‘packets’ of data across shared links that along with the switching nodes, form the wider data network.

One of the roles of the switches is to receive packets on one link, and send them onwards on the most appropriate link. Since links may be at different speeds, and many links may source packets to be sent on any link, there exists a mechanism in the switch to store packets pending transmission, in the simplest case it is a first come first served link queue.

The function of the queue then is to hold packets until they can be sent on the link, and to offer them in first come first served order. That raises two important questions:

  • how long will packets be delayed;
  • how many slots does the queue need.

Queuing theory gives us a method of estimating these quantities.

Lets make some assumptions about the traffic:

  • service requests arrive randomly in time; and
  • service time is exponentially distributed with an average time of 1.

screenshot-19_10_16-08_25_32

Above is a plot of normalised average response time (service time + queue wait) vs resource (link) utilisation (pu means per unit). It can be seen that when the link utilisation is 0.5pu (50% busy), that response time is 2pu (ie twice the average service time), twice that needed to send an average packet at very low utilisation. Response time rapidly degrades:

  • at 70% link busy, response time is 3.3 times packet transmission time; and
  • at 90% link busy, response time is 10 times packet transmission time.

To ensure packets are not discarded, the queue need sufficient slots to hold packets even in most peak bursts. Whilst at 70% link utilisation, the average queue size is 2.3 slots (3.3-1), a larger queue size accommodates bursts better. Discarded packets can severely affect performance, not only are they likely to be resent after some delay and network overhead, they can break a higher level protocol unit in simple systems and waste the link capacity and other links used to send the rest of that protocol unit. Continue reading Queueing in data networks