Network Update: November 2009 - Revised January 2010

Update - January 2010

During December, our network engineers completed all planned work to address the network performance issues being seen.

Having monitored the network closely during this time, we're happy that performance has returned to normal.

Our Service Status level has returned to Green. We have also increased capacity on our edge routers, and upgraded both DNS and load balancing platforms at a second data centre in London.

Further work will be happening to add more traffic management hardware to the network. This is to prepare for future demand and won't affect the current network performance.

There were some isolated problems over the Christmas period that may have led to data transfer problems or network imbalance, which were not directly related to last year's problems.

We would like to offer our apologies again for the problems that occurred late last year. If you are still experiencing any difficulties, please raise a fault with our support centre if you haven't already done so.

You can see updates relating to the performance problems in the Service Status Archive.

Original Article - November 2009

During October and in particular over the last two weeks, there has been an increase in the number of our customers reporting problems with the performance of their broadband connections.  This most notably occurs in the evening when the network is busy. This been reported on our discussion forums, and seen in the increased volumes of tickets and calls being handled by our Customer Support Staff and Faults Teams.

We’re sorry that some of our customers have been affected by these problems and we're doing all we can to fix the underlying issues. We remain committed to providing you with the highest quality of service that you have come to expect, and our Networks, Development and Operations teams have been working hard to get to the bottom of the problem.

Those who have been following our regular Service Status updates will already be aware of some of the work that we've been doing but for the benefit of those that aren't in the know, here's a quick overview.

We've been listening closely to our customers over the last few weeks and some of the main problems being reported to us are listed below:

• Slow speed test results, especially when comparing the results across different gateways.
• Poor download speeds from download sites/servers like Akamai & Limelight Networks. This includes access to popular websites such as Apple/iTunes, Amazon, eBay, BBC News and Facebook.
• Lower speeds than expected for lower priority Internet activities like Peer to Peer and binary Usenet.
• 'Page can't be displayed' or DNS errors trying to browse to certain websites or servers, which then seem to work fine seconds/minutes later.
• High latency when gaming (including those on our Pro package).

We've been investigating these issues and have made many network changes over the past month which include, but aren't limited to, the following:

• Increased the amount of bandwidth available on our network.
• Rebalanced our network to ensure customers see more consistent speeds when things are busy.
• Refined our traffic management rules to ensure that customers are getting the best possible experience the network can provide at any given time.
• Identified and resolved a problem with congestion on a number of our core network interfaces.
• Identified and resolved a problem causing unintentional packet loss across our traffic management platform.
• Identified a problem with the gateways that is occasionally causing large numbers of customers to be disconnected from the Internet; this has been reported to our hardware vendor for investigation.
• Identified inefficiencies with our DNS platform and subsequently upgraded the platform to run on new software.

The network is now performing much better and we'll expect to see further improvements as we continue to make changes, working with our suppliers where necessary.

Until we do resolve all the issues there still remains the potential for some slow downs and performance problems during peak hours, especially after 6.00pm. Having said that, web browsing should seem a lot more responsive now than it has been of late.

Finally, don't forget that we'll be continuing to provide regular updates via our Service Status. To make sure you don't miss any of these announcements then why not subscribe to receive updates via email or RSS.

This article was originally published on 20th November 2009, updated 6th January 2010