Broadband Genie Road Trip 2013 awards: Results Analysis
Editors note: We recently published the outcome of our latest Road Trip, a test of the UK’s mobile broadband providers which aims to put mobile internet under scrutiny in real-world conditions.
The Broadband Genie Road Trip is now in its fifth year and continues to be the toughest and most unique test of mobile data services in the UK.
The train journey is over 4 hours each way and travelling from King’s Cross to Edinburgh covered a big chunk of the UK. Maintaining a connection over that kind of distance, across cities, towns and countryside, is a real challenge for the networks and their dongles.
We also had a bigger team this year. In addition to eight different mobile providers we took the opportunity to test tablets and smartphones. And of course PC Advisor came along for the ride, so also check out their feature on the best mobile networks and the best mobile platform for working on the move, as well as their accompanying Road Trip and smartphone videos which give you an insight into how things played out this year.
We approached each of the UK’s major networks and several Mobile Virtual Network Operators (MVNOs) to get them involved in the road trip, and were supplied with the following devices:
- Vodafone: Huawei K3772-Z (Vodafone K3772)
- Three: Huawei E3256 (Three E3256 Premium Dongle)
- T-Mobile: Huawei E583C (T-Mobile Wireless Pointer)
- EE: Huawei E392
- Samba: ZTE MF190
- O2: Alcatel X230D
- Virgin: Alcatel X230S
- Globalgig: ZTE AC30
Notable for their absence is Orange, who declined the invitation. They appear to be moving away from mobile broadband and directing customers to partner brands T-Mobile and EE. While you can still buy some contract mobile broadband through Orange, trying to get PAYG through their site forwards you to T-Mobile.
We had three MVNOs this time round. Samba and Globalgig both use Three’s network, while Virgin runs off EE.
As with previous years Huawei is the most popular manufacturer for dongle hardware. In most cases the dongles are a slight upgrade on 2012, even if it’s very minor (Virgin previously supplied the Alcatel X220S). Globalgig and T-Mobile were using mobile Wi-Fi units while the other networks gave us USB dongles.
Mobile Broadband Genie speed test
We ran speed tests using our own mobile broadband tool at regular intervals throughout both legs of the journey, recording download and upload results.
Each had a set amount of time to complete, those that could not finish within this limit were marked a failure.
As with both the 2011 and 2012 Road Trip, Three completed more speed tests than any other network, managing all but one of the 13 total. Very close behind was EE with 11 successful, followed by Globalgig (10), T-Mobile (9), O2 and Samba (7), then Vodafone (3) and Virgin (2) bringing up the rear.
Download top speed
EE could have knocked it out the park, being 4G and all, except 4G coverage is still small compared to 3G. Instead it was Three which recorded the fastest download speed with a heady 8.66Mb. We did get the occasional flicker of a 4G signal, though, and EE hit a top speed of 7.11Mb.
Both of these results were significantly better than the remaining networks. Samba Mobile managed 3.33Mb, T-Mobile topped out at 3.2Mb, O2 hit a high of 2.88Mb and Globalgig scraped 1.43Mb. Virgin and Vodafone were disappointing - Virgin’s best result was only 0.34Mb while all Vodafone could muster was 0.26Mb. However, since we were only able to record a couple of speed tests on Vodafone and Virgin there's little we can conclude from their performance.
Virgin and Vodafone aside these top speeds were respectable. EE and Three were obviously excellent, but the 2-3Mb of Samba, T-Mobile and O2 is plenty for the average mobile net user, that’s sufficient to stream HD video and download files without too much hanging around.
The tricky part is maintaining those speeds for long periods, which is where the averages are an important measure.
Download average speed
One or two high scores aren’t much help if your mobile broadband is crawling along the rest of the time, so if you’re weighing up providers the average speed test scores are a good measure of potential performance.
This is where EE really shone. It may not have blown us away with the 20Mb+ promised by 4G, but it had an average of 3.43Mb thanks to numerous 3, 4 or 5Mb results. Not a single one of its successful speed tests fell below 1Mb. Three was runner-up, and did well, but was still about half as fast as EE with 1.87Mb downstream average.
T-Mobile came in third with 1.31Mb and O2 averaged 1.2Mb. The other networks didn’t manage to exceed 1Mb - Samba had an average of 0.97Mb, Globalgig 0.76, then Virgin at just 0.26 with Vodafone coming last at a lowly 0.19Mb.
Upload top speed
EE rocketed to the top of the upload speed rankings with a stonking 8.82Mb, that’s near the performance of a fixed line fibre service and none of the other networks came close on this aspect. In descending order it went Three (2.15Mb), Globalgig (1.61Mb), Samba (1.18Mb), Virgin (0.66Mb), O2 (0.63Mb) and Vodafone (0.48Mb).
Upload average speed
EE’s fast upload result is impressive but it was a one-off, the rest of its scores were a little more modest, though still very strong with several in the 1-2Mb region giving it an overall average of 1.79Mb.
EE was the only network to exceed 1Mb, Three scored 0.94Mb with MVNOs Globalgig and Samba next up with 0.84Mb and 0.74Mb. Virgin actually did okay with 0.51Mb despite only completing two tests, while O2 stumbled in with 0.41Mb. Vodafone once again came last with just 0.23Mb, though that was slightly better than its download average.
All in, this year’s Road Trip speed test results were a mixed bag. EE and Three gave a sterling performance with excellent completion rates and impressive speeds. T-Mobile and O2 also came out of it fairly well, though with fewer successful tests, particularly in O2’s case. Strangely neither Samba nor Globalgig blew us away despite using Three’s network, but it’s worth noting that Globalgig’s success rate was quite strong even if the speed wasn’t stunning.
As for Vodafone and Virgin...we know they are capable of better but they just didn’t seem to be able to hold on to a signal. As you’ll see, disappointment was a recurring theme for the participants using these networks.
Media streaming, downloads and uploads
Speed tests only tell part of the story, it’s also really important to see if mobile broadband services can actually handle the kind of things we’d want to be doing on long train journeys, which is why we also spent a lot of time trying to stream video and music, and download files.
Our tests included viewing videos in various resolutions (360p right up to 1080p HD) from YouTube, Vimeo, iPlayer and 4OD, downloading a selection of files and also uploading to Facebook.
Like the speed tests, each had an allotted time which gave enough wiggle room to account for slow speeds or a loss of connection, while also considering how long you’d really want to sit there waiting for a video to buffer before getting fed up and turning to a book.
All participants were asked to time how long it took to complete each task, or the progress (in percentage, time or megabytes) at the end of the allotted period if they were unable to finish a test.
Streaming video and audio
YouTube videos are a great way to waste time on a long journey, but the unreliable nature of a mobile broadband connection can make streaming video frustrating as the link needs to hold out long enough for the buffer to fill.
All the dongles struggled with video. We had 11 different clips in total, and EE and Globalgig were joint winners in this category, but they achieved that with just five successful streams.
Three and T-Mobile were just behind with four complete, then it was Samba (three), Vodafone and Virgin (one each) and then O2, which didn’t manage to finish a single video test.
Because we asked the participants to record the progress even on failed tests we can also account for videos which came very close to finishing within the allotted time. In Three’s case it had one clip that failed at 99% and another at 89%, so it’s a matter of a few extra seconds to complete the playback. Globalgig also had two which were very near to finishing, while EE had three clips which had gotten to over 60% complete.
The best O2 could manage was 55%, while most of the Virgin and Vodafone videos failed to get more than a few percent into playback before falling over. The only one they completed was a very short clip.
Audio streaming was a little healthier. Every network except T-Mobile managed the first Spotify track, and even then it was very close to finishing. The second attempt didn’t go as well, with only Three, EE and Globalgig finishing, and T-Mobile getting to 72%.
None of the dongles really shone here. If you want to watch video on the move we'd recommend either downloading ahead of time, or bring along a great deal of patience to allow for the buffering!
Downloads and uploads
We tasked our dongles with downloading a variety of files, ranging from a couple of MB up to over 100MB, and attempted uploading pictures and a video to Facebook, for a total of 11 file transfer tests.
Three did well here, completing everything except for one download. EE’s dongle also performed with nine successful, and T-Mobile was right behind with only three failed attempts, followed by Globalgig (seven complete) and Samba (six).
Virgin and O2 couldn’t manage more than one download each, and Vodafone did marginally better with two. None of those three completed a single upload test.
A big problem with downloading/uploading over a mobile broadband connection on the move is that resuming a broken task doesn’t always work. Some servers may allow you to carry on where you left off, but frequently it means starting again from the beginning.
Road Trip 2013: summary
We put the dongles through a total of 37 tasks, including the speed tests. Three managed more than any other network with 30 out of 37, an 81% completion rate, however EE also did very well with 27 out of 37 (72%). Next was Globalgig with 64%, then T-Mobile at 56%, followed by Samba with 45%. The bottom three were O2 (24%), Vodafone (18%) and Virgin (13%).
Consistency and reliability are absolutely vital for mobile broadband which is why we decided on Three as the overall winner of the 2013 Road Trip. While not always the very fastest, the Three dongle held onto its signal and performed far better than most of the competition.
It was a very close match against EE, which provided some outstanding speeds as well as managing to complete a significant number of the tests, gaining it the 'Fastest Network' award. There is the value for money aspect to consider, however, and EE is presently very expensive.
It's interesting to note just how well Three's ultrafast DC-HSDPA network (which it has dubbed '3.9G') did against 4G, proving that there's still life left in the older technology and that you don't necessarily need to be in a rush to upgrade to get the best mobile data performance.
T-Mobile proved to be a good all-round contender with decent speeds and generally strong connectivity.
Globalgig and Samba were reasonable if unspectacular. As with previous years we see differences in the performance of MVNOs on the same network. We’ve queried this with the networks before and been told that they do not prioritise or throttle traffic for partners, so this may simply be due to the equipment. Globalgig uses a decent mobile Wi-Fi unit, while Samba has a cheaper ZTE USB dongle. Both are utilising Three’s network, but Three provided their top of the range ‘Premium Dongle’.
It was a disappointing outcome for O2, Vodafone and Virgin. Virgin was the overall winner in our very first road trip but ever since then has struggled to make much of an impact, despite using the T-Mobile/Orange/EE network.
Vodafone was perhaps the most surprising result. Last year we gave it a ‘Recommended’ award but this year’s outing showed a huge decline in every area. It spent almost the entire journey without a signal and performed very poorly even when it did manage to get a connection.