The 2023 Mobile Network Test in the United Kingdom
For the eighth time, we – umlaut, part of Accenture, and connect – have conducted our comprehensive benchmark of the UK‘s mobile networks. Its results show a clear winner, two contenders ranking in the good mid-field, and one runner-up with the grade satisfactory.
The carefully designed methodology of our 2023 benchmark in the United Kingdom represents a holistic approach to network benchmarking. It combines drive tests and walk tests for executing detailed voice and data measurements under controlled circum-stances combined with a sophisticated crowdsourcing methodology. The drive tests and walk tests allow for the maximum capabilities of the networks to be evaluated. Crowdsourcing provides profound insights into the overall coverage of voice, data and 5G services as well as realworld User Download Speeds and Latencies. We have thoroughly weighed these components in order to give a realistic and conclusive assessment of the rated networks‘ true potential and performance.
Scope
The 2023 umlaut connect Mobile Network Test in the UK consists of drive tests and walk tests conducted from November 9th to November 26th, 2022. Four drive test cars together covered more than 10,000 kilometres, visiting 19 cities and 40 towns. Additionally,two walk test teams visited ten cities and travelled on trains between them as well as to more remote destinations. The test areas accountfor 17.4 million people, or approx. 26 percent of the total population of the UK. In addition, the results of extensive crowdsourcing analyses, considering 24 weeks from mid-May to early November 2022 are included in the score.
A detailed explanation of our methodology can be foundhere.
DRIVE TEST AND WALK TEST FACTS
17.37
million people covered
10,380
km drive test
204,767
data samples
30,758
voice samples
CROWDSOURCING FACTS
2626
million samples
24
weeks (mid-May to early November 2022)
99.3%
of built-up area covered
99.8%
of population covered
The UK Mobile Operators
With almost 32 million mobile subscribers, O2 is the largest mobile network operator. Formerly a subsidiary of British Telecom, O2 plc was purchased by the Spanish telecommunications company Telefónica in 2006. In 2021, O2 entered a 50:50 joint venture with Liberty Global, combining Liberty‘s brand Virgin Media and O2. The joint company also owns half of the mobile virtual network operator Tesco Mobile which operates on the O2 network in the UK. The operators claim to cover approx. 99 percent of the UK population with 4G. Like the other UK operators, O2 is supporting Voice over LTE (VoLTE) in most of its 4G network. O2 is also continuously rolling out 5G, using its 3500 and 700 MHz spectrum for 5GNR. In late 2022, the company claimed to be live with 5G in around 750 towns and cities, targeting to cover 50 percent of the UK‘s population with 5G in 2023.
With approximately 26 million customers, EE (formerly Everything Everywhere) is the second largest mobile network operator in the UK. Since 2016, EE has been part of the British Telecom Group.EE started offering its 4G service in 2012. Regarding 4G/LTE coverage, EE reports geographic coverage instead of population coverage. They quote Ofcom reporting an 85 per cent 4G geographic coverage which equates to more than 99 percent of the population. EE operates a growing number of “4G+“ cells that support up to 5CA (five carrier frequencies) with up to 1 Gbps under the name “4GEE”. Voice over LTE (VoLTE) is available in most of its 4G network.BT with its brand EE claims to offer 5G in more cities and towns than any other operator in the UK. They announced to cover half of the UK population by early 2023 and the “entire“ UK by 2028.
Vodafone UK is part ofthe international Vodafone Group which is also headquartered in the UK. The Vodafone Group owns and operates networks in 22 countries, with partner networks in 48 further ones.Vodafone UK launched 4G/LTE in 2013. With around 17 million mobile subscribers, Vodafone is the third largest mobile network in the UK.In June 2012, Vodafone and O2 signed a deal to “pool” their network technologies, creating a single national grid of 18,500 transmitter sites. Both operators however announced they would continue to use their own independent spectrum. Vodafone claims to cover more than 99 per-cent of the UK population with 4G/LTE offering up to 1 Gbps – as well as Voice over LTE (VoLTE). At the end of 2022, Vodafone UK reports 5G coveage in 124 UK cities and towns and announced a further speed up of its 5G roll-out during 2023.
Three UK is a subsidiary of Hutchison Whampoa and launched its mobile service in the UK in 2003. As a relatively young operator Three started as a 3G-only network supplemented by 2G via national roaming. In December 2013, Three began to roll out its 4G/LTE service and expanded it rapidly all over the UK. With about 9.5 million customers, Three is the smallest mobile network operator in the UK.The company claims to cover more than 99 percent of the UK‘s population with 3G or 4G. Voice over LTE (VoLTE) is available in most of its network. According to the company, “Three’s customers use 3.5x more data than the average Brit“ – competitive pricing and the availability of unlimited data tariffs will certainly play a part in this. The operator claims to offer more spectrum for its 5G service than any other UK network operator. From 66 towns and cities in February 2020, Three has continuously extended its 5G network coverage.
Results at a Glance
As in the previos years, the BT brand is the winner, this time scoring more than 100 points ahead of the second-ranking Vodafone and achieving the grade “very good“ as the only candidate in this comparison. EE‘s lead is manifested in all three test categories, Voice, Data and Crowdsourcing. Also, compared to our previous Mobile Network Test in the UK in 2021, EE managed to improve its score by 14 points.
Vodafone reaches a good second place, scoring par- ticularly strong in the voice tests. This operator showed the best voice performance in the walktest scenarios, as well as achieving the best reliability and the best voice results in London. Nationwide in the voice tests, Vodafone scored on a par with EE. For an even better score, Vodafone should particulary concentrate on the data download scenarios.
Three achieves a good third rank, not too far behind the second-placed Vodafone. Particularly in the data and Crowdsourcing results, these contenders score very closely together. In comparison to our previous Mobile Network Test in the UK in 2021, Three achieved the biggest score improvement with an advance of 17 score points. In our single city assessments, Three isthe local champion in Liverpool.
The UK‘s largest provider ranks fourth with the overall grade “satisfactory“. It scores on a comparable level in the Crowdsourcing, but loses valuable points both in the voice as well as the data tests. In general, O2 shows a good voice reliability as well as a high level of MultiRAB connectivity in the drive tests performed in the cities, towns and on the roads.
"Congratulations to British Telecom for its brand EE winning our Mobile Network Test in the UK with convincing results in all test disciplines. Vodafone shows good results particularly in the Voice reliability across the country, and Three manages to achieve the highest score improvement compared to our previous test. O2 shows its best results in the crowdsourcing.“
Paul Daniels - Shutterstock
Voice
Owning Your Brand’s Social Community Drives More Shoppers and Sales
EE LEADS IN BIG CITIES VOICE DRIVETESTS In the voice tests, conducted by umlaut‘s test cars while driving in the UK‘s big cities, EE takes the overall lead. Vodafone follows at close distance and is ahead in this category in some cities such as London, Manchester, Liverpool or Leed (see page 10). Three achieves the shortest call setup times in the cities. In terms of speech quality, EE and Vodafone are ahead of Three and O2.
VODAFONE AHEAD IN VOICE IN BIG CITIES WALKTESTS In the walktests, conducted in Belfast, Birmingham, Bradford, Cardiff, Edinburgh, Glasgow, Leeds, Liverpool, London and Manchester, in the overall assessment Vodafone achieves the highest score. Three follows on second place and EE on third, both ranking very close together. Three shows the highest call reliability in the cities walk test. In this category, O2 scores behind by a more distinct margin.
EE ALSO AHEAD IN SMALLER TOWN VOICE DRIVETESTS In the voice tests conducted by umlaut‘s test cars while visiting 40 smaller towns of the UK (see route map on page 1), EE takes the lead and – as in the cities – shows the highest call reliability. Also, in the towns, EE provides the shortest call setup times. Three and O2 follow at some distance, but close together. O2 scores better in this scenario than in the other voice scenarios.
EE SCORING BEST IN VOICE TESTS ON ROADS On the voice tests performed while driving on British roads, EE scores best, with Vodafone and Three showing stable mid-field results. In this scenario, O2 shows the best MultiRAB connectivity behind EE, albeit at a narrow margin ahead of Vodafone and Three. Users benefit from this because apps then still have access to a data connection during phone calls.
VODAFONE SCORES AHEAD OF EE IN VOICE TESTS PERFORMED ON RAILWAYS In the voice tests performed by the test teams while travelling in trains through the UK, all operators fall clearly behind their scores in the other scenarios. Here, Vodafone takes the lead, ahead of EE. The distance of Three and O2 in terms of scoring is pronouncedly more distinct in the railway scenario compared to the other ones.
Owning Your Brand’s Social Community Drives More Shoppers and Sales
EE AHEAD IN BIG CITIES DATA DRIVETESTS In the data drivetests conducted in big UK cities, EE clearly takes the lead. Vodafone and Three follow at a distinct gap in mid-field, ranking closely together. O2 follows at a wider gap. In a more detailed analysis, EE benefits from using 5CA (carrier aggregation), which this operator utilizes above 35 percent in the cities. This can be seen for example in the results of live web-browsing or the playback of Youtube videos.
EE ALSO LEADS IN BIG CITIES DATA WALKTESTS The results of the data drive tests are confirmed by the data walktests conducted in the UK‘s bigger cities. EE again leads the field, Vodafone ranks second – in this category at a minimally wider gap to Three. O2 again takes last place. The latter shows particular room for improvement in the YouTube tests, but ranks second in gaming interactivity behind EE.
EE ALSO LEADS IN DATA DRIVETESTS IN TOWNS The lead of EE in the data drivetests becomes even more pronounced when it comes to the visited smaller towns. Here, the gap to the rest of the field is wider than in the bigger cities. Vodafone and Three score very close together in this assessment, while O2 falls behind even more distinctly. This ranking can be seen particularly clearly in the upload categories.
EE AHEAD IN DATA DRIVETESTS ON THE UK‘S ROADS The data tests performed by our test cars on the UK‘s roads are of particular interest for motorists. In this category, EE once again leads the field, but Three advances to second place. Vodafone and O2 follow at a smaller gap compared to the smaller towns, which is good news to car drivers subscribed to their services.
EE AHEAD IN RAILWAYS DATA TESTS ON OVERALL LOW LEVEL, THREE ON SECOND RANK In the walktests that were specifically conducted on British trains, all operators show some room for improvements – in particular when it comes to data uploads. At an overall low level, EE is ahead in this assessment as well, and Three scores on second place. In terms of data performance on railways, Vodafone and O2 rank close together.
THEE SHOWS PARTICULARLY HIGH 5G DATA RATES 5G is assumed to be the standard in our measurements. But to shed light on the progress of the 5G rollout, we look at the results of the KPI “Data rates of the 7 second Download tests“. This gives a good indication of the data rates which are supported thanks to the 5G technology. But as this assessment does not limit the overall results to the 5G-related aspects of factors such as 5G coverage or the measured latencies of 5G-only connections, we do not identify a separate 5G category winner. That said, in this assessment, Three shows particularly high 5G data rates, both average and maximum,in all scenarios. All operators show a strong 5G penetration in the cities. EE, Vodafone and O2 to some extent use Dynamic Spectrum Sharing (DSS), but the share of samples is not very significant. For Three, the tests did not register this type of connections.
EE LEADS IN TERMS OF BROADBAND COVERAGE,O2 SLIGHTLY AHEAD IN COVERAGE REACH CONSIDERATION From the coverage perspective, although EE and O2 are on a similar level in Coverage Reach, EE offers the best coverage quality to their subscribers as well as the best result in the Time on Broadband (4G and 5G) assessment. Vodafone takes second place in the coverage category overall.
EE LEADS IN PASSIVE DOWNLOAD ANALYSIS In the passively observed download data rates, EE leads the field, with the highest fulfilment rates in all three designated speed classes. In the Basic Internet class (minimum of 2 Mbps) and HD Video class (at least 5 Mbps), the ranking corresponds to the overall result. In the demanding UHD Video class (at least 20 Mbps), Three is ahead of O2 and Vodafone.
EE ALSO AHEAD IN ACTIVE DOWNLOAD ANALYSIS The actively performed download tests are conducted to better approximate the maximum performance of an internet connection. In this metric, EE again takes the lead, with Three following due to higher average and P90 (10 percent faster than) results in comparison to Vodafone. As in almost all of the speed metrics, O2 shows some room for improvement.
EE AHEAD IN ACTIVE UPLOAD TESTS The results of the active Download category are also confirmed by the accompanying upload tests. Thus, EE also achieves the highest sub-score in this category. In the P10 KPI (90 percent of the measured values faster than), Vodafone ist slightly ahead of Three and EE. O2 ranks fourth also in this category.
EE PROVIDES THE SHORTEST LATENCIES, O2 SHOWS BETTER LATENCY RESULTS THAN VODAFONE AND THREE EE shows the best resuts in the latency category, both for the more relaxed OTT Voice class (roundtrip times up to 100 milliseconds) as well as in the more demanding Gaming class (up to 50 ms). However, in both latency KPIs, O2 achieves higher scores than Vodafone and Three.
EE LEADS IN HD VOICE AVAILABILITY, THREE RANKS SECOND BEST IN THIS CATEGORY In the analysis of the availability of HD voice connections (i.e Voice over LTE with the current state of mobile network implementations in the UK), EE takes the first place. In this assessment, Three ranks second best, ahead of O2 and then Vodafone.
EE AHEAD IN CROWDSOURCED ASSESSMENT OF TRANSACTION STABILITY In the Stability category, which looks at the success rates of regular transaction tests, the overall ranking is once more confirmed: EE takes the lead, while Vodafone, Three and O2 follow in this order, with comparatively smaller gaps between their results.
Reliability is not an additional category of our tests, but rather a diffent angle of looking at the results: For each KPI, our scoring distinguishes between “Qualifiers“ (the expected basic performance) and “Differentiators“ (the additional performance that exceeds the expected basics). The view at Reliability limits itself to most of the Qualifiers and the basic KPIs of the crowdsourcing – thus conveying an impression of the standards, a user can reasonably expect from a mobile network. The reference values in this representation are therefore only the subset of score points which we assigned to the Qualifiers. The resulting scores state the reliabilty with which an operator offers its network services. This approach concentrates on the compulsory basics instead of the highest peaks of a network‘s performance.
VODAFONE AND EE ON A PAR IN VOICE RELIABILITY In the overall assessment of the Reliability of voice connections, EE and Vodafone score on a par, with O2 ranking third and Three on fourth place. This is exactly the ranking based on the Qualifier scores achieved in the drivetests. However, in the walktests investigating voice performance, Vodafone is ahead of the field, with EE scoring on second place, Three on third and O2 scoring at the tail end.
EE LEADS IN DATA RELIABILITY Looking at Reliability in the Data tests, EE leads – based both on the results of the walktests as well as on the results of the drivetests. The gap to the runner-ups is distinct. In both aggregations (drivetets and walktests), Vodafone follows on a strong second rank, Three on the third rank, and O2 on the fourth.
EE ALSO AHEAD IN CROWDSOURCING, VODAFONE AND THREE ON A PAR, O2 FOLLOWS AT CLOSE GAP In the crowdsourced KPIs, EE once more takes the lead at a distinct gap over the other contenders. The rest of the field ranks close together. Vodafone and Three score on a par in crowdsourcing, and O2 follows at a small gap of only two score points in this assessment.
EE ALSO AHEAD IN CROWDSOURCING, VODAFONE AND THREE ON A PAR, O2 FOLLOWS AT CLOSE GAP In the the Reliability Assessment of the UK‘s largest cities (see next page), EE is ahead in Birmingham, Manchester and Leeds. Vodafone leads by margin of one score point ahead of EE in London, and by the same margin ahead of Three in Liverpool. In Leeds, Three ranks on the second place behind EE, with Vodafone ranking third at a one-point score gap behind Three.
EE ALSO AHEAD IN RELIABILITY CITY SCORES IN BIRMINGHAM, MANCHESTER AND LEEDS. VODAFONE LEADS IN LONDON AND LIVERPOOL
In the the Reliability Assessment of the UK‘s largest cities (see below), EE is ahead in Birmingham, Manchester and Leeds. Vodafone leads by amargin of one score point ahead of EE in London, and by the same margin ahead of Three in Liverpool. In Leeds, Three ranks on the second place behind EE, with Vodafone ranking third at a one-point score gap behind Three.
Traditionally, umlaut and connect take a closer look at the UK‘s capital to see how the operators cover this lively centre of business, politics and culture. Furthermore, for the inhabitants of the other large cities in the UK, it is interesting to see how the different operators perform in their areas. Therefore, we performed additional analysis also for Birmingham, Manchester, Liverpool and Leeds.They reveal very good results for EE in all cities. Vodafone achieves the grade “very good“ in Liverpool and Leeds, and “good“ in the others. Three is “very good“ in Liverpool, and “good“ in the other cities. O2 receives the grade “good“ in London and Leeds, and “satisfactory“ in the other cities highlighted here.
EE AHEAD IN LONDON, VODAFONE ACHIEVES BEST VOICE RESULT AND BEST RELIABILITY Greater London is by far the most densely populated area in the UK. This also makes the nation‘s capital an especially demanding terrain for deploying and maintaining a mobile network. Although the gaps between the candidates are smaller here, we still see that EE is clearly ahead. Vodafone achieves the best voice result in London, followed by Three.
EE DISTINCTLY AHEAD IN BIRMINGHAM In the West Midlands city, the lead of EE is quite pronounced – the BT brand achieves the highest score in all three sub-categories. Vodafone and Three rank relatively close togehther in the mid-field, with Three achieving a higher score than Vodafone in the crowd-sourcing. The gaps between Vodafone, Three and O2 become particularly clear in the data category.
EE ALSO LEADS IN MANCHESTER In the northwestern industrial city, EE leads overall, ahead of Vodafone, Three and O2, which all have distinct gaps between their scores. In the voice cate-gory, Vodafone is one score point ahead of EE. In the crowdsourcing, Three is ahead of Vodafone by a narrow margin of one score point. O2 ranks fourth, but with a stronger result than for example in Liverpool.
THREE IS LOCAL CHAMPION IN LIVERPOOL, CLOSE AHEAD OF EE AND, AT A MORE DISTINCT GAP, VODAFONE In Liverpool, Three manages to outperform EE in the overall assessment, due to the strongest data results in this city. This makes Liverpool the only city in this com-parison where EE is not leading. Generally, here the gaps between the strongest three providers are smaller than elsewhere. Vodafone is closely ahead in the voice re-sults as well as in the Reliability city score, and EE leads in the crowdsourced assessment.
EE AHEAD IN LEEDS, VODAFONE STONGER IN VOICE HERE In the Yorkshire city, EE leads the field with the best scores in the data and crowdsourcing categories. Vodafone is ahead in the voice category, while the data results show the same ranking as the nationwide result. In the crowdsourcing category, the runner-ups Vodafone, Three and O2 score closely together with only one-point gaps between them.
