“Users care if their video conference, gaming, or video stream works. They usually don’t care about ping times or their speed test results,” Domos Head of Research Bjorn Ivar Teigen Monclair said. During the presentation, Monclair pointed out that quality is emerging as a selling point, with broadband service providers beginning to communicate the benefits of lower latency to customers.
Network quality changes frequently and radically, Monclair advised. This could be due to users moving between rooms, a neighbour downloading a big file, or even a microwave being switched on in the house. 
It is also easy to trick yourself into thinking that you have a good network connection and reliable user experience based on one good measurement. But network quality is notoriously a difficult measurement to track. It should be considered a moving target, Monclair stated, as the network measurements can change very quickly.
Asking the user about their experience is one option to measure network quality. But this is expensive and difficult for BSPs. And it can be difficult because it is a subjective measure. Two people may have differing opinions of the same network connection. The likes of gamers or live video or music streamers may notice the smallest changes in latency and network quality. While others may be used to working on a choppy network connection and might just be pleased that it does not break up completely.
In contrast, Application Outcomes are measurable over time and will correlate strongly with the user experience. Applications behave roughly the same way as long as the network does not change. A Quality of Outcome (QoO) approach bridges the gap between Quality of Experience (QoE) and Quality of Service (QoS). For example, in this context, QoE would be asking the customer about their experience. Whereas QoS is measuring bandwidth, latency, and packet loss.
QoO creates value for BSPs, allowing them to better understand their customers and what they are delivering to them. “We want to establish a network quality score that users can understand, so they can judge the service they receive from their BSP,” Monclair explained. “A network quality score should be something the user can trust, cares about, and is honest and objective.
The QoO metric builds on Broadband Forum’s TR-452 specification and its mathematical framework for network quality. In TR-452, network quality is defined as how latency distributes at different loads. The framework can advise if different applications have different network needs, like lower latency for gaming rather than video streaming, for example. QoO has the inherent ability to be probabilistic, acting almost like a weather forecast, i.e. we expect and estimate the application to behave a certain way.
It was highlighted that BSPs and vendors need more in-depth information and root cause analysis into the measurements. TR-452’s composability helps establish which part of the end-to-end network chain is not delivering as it should and affecting the application’s performance.
As part of a joint Domos and Comcast trial, 25 routers were sent to customers with the network quality measured using QoO. The feedback was very positive, with 100% advising that the tool was better at showing network quality than a speed test and 86% felt that QoO accurately reflects experience. Monclair calls for more companies to get involved in the work.