Case Simyo.nl

Improved INP for mobile users (origin P75)

413 ms

150 ms

Improved INP for desktop users (origin P75)

102 ms

52 ms

UX Score Mobile (P75)

82 (Moderate)

100 (Good)

Optimising User Experience: Simyo's Success Story with INP Improvements

Simyo, a mobile virtual network operator (MVNO) in the Netherlands offering prepaid and postpaid mobile phone services on the KPN network, has been focusing on the user experience (UX) of its website for more than 10 years.

Simyo has been voted the best mobile provider 32 times by the Consumentenbond (consumer association in the Netherlands). Furthermore, their customers rate them highly, with an amazing score of 9.8 out of 10 for overall satisfaction.

Simyo has always believed that the website is part of this customer experience which is why they have been working hard to make sure that their site is as fast as it can be. Around ten years ago, Simyo focused on making sure their pages loaded quickly, because Simyo is convinced that a page that loads quickly is an important aspect to offer their customers the best digital experience. In those early years, the focus was on metrics like Start Render, First Contentful Paint and Speed Index. These were user-centric metrics that Simyo could easily and visually relate to its customers' experiences. The objective was to achieve fast page rendering.

With the introduction of the Core Web Vitals, including Largest Contentful Paint (LCP), Cumulative Layout Shifts (CLS) and First Input Delay (FID), the goal was to comply with the Core Web Vital assessment. The website had to perform well on all UX aspects, including loading speed, visual stability and responsiveness.

Challenge

Like many other sites, Simyo did really well on First Input Delay. However, based on synthetic data, it was thought that responsiveness could be better, especially for mobile visitors. Although Simyo only had synthetic data on initial load, it was suspected the high scores for metrics like Total Blocking Time (TBT) and JS Long Tasks were affecting post-load interactions.

Based on collected synthetic data we were able to gain some valuable technical insights. It seemed that the main cause of these INP issues was the use of third-party tooling and tracking. This is probably related to the high level of JavaScript used by third-party vendors. However, we weren't 100% sure about this because we didn't have any data from post-load interactions. Our research, optimisation and removal of third-party resources didn't fully solve the responsiveness problem because it was based on synthetic data for the initial load only.

The introduction of the Interaction to Next Paint (INP) metric as a potential alternative to First Input Delay as a responsiveness metric has proven to be a valuable insight. Initial suspicions were confirmed by the Chrome User Experience Report (CrUX): the responsiveness of page interactions for mobile users was found to be below expectations. With this data, we also knew which pages had the highest INP scores.

The biggest challenge was to work out which specific interactions were delayed and by what causes.

Solution

The CrUX API provided insight into the INP values of the origin, page types and specific URLs. All pages were also monitored using RUM tooling, but INP measurement was not yet available. More specific insight was needed and this was found by using RUMvision, a RUM tool that makes use of the Google Web Vitals library. Not only was RUMvision able to monitor INP per page, the ability to provide attribution data using the Google Web Vitals library allowed Iron/Out to quickly identify which interactions were causing delays for mobile users.

While learning about specific HTML elements and INP breakdowns provides very valuable insight, the introduction of the Long Animation Frames (LoAF) API in Chrome made a significant difference. The Long Animation Frames API is a relatively new tool that helps developers identify and fix performance issues caused by long animations. According to Google, LoAF helps provide a better understanding of slow user interface (UI) updates. It gives insight into the duration of animations and their impact on page responsiveness. This can be a useful tool to identify slow animation frames which may affect the Interaction to Next Paint (INP) metric. It can also be used to identify other UI jank which affects smoothness.

Curious how we can improve your performance?

Check the performance of your website using the Core Web Vitals and Lighthouse performance reports.

At the end of 2023, LoAF was only available in Chrome as an Origin Trial. However, RUMvision was already using the LoAF API while in Origin Trial, so we could already make use of it. RUMvision offers a third-party dashboard that shows the "pure JS execution time" of first- and third-party scripts. With this data in hand, we were able to pinpoint the source of performance bottlenecks. 

Example of the third-party dashboard in RUMvision

It became evident that the responsiveness issues were caused by a global first-party resource. Based on the analysis of performance traces conducted by running performance profiles in Chrome Devtools, it emerged that this resource was pushing data to Google Tag Manager (GTM) when a user was interacting with elements on a page. While it is relevant to analyse how visitors use the site, this is not critical for the visitor during interactions. Visitors expect an immediate response in the form of a visual update as they interact with the elements on the page.

In collaboration with developers at Simyo, we provided and applied solutions to prioritise UI updates within the browser's main thread and only sending data to GTM when the UI has been updated. This ensures that the user experience meets visitor expectations.

Results

Following the implementation of these optimisations, we achieved a significant improvement in the INP for mobile traffic, with more than 75% of users experiencing an INP of less than 200 ms. This resulted in a notable reduction in the INP for the origin, from 413 ms to 150 ms, representing a 64% improvement.

Percentiles (Chrome UX Report)

Distributions (Chrome UX Report)

The experience for desktop users also saw a notable improvement of 49%, with INP dropping from 102 ms to just 52 ms.

In our communication with clients, we report a UX score for pages and origins. This score is based on the key factors that matter to website visitors. It is easy to understand and provides insight into whether the site is fast and works well for everyone. The UX score enables IT staff and business leaders to make informed decisions that enhance the user experience of their websites. 

This improvement in INP has led to the origin achieving a UX Score of 100 points, reflecting the enhanced user experience for mobile users. A total score of 100 indicates excellence across loading speed, visual stability, and responsiveness.

While the numbers speak for themselves and show that the user experience has improved, the videos below show you how the interaction on a ‘handset’ page (product detail page) has improved significantly. In both videos you can see that a user selects different colours for the handset. After tapping on a colour, this choice is confirmed in the colour picker and the image of the device is rendered in the corresponding colour. Before the changes, there was a noticeable lag when updating the UI, but after the optimisations, the experience is so much smoother.

Before optimisations had been implemented

After optimisations have been implemented

Both tests were carried out on the exact same device, a budget Android smartphone (Samsung Galaxy A40) with limited processor power, making the chances of INP issues relatively high. However, we know that many other users visit the site with devices using the same or similarly less powerful processors.

Conclusion

In conclusion, Simyo's dedication to improving the user experience on their website has been evident through their focus on speed and responsiveness. By identifying and addressing issues with delays in page interactions, particularly for mobile users, they were able to significantly enhance the overall user experience. The implementation of solutions led to a remarkable improvement in key metrics such as Interaction to Next Paint (INP), resulting in a UX Score of 100 points for mobile users. With these improvements, Simyo has demonstrated their commitment to providing an excellent digital experience for their customers, further solidifying their reputation as one of the best mobile providers in the Netherlands.