The Four Levels of Mobile Browser Testing

If there’s one item that doesn’t change from project to project, it’s timelines and budgets. So how do we manage the growing complexity of development and testing within the constraints of time and budget? In this article, I’m going to share some tips that will help you embrace the constraints of your projects while testing in mobile browsers.

Four Questions#anchor

We have sophisticated build tools, automated testing, design systems, and an unending onslaught of acronym-named projects at our disposal, but there’s no replacing manual testing. Planning out your testing strategy is crucial. But first, you need a general understanding of the project. Ask yourself:

• What’s the lifecycle?
  Is this going to be a one-off promotional project that will disappear in a couple months? An evergreen application critical to business success?
• What’s the scale and complexity?
  A small promotional site or a large multi-site system? It’s also a good idea to factor in the inherent complexity of a CMS if one will be used.
• Who is your user?
  Do you have any analytics or insight into your audience and browsing behavior? Ideally, what are your priority devices?
• What’s the timeline and budget?
  This sets the clearest constraints that we need to work within.

Depending on the answers to the questions above, I concede that compromises may be necessary, but you will still need to do some mobile browser testing. There’s no timeline short enough, no site simple enough, and no budget small enough to completely cut testing out of the process altogether. So let’s talk about the four levels of mobile browser testing and how each can fit into your workflow.

Four Levels#anchor

Level 1: Resizing a standard desktop browser#anchor

Trust me. In theory, it should work. Resizing a desktop browser down to mobile sizes is useful as a real-time sanity check during initial development. When you’re still working out your breakpoints and how page components will react to different screen sizes, it’s nice to see how it stacks across them all.

Pros – Quickest and easiest, readily available all the time.

Cons – Not indicative of the mobile user experience, browser bugs, or touch events.

Level 2: Desktop browser mobile emulators#anchor

Trust others; they say it should work. Chrome’s Dev Tools Device Toolbar and Firefox’s Responsive Design Mode are great options here, useful for deeper dives when special consideration is needed for mobile features like touch events. They also have advanced options to throttle connection speeds and simulate mobile device sensors like Geolocation and Device orientation.

Pros – Great for initial checks on basic mobile functionality, constraining to more realistic device screen sizes.

Cons – While better than simple browser resizing, this method still fails to reveal browser/device bugs and overall performance issues.

Level 3: iOS/Android simulator with stock browsers#anchor

Trust me (really this time). With actual device simulators, you’re able to test in a full mobile device environment in the native stock browser. Useful for identifying real mobile browser bugs and for getting a better sense for the page layout within the chrome of the device. Unfortunately, the list of browsers available is limited because you can’t install apps as you would normally on a physical device. On Android, you have access to the Android Browser, the Chromium Content Shell and Firefox. On iOS, you really only have access to mobile Safari. Special shout out to cloud-based emulator providers like BrowserStack and SauceLabs that offer further options that are not available or practical through OOTB simulators.

Pros – Using the actual mobile browser and surrounding chrome, better for finding layout bugs.

Cons – Limited browsers available for use in emulators, often takes a lot of system resources so not realistic to leave open all day, still not indicative of performance and touch optimization.

Level 4: Actual devices#anchor

The real deal. There is no comparison to testing on a physical device in your hands, it’s the only way to test mobile-specific details like touch target sizes and placement of controls. The time needed to test on all the mobile devices you have access to will increase as you continue to add devices. Luckily there are tools like GhostLab and Browser-Sync that can help speed up simultaneous testing on all devices.

Pros – Real performance and user experience of your project on a mobile device

Cons – A queue of devices that cover a reasonable subset of the mobile landscape can be expensive and time to set-up and test is higher.

Wrapping Up#anchor

Mobile browser testing is a delicate balance of the ideal test-everything-everywhere and the practicality of limited time and budget. Frame it in the scope of your project and use the strengths of the various levels defined above to your advantage.