Measuring Mobile Web Performance: 101
From the end users perspective on a real device, connected to a Carrier network, anywhere in the world.
Well here things get a little more complicated. There are always two different ways to measure User-Agent (the device) performance, whether it’s on the desktop or Over the Air (OTA)/On the device. This translates into two different “categories” of performance testing.
Over the wire:
The first is simply testing to see what happens with any particular application “on the wire”. This includes all the simple “packet capture” style tools that are out there. These kind of test, historically, may or may not help you solve performance problems being seen with any particular User Agent (the device). Just watching some downloads take place doesn’t really tell the “whole story”… especially when it comes to analyzing something as complex as a modern HTTP protocol exchange.
Before you can measure anything on mobile you have to understand the fundamental differences between two categories.
Finally – Any testing MUST include testing the actual Carrier performance on a real device which a user would hold in his or her hands. There’s lots of “back end” fake Mobile sites around – sorry these don’t count as the real user experience. Carrier network performance is NOT the same as a copper connection to the Internet at your desktop. It’s something entirely different.
Finally one more critical item to consider – the devices real time location. Just as the Carrier network effects performance so does the location of the actual device.
Ignoring any of these conditions means that you’re not measuring Mobile performance.