3PHealth Blog

3P Mobile – The first “P” is for Performance

Monday, August 29th, 2011

This blog post shows you how easy it is now to measure Web performance on your Android Phone. All you need is our Mobile Browser (Sign up here – or just download the App).

Once it’s installed all you have to do is launch the app and navigate to a Web page. (This will give you a summary performance result – for the full set of results you will need to have signed up for the free Web service). Let’s step through some simple screen shots from the Android Emulator to see how it works.

Screen shot 1: Browser 2 is running, a Web page is loaded, and you can see from the red arrow at the top that the device is using GPS.

Screen shot 2: With Browser 2 running, simply click on the Menu key for a pop menu to appear. You’ll notice the “Performance” menu – click on that

Screen shot 3: Click on the Performance menu brings up the following display window. You can select either from the Options or Summary Tab. You know there’s a performance report available by checking in the top right hand corner for a “number”. 1 or higher indicates a report is available

Screen shot 4: Clicking on the “Summary” tab brings up the following screen. Here you can see a quick summary of the Mobile Performance report. To see the full report you will need to have an account at www.3pmobile.com (for more information click here)

Screen shot 5: Assuming you’ve signed up for the full account you will use the Performance Options tab to send the report to 3P Mobile.

You will then be able to see the full performance report including a real time map of where the test was ran. Below you can see the Emulator was set to Rosemead Avenue in Cape Town South Africa

And that’s all there is to it.

Now there’s an app for that – Mobile Performance measured in the palm of your hand.

Measuring Mobile Web Performance – Simulated Vs. Real Time? (That is the Question)

Monday, August 22nd, 2011

I’m going to start this blog with a Quote from Shakespeare…

To be, or not to be, that is the question:
Whether ’tis nobler in the mind to suffer
The slings and arrows of outrageous fortune,
Or to take arms against a sea of troubles
And by opposing end them.

Taking a “little” liberty with this famous soliloquy I’m going to look at measuring Mobile Web performance from a 50,000’ overview and address the thorny question of simulated vs. real time measurement.

The story has to begin with “measure what”? As in, what do you want to measure? Let’s take a quick peak at the OSI model

In a previous blog post “Measuring Mobile Web Performance 101” I talked about “Over the Wire” (the transport layer) vs. the “Application Presentation” layer (the browser). So the question has to be – do you want to measure the network performance or the user-experience?

Up until Smartphones came along I would happily agree (rather “than suffer the slings and arrows) with the simulation experts, that there’s virtually no difference between the packets arriving on the desktop and the browser processing them and displaying the content. If there is a difference, it’s not enough to materially impact the quality of the experience or most company’s bottom lines. Running simulated network tests is good enough for the desktop.

Well that’s all changed now. (Time to suffer the slings and arrows).

When it comes to the Mobile Web there is a difference. And in some cases a pretty big one. Our focus was to look at the user experience and apply our measurement tools squarely from the mobile user’s perspective – inside the browser (The Application/Presentation layer).

In my opinion, as we move to Mobile, not measuring the real user experience is a costly mistake. Mobile Web experience measurement is pushing us to transition from our current Quality of Service(QoS) thinking (simulated networks) to consider the total Quality of Experience (QoE) (real time networks).

What exactly is QoE? Well in short, it’s where the “user consumes IT services”. It’s also where IT and Marketing objectives intersect. That means not only does your network performance matter, but you also have to consider everything else that happens from the time the content leaves your Web Server to when it is displayed on the smartphone screen. The transition from QoS to QoE requires the alignment of IT and Marketing strategies to deliver a compelling mobile Web experience to mobile customers and workers. QoE measurement for the mobile Web is about extending the QoS concept out of the data center and into the real world.

So I ask you – how can a simulation of this possibly work?

Well in short it can’t (more arrows). And here’s why – until you’re measuring inside the browser on a real device, on a real carrier network, anywhere you can connect to the network, then you’ll never know the real experience. There’s simply not enough statistical data out there (as there is for the desktop) to build adequate simulations spanning the range of mobile user experiences.

In the old days, (pre-Mobile), approximation was “good enough”. Not anymore. There are more variables in play, you can’t simulate a real carrier network, you can’t simulate every location or connection and you can’t simulate every device. It will always fall short.

What really counts and what we should all be measuring is what the user – the Consumer – gets to experience/consume the results of what IT and Marketing put together.

And finally what of the browser? Does this factor into Simulated or Real? Fortunately another line from Hamlet comes to the rescue – “ay, there’s the rub”

And the “rub” is a big one. We’ve had years to perfect the browser on the desktop. Routines have been re-written and optimized for new processors, more memory is allocated to load pages faster. This is NOT the case on mobile. It’s still in the opening act.

Compare my desktop activity monitor (525MB allocated to Safari)

To my iPhone… total active memory is 38MB! (and Mobile Safari looks like it’s using about 4.5MB)

To my HTC Android phone (1.72MB – sorry no picture).

The delta between the desktop and Mobile device is huge. The processors are different, the memory is different – in short everything is different.

So desktop simulation now has to give way to a “more precise” measurement. One that includes the Carrier Network, the actual device location, the device’s Operating System version (yes we’ve found that it makes a huge difference).

In my opinion it would be pure folly to insist that simulation of Mobile performance will come anywhere close to really showing us what the Quality of the User Experience (QoE) really is.

If you have a different notion – be sure to send me an email and please include your data. When it comes to your mobile Web customers, the only thing that matters is the experience. If we can’t accurately measure it, we can’t improve it. .

Why Measuring the Real Mobile Web User Experience is so Valuable to your bottom line

Tuesday, August 16th, 2011

Quality of Experience (QoE) is the ultimate barometer of whether or not an IT service is “successful” from a customer/consumer perspective. New application environments such as mobile require new types of metrics to assess not just response time and availability, but effective navigation and interaction with the application and application-supported processes at hand. (source: EMA Research’s Advisory Note: An Adopter’s Guide to User Experience Management – How to Pick the Right QoE Solution for You)

The PC Industry is clearly in transition. We’re now entering the “Post-Desktop” era where Mobile devices will now dominate. Technologies that were designed to measure Web site performance are going to become increasingly irrelevant in this mobile era. Tools that are considered adequate for the desktop user experience are not “precise” enough and lack context when it comes to accurately assessing response time and availability of mobile services.

If the desktop era was defined by Quality of Service then the Mobile era will be defined by the Quality of Experience. A simple slide illustrates the stark differences:

We’ve identified 6 critical “real time” elements to consider when measuring the Quality of the consumers Experience (QoE):

The Carrier Network
The user’s current location
The OS and Browser
The Device’s capabilities
The need for a custom timing framework to further refine the experience
The ability to personalize the Web page for the consumer and then measure its effectiveness

Here’s a sample test that illustrates the first 4 items above.

Methodology:

The test URL was http://m.cnn.com We then used Blaze.io to perform a performance test using an iPhone based in Ottawa, Canada. Next we followed with Firefox on a iMac configured to send an iPhone User Agent (tells CNN that it’s a mobile browser making the request), and we used the Network Link Conditioner utility (part of X-Code on the Mac) to “simulate” a good 3G network and one with a 5% packet loss on the download. Finally we ran two tests on an actual Android device running on Sprint’s network in Castle Rock, Co, and then connected via Wi-Fi to the Internet.

It’s clear from the table below that, in this instance, “simulating” the network, browser and OS all lead to under reporting the actual user experience. (other examples may lead to overstating performance).

Test URL: http://m.cnn.com	# of Requests	Size/KB	Time/Sec	% faster
Blaze.io (link) – Ottawa, Canada (Eastern Time Zone)	25	144	10	–
Firefox Desktop (User Agent = iPhone – 3G Avg. 5% Download Packet loss)	23	67	10	–
Firefox Desktop (User Agent = iPhone – 3G Good. No packet loss)	22	66	8.0	20%
Android HTC on Sprint EVDO_A (3P Mobile: Detail mode, Castle Rock, Co)	26	148	4.0	60%
Android HTC on Sprint Wi-Fi (3P Mobile: Detail mode, Castle Rock, Co)	23	121	2.72	73%

The Bottom line…

It means that new metrics are required to measure the quality of the Mobile user experience. The first three tests represent the old way of doing things. The lack of precision is clear. Moving to a new set of metrics to assess not just response time and availability, but effective navigation and interaction with the application and application-supported processes at hand is now imperative if an IT service is to be considered “successful” from a customer/consumer perspective.

Here’s who benefits from a B2B & B2C perspective, and more importantly how they benefit.

Usability – Key to the Mobile Web Experience

Monday, August 15th, 2011

Jakob Nielson hits it on the head with this Click Z interview, Jakob Nielsen on Usability for Mobile Sites and Apps conducted by Melinda Krueger.

Performance Matters. It is not about short, it is about “ultra short”. Mobile use is secondary to the main activity of the user (shopping, dining, changing a tire, watching TV…). Make it fast and easy or you’ll loose their interest.

Mobile Web has “low commitment”. It’s easy to go to another site. Make what you do on your site count – fast and relevant.

The key is context. Know your user’s device. Know their location. And even better yet, know them! With all of that, you can create a compelling mobile Web experience and keep user coming back with right-sized, relevant content.

Privacy: Do You Know Where your USB Ports Have Been Lately?

Wednesday, August 10th, 2011

You can’t control everything, and let’s face it, data privacy policy management is not the most glamorous part of your day. In fact, if you are not in a regulated industry, there is a pretty good chance you are not even monitoring where your data goes after an employee accesses it, let alone accesses it from a mobile device.

In this NetworkWorld slide presentation “USB Device: The Big Hole in Network Security” you’ll see that of those businesses surveyed, only 50% have an “approved” USB device for employees. There are some great products out there to help with securing transportable data, such as the Encryptx/Imation SecureFlash solution. But how does this help on mobile devices?

Your mobile data privacy policy management depends on knowing what that device (and port) is doing in real-time – while you have the ability to proceed with, or block the download. If it’s going to an approved device, such as a company sponsored smartphone or the Imation SecureFlash drive, let the download proceed. If it is an unapproved, device, like the employee’s personal laptop (via their phone), you can prevent the download from happening.

Remember, mobile is different. You need to have access to some real-time context about the device, the user and their location if you want to effectively extend your data privacy policies to mobile users. So not only can you know where your USB ports have been lately, but also your memory card or any other “attached” device or drive.

A little bit of planning and some real-time mobile context can help you effectively manage the 2nd “P” of mobile Web success – Privacy.