A/B Testing

One day during a coffee break, a fellow designer asked me what’s my secret sauce in A/B testing. My answer was this: a whole lot of researching up front, which helps yield a whole lot of great ideas, which ends up with many low hanging fruits: tests with easy small modifications leading to big behaviour changes.

Disclaimer: As a promise to my friends Mickey & Donald, I've intentionally left out many details on this project.

The art of stealth

A/B tests and multivariate tests are like traps set up by a team of ninjas with ancient art of stealth techniques. Some of these ninjas are developers who build the tests. Some are yield optimization analysts—ninjas that mastermind and analyze the tests. And some, like myself, are designers—we conduct research, uncover opportunities, explore design options, and create design blueprints for the tests.

Enter the mysterious land of apps

While Disney has been ruling the land of world wide web— mobile apps testing on the other hand, just got in scope. I was fortunate enough to become a half-time ninja while I continued to work on other features. Our Testing and Optimization team is centered in Glendale and Orlando, which made me the one and only Seattleite-Ninja, bringing my mobile app speciality into the sacred clan.

Mobile Ordering

What type of tests I’ve worked on in the clan you’ve asked? Take this really awesome mobile ordering feature in the app test for example. Mobile Food and Beverage Ordering allows guests like you to order food from restaurants wherever you are, allowing you to skip the ordering line. While this feature has been favorited by manys since launch, one opportunity we saw was to increase the presences and awareness of this feature within the app. I applied my secret sauce: conducting lots of research, which leads to lots of ideas, which helps narrow down to easy wins options.

Unfortunately, I cannot go any further in detail regarding this secretive test, or I will be chased down by a thousand angry ninjas with mickey-shaped throwing stars.

Final Thoughts

Being the only Seattleite-Ninja, working remotely requires a lot of other special ninja skills: The ability to communicate quickly, directly and on topic; the positive attitude to handle stressful situations; the mind of a doer that create plans, acknowledge important details and start working effectively.