Authors
Anish Koppula, Senior Statistical Marketing Analyst, Starcom MediaVest Group
Erich Gundlach, JMP Systems Engineer, SAS erich.gundlach
Creative testing is an integral part of every digital marketing strategy. Evaluation of the performance of media assets is vital so that media dollars can be effectively allocated across the test groups. Today, most marketers rely on simple A/B tests to compare the performance of ad-copies using traditional hypothesis testing. A design of experiments approach provides a more efficient and informative comparison of ad-copies, thus allowing advertisers to effectively execute creative rotation and measure consumer response. This idea can also be extended to web analytics, which requires optimization of a set of elements on a website to provide the best landing page experience, ultimately leading to improved conversions and return on ad investment. This paper proposes and provides examples of: (1) various experimental designs (DOEs) replacing A/B tests, which report on efficiency in execution times using minimum sample test size determinations; and (2) methods for analyzing the DOE data that allow better visualization of the results and more informed decision making compared to methods used in A/B testing. JMP is used at both the design and analysis phases of each example to report on statistical measures aiding decision making, including sample size, power and correlations, along with other practical considerations.
