Skip to Main Content
The online home for the publications of the American Statistical Association
653
Views
3
CrossRef citations to date
0
Altmetric
Original Articles

Sliced Designs for Multi-Platform Online Experiments

, &
Pages 387-402 | Received 15 Dec 2017, Accepted 12 Jul 2019, Published online: 30 Sep 2019

Abstract

Multivariate testing is a popular method to improve websites, mobile apps, and email campaigns. A unique aspect of testing in the online space is that it needs to be conducted across multiple platforms such as a desktop and a smartphone. The existing experimental design literature does not offer precise guidance for such a multi-platform context. In this article, we introduce a multi-platform design framework that allows us to measure the effect of the design factors for each platform and the interaction effect of the design factors with platforms. Substantively, the resulting designs are of great importance for testing digital campaigns across platforms. We illustrate this in an empirical email application to maximize engagement for a digital magazine. We introduce a novel “sliced effect hierarchy principle” and develop design criteria to generate factorial designs for multi-platform experiments. To help construct such designs, we prove a theorem that connects the proposed designs to the well-known minimum aberration designs. We find that experimental versions made for one platform should be similar to other platforms. From the standpoint of real-world application, such homogeneous subdesigns are cheaper to implement. To assist practitioners, we provide an algorithm to construct the designs that we propose.

Acknowledgments

The authors thank the Editor, Associate Editor and referees for useful comments and the IMC and Alumni Relations teams at the Wisconsin School of Business for their generous help with the Update Magazine study for this research.

Notes

1 Please refer to Cheng, Steinberg and Sun (Citation1999) for a formal definition of estimation capacity.

Log in via your institution

Log in to Taylor & Francis Online

Article Purchase

  • 24 hours access to article PDF & online version
  • Article PDF can be downloaded
  • Article PDF can be printed
USD 46.00 Add to cart

Issue Purchase

  • 30 days online access to complete issue
  • Article PDFs can be downloaded
  • Article PDFs can be printed
USD 101.00 Add to cart

* Local tax will be added as applicable