Detecting Preference Cycles in Forced-Choice Conjoint Experiments

Abstract

Conjoint analysis has become a standard tool for preference elicitation in political science. However, the typical estimand, the Average Marginal Component Effect (AMCE), requires additional theoretical assumptions to be meaningfully interpreted in terms of individual preferences. One crucial assumption is transitivity in preferences. Because the AMCE averages over both direct and indirect comparisons between features, preference cycles can result in positive AMCEs even when, on average, respondents are less likely to choose a profile with one feature over another in a head-to-head comparison. This paper introduces an alternative estimand, the Average Feature Choice Probability (AFCP), which describes the probability of a respondent choosing a profile with one feature against another in a binary forced-choice comparison. It decomposes the AMCE into a weighted average of different AFCPs and describes the necessary conditions under which a positive AMCE implies an AFCP is greater than one half. It develops a statistical test for the presence of preference cycling and illustrates this method with a reanalysis of a conjoint experiment on immigration preferences.