RIS ID
67730
Abstract
One of the main sources of knowledge development in tourism has been survey research. Through surveys of tourists, surveys of the tourism industry and surveys of residents living in tourism destinations, our understanding of the positive and negative impact of tourism has increased and we are now better able to understand and respond to tourists’ needs. The underlying measure in survey research is the question respondents are asked. In the physical sciences measures are highly calibrated and objective. For example, a thermometer – if not faulty – will show the true temperature of a location at any given time. A ruler will measure the true length of an object. This is not the case in the social sciences. Although, as Nunnally and Bernstein (1994, p. 6) state, the “major advantage of measurement is taking the guesswork out of scientific observation”, in survey research it appears that little attention is paid to the actual measure used. Proof for this statement can be derived from a review of all survey-based research studies published in the Journal of Travel Research over the past five years. Thirty-two percent used a seven point answer format and another 33 percent used a five point answer format, most of them not providing a reason for this choice. In the few cases where a reason was provided, it was simply that the answer format was based on previous measures of the construct under study or similar. It appears that the use of five and seven point answer formats has become a habit that, through repetition, has become accepted as the best measure of beliefs. As a consequence, reviewers of tourism journals no longer request a justification for the use of such answer formats. If, however, the authors have good reasons to use an alternative answer format, such as a simple yes-no binary format, reviewers are very sceptical and demand justification at best, or the collection of a new data set at worst. We challenge the currently dominant belief that five and seven point Likert items are the single best measure to capture human beliefs. There are (at least) three good reasons why Likert items in general, and seven point Likert items specifically, can be a dangerous default choice for survey researchers:
- Seven point Likert items suffer from response style bias. Response styles are response biases which respondents display independent of the content of the questions. Response biases are defined as “a systematic tendency to respond to a range of questionnaire items on some basis other than the specific item content (i.e., what the items were designed to measure)” (Paulhus, 1991, p.17). It is widely acknowledged that not only do individuals differ in their response styles, but, more critically, groups of respondents coming from different cultures display different response styles (Chun, Campbell and Yoo, 1974; Hui and Triandis, 1989; Marin, Gamba, and Marin, 1992; Marshall and Lee, 1998; Roster, Rogers and Albaum, 2003; van Herk, Poortinga and Verhallen, 2004; Welkenhuysen-Gybels, Billiet and Cambre, 2003; Zax and Takahashi, 1967).
- Such cross-cultural response styles are a serious problem in tourism research, where often respondents from different cultures are surveyed. Once the data is collected it is hard to differentiate whether, for example, a “7” on the seven point answer format indicates a high level of agreement, or indicates slight agreement plus a response style.
Peabody (1962) concluded that Likert items “primarily represent response sets, and only to a secondary degree actual differences in intensity” (p. 73). He also postulates that Likert items primarily capture direction (positive or negative) and to a much lesser degree intensity (level of agreement or disagreement). It is for this reason that the binary answer format (offering respondents just a “yes” and a “no” option) represents a viable measurement alternative: it captures direction very effectively, but avoids contaminating data with intensity, which is more reflective of response styles than intensity of beliefs. Peabody, along with other authors (e.g. Komorita and Graham, 1965), recommended the use of binary answer formats to avoid response styles many decades ago. But, unfortunately, their voices were not heard by the scientific community.
- Seven point Likert items are less stable than binary answer formats. This is demonstrated by our work, for which we have collected the data and conducted preliminary analysis. The full results will be presented at this conference.
Bendig (1954), Komorita and Graham (1965), Matell and Jacoby (1971a and 1971b), Martin, Fruchter and Martin (1974) and Schutz and Rucker (1975) all conclude from their (mostly empirical studies) that binary answers perform as well as multi-category answer formats on all criteria.
- Both five and seven point Likert items take longer to complete. In studies comparing the time it takes respondents to complete the questionnaire using different answer formats, Jones (1968), Preston and Colman (2000), Dolnicar (2003) and Dolnicar and Grün (2007a and b) conclude that the binary answer format was completed faster than multi-category answer formats, supporting a finding by Preston and Coleman (2002). A number of previous authors have come to similar conclusions. For example, Komorita and Graham (1965) argued that the binary answer format is easier to administer and easier to score. Jones (1968) found that binary answer formats were easier for respondents to complete.
From the work of various pioneers in the area of questionnaire design and our own work, a number of key conclusions can be drawn which have very direct and practical implications for survey researchers in tourism, both in industry and in academia:
- First and foremost: a survey question is a scientific measure and must be treated as such. Its format has to be thought through, justified and pretested before it can be assumed to be a valid measure of the construct under study. The results of this thinking and pretesting process need to be made available to readers of academic articles as well as market research reports.
- Secondly, there is no good explanation for the current dominance of the five and seven point Likert items. While valuable in some instances, simpler, quicker, easier and more valid measures could be used without loss of predictive validity.
Publication Details
This conference paper was originally published as Dolnicar, S, Grun, B, Leisch, F and Rossiter, J, Three good reasons NOT to use five and seven point Likert items, CAUTHE 2011 : 21st CAUTHE National Conference, Adelaide, Australia, 8-11 February 2011.