Why quantitative papers based on primary data get desk-rejected by Annals of Tourism Research

 

Annals receives over 1100 manuscripts annually–60% are desk-rejected. This viewpoint explains why this is the case for quantitative papers based on primary data.

Problematic research design

Invalid measures of behaviour

Many researchers want to understand the behaviour of tourists, managers, workers, or policy makers. Yet, they limit themselves to measuring self-reported behavioural intentions or past behaviours. Such measures fail to accurately reflect behaviours, especially those viewed as socially expected (e.g. environmentally sustainable behaviour can be over-estimated by a factor of four; Karlsson & Dolnicar, 2016). If self-report measures are unavoidable, their use must be justified, and implications for the validity of conclusions must be explained.

Causal conclusions from non-causal research designs

The only research design that allows causal conclusions is the experiment. Experiments can be implemented in a laboratory, in the field, or as a survey with an intervention, as long as the researcher controls the presence/absence of the hypothesized cause (Viglia & Dolnicar, 2020). Correlation-based methods (including structural equation modelling) applied to one-off cross-sectional survey data without intervention provide insights into associations, not causes.

Problematic survey answer formats

Five- or seven-point multi-category ordinal answer formats – often incorrectly called Likert scales (Likert, 1932) – lack metric properties (limiting data analysis; Kampen & Swyngedouw, 2000); are prone to response biases (Paulhus, 1991), including cross-cultural response bias; and generate unstable responses (Dolnicar & Grün, 2013). No single optimal survey answer format exists, but many options are available, ranging from forced-choice full binary (yes-no) formats to metric slider scales (see Dolnicar, 2013; Dolnicar & Grün, 2013). For each question, researchers must choose the response options that minimize bias, maximize permissible mathematical procedures, and enable respondents to express themselves. Pre-tests reveal people’s natural responses to questions; suitable options can be tested in a talk-aloud setting.

Adapted scales are problematic because the unadapted scale is validated. Unavoidable adaptations must be explained and justified. Changing the context is least problematic. Changing item wording, removing or adding items alters the scale fundamentally.

Convenience sampling is convenient, not random. It does not generate representative samples. If unavoidable, a strong justification must be provided.

 

Figure 1. Main reasons for desk-rejection of quantitative manuscripts based on primary data (based on 63 decisions, 17.4.-29.5.2020)

Uncritical use of scale development paradigms

Churchill introduced psychometrics to marketing. Retrospectively, he concluded his approach has “become a rote process…drill…lending legitimacy to…a…thoughtless, rather than thoughtful, effort” (1998, p.30). Factor loadings, Cronbach alphas, and other statistical indicators cannot prove the quality of an empirical measure. Content validity must be assessed by experts (Rossiter, 2011). Authors can demonstrate that their questions measure the construct under study by making survey questions accessible to readers. Statistical criteria can support, but not substitute logical assessment of validity.

Text duplication

Elsevier routinely checks for text duplication. Copyright legislation prevents the publication of  manuscripts containing text blocks from not-compliant sources (Dolnicar, 2019). Duplication with abstracts, theses, and electronic preprints is permissible. Running the manuscript  through plagiarism software before submitting helps prevent desk-rejections.

Lack of contribution to knowledge

Annals aspires to publish big, bold, new ideas, generating substantive new knowledge. Annals does not publish articles contributing only marginally (e.g. another study associating tourist satisfaction, loyalty and intention to return).

Lack of contribution can result from applied studies or scale development

Annals does not publish pure scale development papers. Survey questions and sets of survey questions (scales) must relate closely to the research question. They are best reported in the Methodology section. Scales are not  contributions to knowledge in their own right.

Applied research starts with the “practical need for making an informed decision” (Lens, 1987, p.454). Applied research guides action by decision makers in specific real-world contexts. Basic research starts with the desire to know “why things are and how they work the way they do” (Lens, 1987, p.453). Basic research can have practical implications, and applied research can lead to knowledge discovery, but these are unintended side-effects.
Figure 2 illustrates this differentiation. Content published by Annals is marked in green.

 

Figure 2. Applied versus basic research (the scope of Annals in green)

Not suitable to send out for review

Some manuscripts are poorly presented. Others use highly technical language, making content inaccessible to the diverse Annals readership, which covers a broad social science spectrum. Other manuscripts again do not study tourism.

Scholarly self-reflection

Scientific communities tend to self-reflect: Are male authors cited more? What does a bibliographical analysis say about knowledge production in a field? Why are quantitative papers desk-rejected by Annals? These important questions are best discussed in opinion pieces, not in research papers.

Ignoring key developments

Some events (e.g., COVID-19) or developments (e.g., the emergence of Airbnb) disrupt business as usual in tourism and hospitality significantly, affecting many research topics. Studies investigating such topics cannot ignore such events and developments.

References

Churchill, G.A. (1998). Measurement in marketing: time to refocus? In: Hess, J.D. and Monroe, K.B. (Eds.) Paul D. Converse Symposium. Chicago: AMA, pp. 25–41.

Dolnicar, S. (2013). Asking good survey questions. Journal of Travel Research, 52(5), 55 –574.

Dolnicar, S. (2019). Reduce reuse…and don’t recycle! Annals of Tourism Research, 77, 62–63.

Dolnicar, S. & Grün, B. (2013). Validly measuring destination image in survey studies. Journal of Travel Research, 52(1), 3–14.

Kampen, J. & Swyngedouw, M. (2000). The ordinal controversy revisited. Quality and Quantity, 34(1), 87–102.

Karlsson, L. & Dolnicar, S. (2016). Does eco certification sell tourism services? Journal of Sustainable Tourism, 24(5), 694–714.

Lens, W. (1987). Theoretical research should be useful and used. International Journal of Psychology, 22(4), 453–461.

Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology, 36(2), 44–53.

Paulhus, D.L. (1991). Measurement and control of response bias, In: J.P. Robinson, P.R. Shaver and L.S. Wrightsman (Eds.), Measures of Personality and Social Psychological Attitudes. San Diego: Academic Press, pp.17–59.

Rossiter, J.R. (2011). Measurement for the Social Sciences. New York: Springer.

Viglia, G. & Dolnicar, S. (2020). A review of experiments in tourism and hospitality. Annals of Tourism Research, 80, 102858.