A typology of quantitative approaches to discovery

A typology of quantitative approaches to discovery in tourism and hospitality research (YouTube, 2m 39s)

Introduction

Tourism research has a proud history of contributing to theoretical knowledge and creating practical insights that are of value to many stakeholders, including tourism businesses, destination management organisations, governments, communities, travellers, and many more. Each quantitative approach to discovery has distinct advantages and disadvantages. Yet, the field of tourism research – along with most other social science disciplines – currently relies predominantly on one-off cross-sectional surveys for discovery. Alternative approaches such as laboratory and field experiments are largely neglected (Fong et al., 2016; Falk & Heckman, 2009). In tourism and hospitality research, “only a handful of the articles reviewed employed an experimental design” (Mattila, 2004, p. 454). Similarly, among tourism marketing studies conducted between 2008 and 2012 (Dolnicar & Ring, 2014) only 7.3% generated insights about causal relationships, although the number of experimental studies has been increasing since 1972, especially in hospitality research (Fong et al., 2016). Today, survey studies still dominate both tourism and hospitality research, with the popularity of experiments peaking in 2016, then declining until 2018, and increasing again from 2018 to July 2019 (Sun et al., 2020). While survey studies form an important part of discovery, the heavy reliance on surveys limits the nature of conclusions that can be drawn. Selecting surveys as the preferred data generation mechanism is partially caused by the convenience and affordability of such data. In part, we argue, it may also be because most studies appear to be using surveys, thus suggesting that surveys represent the gold standard in tourism research.

The 50th birthday of Annals of Tourism Research presents an opportunity to reflect on the quantitative methods predominantly used in tourism research and clarify the role alternative approaches can play in making discoveries in the field of tourism. Discovery involves learning previously unknown things about “objects, phenomena, mechanisms, cures, technologies and theories” (Gobet et al., 2019, p.2) and can result from many different methodological approaches. In this article, we focus on quantitative approaches. Specifically, we focus on studies that rely on quantitative measures and test specific research hypotheses. This includes both primary quantitative approaches (such as experiments and surveys) as well as secondary approaches (such as relational research based on historical data). We develop a typology of quantitative approaches in tourism and hospitality research and investigate their frequency of use. The typology, we hope, will spark conversation about research design choices beyond those that are most common. Additionally, at the planning stage, the typology can offer guidance on the choice of research design given the construct under investigation and the form of discovery to be created. At the communication stage, the typology allows disclosure of the types of discoveries being made. Such disclosure makes it easier for readers to understand the strengths and limitations of published work.

Implications of different quantitative approaches

The importance of clearly distinguishing between different methods of quantitative discovery becomes apparent when reflecting on implications for the internal and external validity (generalisability) of research findings. Internal validity is the extent to which correct conclusions can be drawn based on the variables of interest – rather than alternative causes or confounding factors (Slack & Draugalis, 2001). To achieve high internal validity, confounding variables must be controlled (Slack & Draugalis, 2001). While internal validity is a key requirement (Campbell & Stanley, 1963), “the lack of external validity is potentially as harmful as the lack of internal validity” (Findley et al., p. 376). External validity reflects the extent to which findings are generalisable, for instance, across populations (population validity) and across environments (ecological validity; Bracht & Glass, 1968).
While the ability to control confounding variables means that, for instance, laboratory experiments can achieve high internal validity, the external validity of such experiments is often low (Roe & Just, 2009). In contrast, assessing actual behaviours in the field generates more generalisable findings (high external validity; Roe & Just, 2009). Field experiments maintain a medium to high internal validity as the research can still control some external factors, while field data by itself has a low internal validity (Roe & Just, 2009). This comparison shows that there is often a trade-off between internal validity and the generalisability of findings (external validity). Therefore, all types of discovery play a role in the collective scholarly advancement, as long as researchers communicate clearly and unambiguously the type of discovery and its implications for internal and external validity. A shared understanding of different types of quantitative discovery in tourism and hospitality would facilitate the unambiguous disclosure of the form of discovery that researchers are engaging in. Offering a typology as the basis for such a shared understanding is the overarching aim of this study.

Forms of knowledge creation

The starting point for developing the typology is the framework of forms of knowledge proposed by Rossiter (2001; 2002). We chose this framework because it is helpful in differentiating types of research questions, which require different quantitative approaches. First-order knowledge, as it is referred to by Rossiter (2001; 2002), is the investigation of concepts without attempting to learn anything about associations between constructs or cause- and-effect relationships. Creating first-order knowledge in the form of defining concepts is an essential first step before associations and causality can be investigated. An example in the field of tourism and hospitality research is the emergence of concepts such as couch-surfing (e.g., Steylaerts & O’Dubhghaill, 2012; Schuckert et al., 2018) – staying at a stranger’s place for free while on vacation. As such new concepts emerge, they need to be described to be understood; initially in isolation from other concepts.

Second-order knowledge, as it is referred to by Rossiter (2001; 2002), includes both structural frameworks (establishing nominal relationships) and empirical generalizations (establishing associations). Second-order knowledge moves away from investigating constructs in isolation and towards an understanding of how concepts interact. Arguably one of the most common second-order knowledge research questions investigated in tourism and hospitality research is the link between guest satisfaction, loyalty, and revisitation intentions (e.g., Papadopoulou et al., 2022; Yang et al., 2022a; Peng et al., 2023; for an overview see: Dolnicar et al., 2015). Second-order knowledge offers no insights into causal relationships between concepts; if data is collected in a one-off cross-sectional study where all constructs (e.g., satisfaction, loyalty, and revisitation intention) are measured at the same point in time without any change occurring in between the measurement of those constructs, it cannot be determined whether satisfaction drives loyalty, which drives behavioural intention or, alternatively, whether the intention to return leads to guests expressing higher satisfaction and loyalty.

To be able to determine which concept drives another concept, third-order knowledge is required (Rossiter, 2001; 2002). Third-order knowledge involves an initial baseline measurement of a construct of interest, a researcher-induced change (or interventions) and a re-measurement of the construct of interest. Because the researcher is in control of the intervention – and other possible intervening variables either remain unchanged (ceteris paribus) or are controlled for – it can be determined if the intervention caused a change in the construct of interest.

Typology of quantitative approaches to discovery

Our typology builds on Rossiter (2001; 2002) to distinguish different forms of discovery in tourism and hospitality research. However, in our typology, we do not use the terms first, second and third-order discovery to prevent the perception that any form of discovery is, a priori, superior. The proposed typology, therefore, supports both theory-first approaches as well as emerging empirics-first approaches (e.g., Golder et al., 2022). An example of an empirics-first approach supported by the typology is an observation in a field experiment that leads to the development of a novel theory or an association study that takes the first important step in exploring new phenomena or developments. The aim of the proposed typology is to offer a clear differentiation between different forms of discovery.

Our typology distinguishes nine forms of discovery. The columns of the typology (see Figure 1) reflect Rossiter’s forms of knowledge creation. The first column relates to research that aims to gain an understanding of concepts (we use the abbreviation “Co” for concept), which – while important in scholarly discovery – is not the primary focus of attention of the empirical part of the present study. The second column relates to quantitative research that reveals associations between constructs (abbreviated as “A” for associations). The third column relates to quantitative research that reveals a causal link between constructs (abbreviated with “Ca” for causal). To illustrate the focus of the current study, Figure 1 displays a line between research focused on learning more about specific concepts (not included in our review) and research focused on associations between constructs and causal links (the focus of our review). In our typology, we postulate that – in most instances – an experimental design is required to make causal inferences. However, we acknowledge that recent developments in methodologies support insight into cause-effect relationships without using experimental designs (e.g., Inferred Causation Theory; Pearl & Verman, 1995). We discuss this methodology in more detail in our section on future directions.

The rows of the typology represent different types of constructs under investigation. Constructs that cannot be observed in the field are latent (abbreviation “L”). When measuring latent constructs, researchers often rely on self-reports by study participants, which they typically obtain by conducting a survey. Truly latent variables refer to internal processes that cannot be directly observed in the field even given all required resources (e.g., time, money, ethics approvals). While, truly latent variables can be inferred from self-reports, proxy measures, or behaviours, they cannot be directly measured by any of these approaches.

Examples of truly latent constructs are travel-related emotions (Demeter et al., 2022; Su et al., 2023) and perceptions of tourist destinations (Dolnicar & Grün, 2013) or service quality (Weiermair & Fuchs, 1999). In contrast, behaviours are not latent and could be observed as actual behaviours given the required resources. Therefore, any self-report measures that aim to infer actual behaviours, such as intentions to show a certain behaviour, are preferably assessed as actual or proxy behaviours in a context that resembles a realistic setting as closely as possible prior to making practical recommendations (e.g., Dolnicar et al., 2023). For many behaviours of interest in tourism and hospitality research, such studies would be conducted at destinations, tourist attractions, or in hotels and restaurants.

Key advantages of survey research are that it is quick and cost-effective. Recent years have further illustrated the benefits of survey research: It enables researchers to collect data from otherwise difficult-to-reach individuals and to continue research even when circumstances change dramatically, as was the case with COVID-19-related lockdowns. Yet, one-off cross-sectional survey studies (Type AL) do – in most instances – not permit causal conclusions, and they are often prone to capturing response biases, which undermine data quality (e.g., Dolnicar, 2013). Further, survey research relies on self-reported behavioural intentions or self-reported past behaviour which does not necessarily reflect actual behaviours (e.g., Rhodes & Dickau, 2012). Yet, despite these drawbacks, surveys represent a key data collection instrument in tourism research and the social sciences more broadly. An alternative to survey research when conducting association studies is the use of big data and AI, although those approaches can raise ethical concerns regarding privacy and security (Car et al., 2019) and must account for self-selection of people whose behaviour or views are being captured (Tam & Kim, 2018).

One way of benefitting from the convenience and affordability of survey research, while moving from conclusions about associations towards insights relating to cause-and- effect, is to conduct survey experiments (Type CaL studies). Survey experiments include between-group comparisons (comparing a control and experimental group) or repeat measurements: the first measurement captures the baseline before an intervention; the second measurement captures the change (if any) caused by the intervention. Experiments allow causal conclusions. The value of the derived insights depends, however, on the strength of the measured variables. Frequently the dependent variable is a behavioural intention (see e.g., Afshardoost & Eshaghi, 2020). Behavioural intentions are never the real construct of interest. Ultimately, any study investigating behavioural intentions is hoping to learn about actual behaviour. Because behavioural intentions can be affected by response biases, it is not safe to assume that a causal conclusion drawn from a survey experiment will also be replicable when actual behaviour serves as the dependent variable.

Figure 1. Typology of quantitative approaches to discovery

For studies where a dependent variable is an actual behaviour that can be measured in the field, field experiments represent a valuable approach to discovery (Viglia & Dolnicar, 2020). Survey experiments play a key role in the lead-up to field experiments, especially in academic research, which aims to not only influence behaviour but also understand the underlying psychological processes that lead to behaviour change. In this case, survey experiments are typically used as manipulation checks (e.g., Stoner et al., 2022). Researchers use manipulation checks to verify the effectiveness of interventions before deploying them in the field (Viglia & Dolnicar, 2020). For instance, providing refunds or vouchers to hotel guests who opt out of room cleaning aims to alleviate their doubts and create a sense of fairness. The success of this approach depends on achieving a perception of fairness before assuming any behaviour change (Dolnicar et al., 2019).

To avoid relying on self-report measures, researchers can use proxy measures (abbreviated as “P”). Proxy measures are depicted in the second row of the typology in Figure 1; they can be used to indirectly measure latent variables (e.g., emotions inferred from skin conductance and facial electromyography data; Li et al., 2018) or actual behaviours (e.g., revisit behaviour inferred from online reviews; Fan et al., 2022). A rich array of psychophysiological measures such as eye tracking or skin conductance can function as proxy measures, and they are increasingly being used to push the boundaries of discovery (e.g., Li et al., 2018 & 2022). Next to psychophysiological measures, choice experiments are a further type of proxy measure that researchers can utilise to infer latent constructs or actual behaviours (for a discussion of the advantages of choice experiments see e.g., Huynh et al., 2018). While the observed constructs are not the constructs of interest that ultimately matter in the actual environment (they are proxies), critically important insights can be derived from such work.

Proxy constructs can be included as part of a proxy construct study that focuses on associations (Type AP). However, where a proxy measure can be established and the aim is to reveal a causal conclusion, proxy construct experiments (Type CaP) represent the preferred methodological option. For example, a researcher may want to measure the baseline attention (a latent variable) a restaurant patron pays to the menu inferred through eye-tracking data. The researchers might then change the menu and determine whether menu alterations influence where patrons direct their attention (see e.g., Babakhani et al., 2020).

The third row in the typology represents constructs that can be observed as actual behaviour in the field (abbreviated as “B”) – either through behavioural observations or through the analyses of secondary data. Such secondary data analyses of actual behaviours are becoming more common in tourism research as much of the actual behaviour is occurring on the internet (Chen et al., 2023; Lee et al., 2022). An example is research that looks at an association between spatial, temporal, and cultural distance to the travel location and customer review lengths (e.g., Zhang et al., 2022). Importantly, such studies of actual behaviours (Type AB) can – in most instances – only support conclusions about associations because researchers are not in the position to manipulate any constructs. We class natural experiments as a sub-group of Type AB discovery. Natural experiments do not meet the requirements of true experiments: the intervention or independent variable is not controlled by researchers and random allocation of study participants to experimental conditions is not possible. Instead, such studies examine the effect of naturally occurring changes either through longitudinal designs that measure the change in the dependent variable before and after an external event occurred, or the difference in a dependent variable across groups of people exposed or not exposed to an external event (Leatherdale, 2019). As such, causal conclusions from natural experiments must be drawn with great care.

Finally, when constructs are observable – as is the case with most behaviours – and tourism and hospitality researchers want to draw conclusions about causal relationships, field experimentation (Type CaB) offers a promising approach to discovery. Field experimentation implies collecting baseline measurements of the behaviour, then introducing a change, and measuring the post-intervention behaviour. For a field study to be a true experiment, the independent variable must be manipulated by the researchers and the study participants across experimental groups must be randomly assigned (Miller et al., 2020). This is rarely the case because it is impossible to, for instance, assign tourists to specific vacation times randomly.

Many field studies in tourism, therefore, adopt a quasi-experimental approach in which the independent variable is manipulated but participants are not randomly allocated to conditions (e.g., Dolnicar et al., 2020; Miller et al., 2020). In such instances, variation in the guest mix, for example, must be controlled for to ensure it does not interfere with the conclusions drawn about the researcher-induced intervention.

While field experimentation has many benefits and tests behaviour directly in the context of interest, it has its drawbacks also: Field experiments require very thorough ethical considerations, especially in instances where individual consent cannot be obtained. Field experiments conducted at one specific site may not generalise to other sites with different characteristics. Practically, for researchers, a key challenge of field experiments is the need for mutually beneficial relationships with industry. The reliance on industry partners can determine which experiments and experimental conditions are being implemented in the field; industry partners are unlikely to test any experimental conditions with potential negative outcomes or take part in experiments that will let them appear in a bad light. Finally, field experimentation can be very vulnerable to any changes in surrounding circumstances. This has become evident during, for instance, the COVID-19 pandemic, which made it impossible for many researchers to collect field data. Research does not happen in isolation. Any study, irrespective of the research design, contributes to the collective understanding of a phenomenon in an incremental nature.

Therefore, different forms of discovery and – in the case of experiments – many experiments testing a phenomenon across different contexts are most powerful. While not formally part of our typology, meta-analyses are of critical importance because they synthesize and communicate collectively generated discovery, thus pushing the field of research forward.

Currently dominant types of quantitative discovery in tourism research

To determine how common the types of quantitative discovery currently are in tourism and hospitality research, we conducted a systematic quantitative literature review (Pickering et al., 2014). We analysed articles published in 2022 or in press as of 1st December 2022 in three of the leading international tourism journals: Tourism Management (Impact factor 2021 = 12.879; SCImago Journal Rank Indicator = 3.38), Annals of Tourism Research (Impact factor 2021 = 12.853; SCImago Journal Rank Indicator = 3.15), Journal of Travel Research (Impact factor 2021 = 8.933; SCImago Journal Rank Indicator = 3.29; Scimago, 2022). For these journals, we exported the title, author name, journal name and year of publication to an MS Excel spreadsheet.

We focused on papers that rely on quantitative measures (both primary and/or secondary). We screened the records and only included articles that test for an association or causal link in at least one study. This meant that we did not include descriptive studies and exploratory. We further excluded papers that were qualitative in nature, meta-analyses, systematic literature reviews, studies that focused on scale or model development without testing for associations or causal links, general forecasting models, case studies, conceptual articles, and letters. We performed eligibility assessments by screening the full texts of the remaining articles independently. From the studies selected for analysis, we excluded pre- and pilot studies. For mixed-method papers, we only coded quantitative studies. The final sample consists of 292 papers (see Figure 2).

Figure 2. Overview of the paper selection process (total number of articles screened; 2022)

The coding unit is the article; we captured how many studies were conducted per article. We then classified the studies into one of the six quantitative types of discovery in tourism: survey studies (Type AL); survey experiments (Type CaL); proxy construct studies (Type AP); proxy construct experiments (Type CaP); field study (Type AB); field experiment (Type CaB). Under Type CaB studies, we distinguish between field experiments that meet the conditions of (1) the researcher manipulating the independent variable and (2) random allocation of participants (Miller et al., 2020) and quasi-experimental field studies which cannot guarantee the random allocation of participants. For each article, we determined the number of studies falling into each of those types. On article level, we further coded whether the ultimate outcome of interest was actual behaviour. If this was the case, we coded whether the article assesses the variables in at least one field study or experiment.

While some studies included self-reports of actual past behaviours (e.g., behaviours on past holidays) we coded this as a self-report measure rather than a measure of actual behaviour due to accuracy issues with recall data of past behaviours (e.g., Karlsson & Dolnicar, 2016). Further, surveys administered in the field were still classed as survey research (in contrast to field experiments which measure actual behaviours). Finally, we coded the highest claim made in each article (causality versus association). The following or similar phrases were coded as causality: “A influences B”; “A impacts B”; “A affects B”; “the effect of A on B”; “A increases/ decreases B”. The frequencies resulting from this analysis capture the status quo of tourism and hospitality research in terms of methods of discovery and claims made.

Figure 3 shows the distribution of empirical studies published in Tourism Management, Annals of Tourism Research and Journal of Travel Research in 2022 across the six investigated types of discovery, where the frequency is illustrated additionally using box shading. As can be seen, survey studies and experiments dominate with 72% of all studies relying on survey data; 31% are of Type AL and 41% of Type CaL. This stands in contrast to the ultimate construct of interest of the included articles. We analysed the ultimate construct of interest for each article, irrespective of the number of studies each article included. This analysis revealed that only 15% of all articles (n = 44) aimed to investigate truly latent variables that were not observable in the field. Examples of such truly latent variables include emotions (e.g., overall and eudaimonic happiness as assessed by Yang et al., 2022b); attitudes and perceptions (e.g., attitudes towards a destination as assessed by Hadinejad et al., 2022) and subjective well-being (e.g., in Yi et al., 2022).

Figure 3. The distribution across discovery types of quantitative empirical studies published in Tourism Management, Annals of Tourism Research and Journal of Travel Research in 2022 (N = 475; dark grey shading >30%, light grey shading 10-29%)

The main outcome of interest for most articles (85%, n = 248) were actual behaviours. Yet, of these 248 articles, only 35% (n = 88) included at least one actual behaviour of each main variable of interest. This means that – instead of assessing actual behaviour and measures in the field – most papers relied on survey measurements exclusively. For instance, instead of assessing actual behaviour, those papers included intention measures and self- report items on the willingness or likeliness to show a certain behaviour. Similarly, rather than analysing actual evaluations and online recommendations by guests, those papers included survey items capturing evaluations and recommendations.

Proxy constructs are rarely investigated, Type AP studies represent only 1.5% and Type CaP studies only 5.5% of all empirical studies recently published in three of the leading tourism journals. An article by Kim et al. (2022) includes both Type AP and Type CaP studies. First, the authors test for an association between perceived COVID-19 threat and extremeness aversion tendency which was inferred from participants’ selections in a choice task (Type AP study). To test for a causal link, they then manipulate the level of perceived threat (low vs high) and test the effect on extremeness aversion (Type CaP study).

Field investigations of Type AB represent 20% of studies. This also includes studies classed as natural experiments where the independent variable is not manipulated by the researcher and random allocation of participants is not possible (see Leatherdale, 2019). Type AB studies are observational; the ability to extract behavioural data from the internet has substantially improved access to measures of actual behaviour. Creating big data with the help of web-scraping offers a further option to assess actual behaviours. Web-scraping refers to the automatic extraction of large amounts of information from multiple websites, that can be saved and later analysed (Landers et al., 2016). For example, Mitra and colleagues (2022) use web data to test for a relationship between tourism development and gender inequality. Zhang and colleagues (2022) use web-scraping (specifically web crawlers) to extract review data from TripAdvisor. In contrast, web-tracking refers to mechanisms such as cookies being used to track people’s browsing behaviour and build digital traces of users which can raise privacy concerns (Samarasinghe & Mannan, 2019).

Type CaB discovery is rare, representing only 1% (four studies) of the analysed studies. All four Type CaB studies were quasi-experimental designs, meaning that while the researchers manipulated the independent variable in the field, participants were not randomly allocated to experimental conditions (see Miller et al., 2020).

The final analysis we conducted was the assessment of causal claims made in articles, and whether such claims were permissible given the discovery type. Results from this analysis reveal that only 10% of articles (n = 28) consistently use language reflecting that the relationship between constructs is associative only; 90% (n = 264) make causal claims at some point in the article. Of those 264 articles that made causal claims, 36% (n = 96) relied on survey studies only (Type AL studies), thus not permitting causal claims.

Discussion and limitations

Tourism and hospitality research has come a long way since its early beginnings and the first issue of Annals of Tourism Research half a century ago. Sophisticated new data collection methods are now available, and the number of tourism and hospitality studies has increased dramatically over the past few decades. Yet, many of the limitations that were undermining the validity and generalisability of conclusions drawn since the early beginnings of tourism and hospitality research remain; little progress has been made in terms of harvesting the many opportunities new technologies offer, for example, moving away from the over-reliance on self-report data.

The shift from association studies to studies that permit causal conclusions has been more distinct, with survey experiments now accounting for some 40% of work published in leading tourism journals. Survey experiments have high internal validity but the ability to generalise to actual behaviour is limited. This is a critical limitation because many tourism studies aspire to inform some form of decision-making in the real world. Research that is unable to support a causal effect of a specific intervention in the actual context is – in most cases – not a sufficient basis for offering such guidance. This requires field experimentation – an area that has not yet been widely adopted by tourism and hospitality researchers, probably because it is expensive, complicated, labour-intensive and, therefore, risky to conduct. Yet, collectively, field experimentation plays an important role in testing the impact of interventions on the desired outcome, both in terms of theoretical advancement and the ability to offer valid recommendations based on the findings. Therefore, field experimentation has the potential to diversify and enrichen currently dominant research approaches.

Our review highlights also that even behaviours that could be observed relatively easily are mostly captured via self-report measures, such as stated behavioural intentions. This reflects a collective acceptance of the fact that behavioural intentions are good predictors of actual behaviour. Yet, across many disciplines, researchers have demonstrated repeatedly that stated behavioural intentions fail to predict behaviour. In tourism, measuring stated behavioural intentions can vastly overestimate actual behaviour. For example, 59% of tourists reported that the environmental impact of a boat tour affected their booking decision, yet 87% of tourists purchased tickets for a not eco-certified boat (Karlsson & Dolnicar, 2016).

Similarly, 46% reported intentions to purchase carbon offset but only 6% did (Mair, 2011). A meta-analysis of experiments that manipulated intentions in the physical activity domain concludes that intention and behaviour show a weak relationship only (Rhodes & Dickau, 2012).

For research interested in understanding actual behaviour, future discovery would benefit from tourism researchers considering field studies and field experimentation. This may require a change in mindset among editors, associate editors and reviewers – instead of accepting stated behavioural intentions as a dependent variable and, in so doing, perpetuating the collective acceptance of intentions as a satisfactory construct of investigation, editors, associate editors and reviewers should encourage researchers to make the extra effort to collect data in the field where applicable.

Additionally, the field would benefit from tourism and hospitality researchers disclosing clearly and unambiguously which discovery type their work represents. The typology in Figure 1 assists with this. By disclosing the discovery type, it is clear to both the researchers and the readers of a study, which kinds of conclusions are permissible, and which are not. Clarity around the form of discovery being created further assists researchers in avoiding the use of misleading language. Currently, it is not uncommon for authors who conducted association studies to draw causal conclusions. Typically, this is not permissible and, can mislead academic researchers who build on this work as well as the wider industry. Managers may be tempted to implement recommendations by tourism researchers, assuming that doing so will achieve a specific outcome. However, if the study did not actually implement the measure in the field, it is unclear whether the measure has an actual impact in real life or whether an effect only emerges in a laboratory experiment under highly controlled conditions.

Our study is not without limitations. We have analysed only articles published recently and in three of the leading tourism journals. We argue that this selection ensures that state-of-the-art high-quality tourism and hospitality research was adequately reflected but acknowledge that it is only a small fraction of work being published in the field. The percentages shown in Figure 3, therefore, would likely change with the inclusion of additional journals and years. The review focused on the year 2022. Research published in this year has been influenced by the COVID-19 pandemic which likely contributes to the low number of field studies and experiments. It will be interesting to compare these findings to numbers emerging from 2023 and 2024 to monitor whether field studies and experiments will become more common. Further, the current review does not include qualitative work which forms a further key part of discovery in tourism. While this was not within the scope of the current paper, the provided framework accounts for qualitative research and future reviews could add to the framework by distinguishing different types of discovery in qualitative research.

Future directions

Throughout our manuscript, we discuss the benefits and drawbacks of different forms of discovery and point to the current over-reliance on specific approaches. Using a wider variety of research approaches and selecting the approach most suitable for a specific research question, are important steps to overcome this. Another step is to be aware of and consider emerging and novel approaches. Undoubtedly, moving forward, the proposed typology will expand and change as such new approaches increasingly contribute to the research landscape. In the following, we discuss a few examples of such emerging and promising approaches and methodologies: Inferred Causation Theory; Virtual reality; Computational modelling and simulations; Artificial Intelligence and machine learning tools and technologies.

We argue that studies that aim to draw conclusions about actual behaviour should strive towards measuring actual behaviour. Further, we argue that causal conclusions should be based on experimental designs. However, field experimentation is not always possible or feasible. Some variables cannot be manipulated and/or measured in an experimental setting, for instance, due to ethical issues or high costs associated with such a study. We further pointed out that most field research in tourism and hospitality is quasi-experimental because randomisation cannot be guaranteed, thus limiting the ability to draw causal conclusions unless other factors can be controlled. Inferred Causation Theory (Pearl & Verman, 1995) offers a possible solution to the practical difficulties associated with implementing field studies that permit causal conclusions. If specific conditions are met (for an overview see e.g., Mazanec, 2006), Inferred Causation Theory enables causal conclusions without the need for “experimental manipulation and even a temporal sequence of measurements” (Mazanec, 2006, p.42).

Further promising approaches include computational modelling and simulations, Virtual Reality, and Artificial Intelligence and machine learning tools and technologies. Although computational modelling and simulations are in their early stage of development in tourism and hospitality research, such models are considered effective tools for analysing complex systems and phenomena (Johnson et al., 2017). For instance, networks and agent- based models enable researchers to examine and describe the characteristics (static and dynamic features) of the study object and thus investigate complex phenomena (Baggio, 2020). Simulations can, for instance, be used to test hypothetical scenarios and conditions to learn more about how tourism demand and behaviours are linked to different parameters included in the simulation and thus help solve practical problems (Buchta & Dolnicar, 2003).

Virtual Reality and Augmented Reality offer new opportunities to move from self- reported behavioural intentions to observing actual behaviours in a scenario that is more realistic than surveys (e.g., Doborjeh et al., 2022). Virtual Reality research has several advantages over field studies: it can be cheaper (once relevant tools have been obtained), faster, and able to record more detailed behavioural data (Burke, 2018). Further, it does not require managing industry relationships and can be more resilient than field studies. Another opportunity for future research lies in the application of Artificial Intelligence and machine learning tools and technologies (e.g., Guo et al., 2017; Mazanec, 1992). With an increase in studies using “Big Data”, advanced analytical techniques based on artificial intelligence and machine learning are increasingly required to gain insights and aid the analysis of such data sets (Jordan & Mitchell, 2015).

Taken together, new methodologies, theories and statistical approaches will increasingly add to discovery in tourism and hospitality research moving forward. We have discussed a few examples of such developments to illustrate their future value to the field but there are many more. Such approaches can help to overcome limitations of existing forms of discovery or extend the typology.

Conclusions and recommendations

The key insight from this analysis is that there is a distinct lack of diversity in the quantitative research approaches used by tourism and hospitality researchers. Most studies rely on self- report measures (even if the construct of interest is actual behaviour) and on cross-sectional data (even if the effect of an intervention is the core research question). This could be overcome by implementing Type AL studies as Type CaL studies (self-report experiments to improve validity), as Type AP or AB studies (to improve generalisability) or even as Type CaB studies (improving both validity and generalisability).

In the next 50 years, tourism and hospitality researchers could more effectively push the boundaries of discovery than they have in the past through (1) increased awareness of existing forms of discovery, along with an understanding of the limitations associated with each form of discovery; (2) unambiguous disclosure of the form of discovery generated in a study, enabling readers to understand which conclusions and practical implications can legitimately be drawn; and (3) disclosure of the reasons for choosing a specific form of discovery. Finally, (4) reporting results requires the use of precise language. When the discovery type allows only conclusions about associations, for example, it is not permissible to make statements that imply causality. This is a joint responsibility of authors, reviewers, and journal editors; where authors use misleading language, reviewers and journal editors must insist it be corrected to prevent readers from misinterpreting the findings. Most importantly, however, this study highlights the value of exploring the best quantitative research design for every research question asked, rather than falling back on established, but often suboptimal approaches.

We hope that the proposed typology encourages critical discussion of research approaches used in tourism and hospitality and welcome extensions of the framework. The aim of the typology is not to promote one form of discovery over another, nor to provide an exhaustive list of advantages and disadvantages of each approach. Rather, the aim is to offer a structure that supports methodological conversations and guides readers in understanding the limitations of published work.

References

Afshardoost, M., & Eshaghi, M. S. (2020). Destination image and tourist behavioural intentions: A meta-analysis. Tourism Management, 81, Article 104154. https://doi.org/10.1016/j.tourman.2020.104154

Babakhani, N., Lee, A., & Dolnicar, S. (2020). Carbon labels on restaurant menus: do people pay attention to them?. Journal of Sustainable Tourism, 28(1), 51-68. https://doi.org/10.1080/09669582.2019.1670187

Bracht, G. H., & Glass, G. V. (1968). The external validity of experiments. American Educational Research Journal, 5(4), 437-474. https://doi.org/10.2307/1161993

Buchta, Ch. & Dolnicar, S. (2003) Learning by Simulation – Computer Simulations for Strategic Marketing Decision Support in Tourism. International Journal of Tourism Sciences, 3(1), 65-78. https://doi.org/10.1080/15980634.2003.11434540

Burke, R.R. (2018). Virtual Reality for Marketing Research. In: Moutinho, L., Sokele, M. (eds) Innovative Research Methodologies in Management. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-319-64400-4_3

Campbell, D. T., Stanley, J. C., & Gage, N. L. (1963). Experimental and quasi-experimental designs for research. Boston: Houghton Mifflin

Car, J., Sheikh, A., Wicks, P., & Williams, M. S. (2019). Beyond the hype of big data and artificial intelligence: building foundations for knowledge and wisdom. BMC medicine, 17(1), 1-5. https://doi.org/10.1186/s12916-019-1382-x

Chen, S., Chen, W., & Luo, X. (2023). Some stay and some quit: Understanding P2P accommodation providers’ continuous sharing behavior from the perspective of feedback theory. Tourism Management, 95, Article 104676. https://doi.org/https://doi.org/10.1016/j.tourman.2022.104676

Demeter, C., MacInnes, S., & Dolnicar, S. (2022). Defining and operationalizing eight forms of eudaimonia and hedonia and assessing tourism-specific context-dependency. Journal of Travel Research, Article 00472875221133042. https://doi.org/10.1177/00472875221133042

Doborjeh, Z., Hemmington, N., Doborjeh, M., & Kasabov, N. (2022). Artificial intelligence: a systematic review of methods and applications in hospitality and tourism. International Journal of Contemporary Hospitality Management, 34(3), 1154-1176. https://doi.org/10.1108/IJCHM-06-2021-0767

Dolnicar, S., Coltman, T., & Sharma, R. (2015). Do Satisfied Tourists Really Intend to Come Back? Three Concerns with Empirical Studies of the Link between Satisfaction and Behavioral Intention. Journal of Travel Research, 54(2), 152-178. https://doi.org/10.1177/0047287513513167

Dolnicar, S. & Ring, A. (2014) Tourism marketing research – Past, present and future. Annals of Tourism Research, 47, 31-47. https://doi.org/10.1016/j.annals.2014.03.008

Dolnicar, S. (2013). Asking good survey questions. Journal of Travel Research, 52(5), 551-574. https://doi.org/10.1177/0047287513479842

Dolnicar, S., & Grün, B. (2013). Validly measuring destination image in survey studies. Journal of Travel Research, 52(1), 3-14. https://doi.org/10.1177/0047287512457267

Dolnicar, S., Gray, A., Grün, B., Li, H., & Portmann, M. (2023). Automatically monitoring environmental performance in tourism–The example of plate waste at all-you-can-eat buffets. Annals of Tourism Research Empirical Insights, 4(2), 100100.

Dolnicar, S., Juvan, E., & Grün, B. (2020). Reducing the plate waste of families at hotel buffets–A quasi-experimental field study. Tourism Management, 80, Article 104103. https://doi.org/10.1016/j.tourman.2020.104103

Dolnicar, S., Knezevic Cvelbar, L., & Grün, B. (2019). A sharing-based approach to enticing tourists to behave more environmentally friendly. Journal of Travel Research, 58(2), 241–252. https://doi.org/10.1177/0047287517746013

Falk, A., & Heckman, J. J. (2009). Lab experiments are a major source of knowledge in the social sciences. Science, 326 (5952), 535-538. https://doi.org/10.1126/science.1168244

Fan, W., Li, Y., Upreti, B. R., Liu, Y., Li, H., Fan, W., & Lim, E. T. K. (2022). Big Data for Big Insights: Quantifying the Adverse Effect of Air Pollution on the Tourism Industry in China. Journal of Travel Research, 61(8), 1947–1966. https://doi.org/10.1177/00472875211047272

Findley, M. G., Kikuta, K., & Denly, M. (2021). External validity. Annual Review of Political Science, 24, 365-393. https://doi.org/10.1146/annurev-polisci-041719-102556

Fong, L. H. N., Law, R., Tang, C. M. F., & Yap, M. H. T. (2016). Experimental research in hospitality and tourism: A critical review. International Journal of Contemporary Hospitality Management, 28(2), 246-266. https://doi.org/10.1108/IJCHM-10-2014-0506

Gobet, F., Addis, M., Lane, P.C.R., Sozou, P.D. (2019). Introduction: Scientific Discovery in the Social Sciences. In: Addis, M., Lane, P.C.R., Sozou, P.D., Gobet, F. (eds) Scientific Discovery in the Social Sciences. Synthese Library, vol 413. Springer, Cham. https://doi.org/10.1007/978-3-030-23769-1_1

Golder, P. N., Dekimpe, M. G., An, J. T., van Heerde, H. J., Kim, D. S., & Alba, J. W. (2022). Learning from Data: An Empirics-First Approach to Relevant Knowledge Generation. Journal of Marketing, 87(3), 319-336. https://doi.org/10.1177/00222429221129200

Guo, Y., Barnes, S. J., & Jia, Q. (2017). Mining meaning from online ratings and reviews: Tourist satisfaction analysis using latent dirichlet allocation. Tourism Management, 59, 467-483. https://doi.org/10.1016/j.tourman.2016.09.009

Hadinejad, A., Kralj, A., Scott, N., Moyle, B. D., & Gardiner, S. (2022). Tourism marketing stimulus characteristics: A self-validation analysis of Iran. Journal of Travel Research, 61(2), 235–251. https://doi.org/10.1177/0047287520981141

Huynh, E., Araña, J. E., & Prior, J. (2018). Evaluating residents’ preferences for remediation technologies: A choice experiment approach. Science of the Total Environment, 621, 1012-1022. https://doi.org/10.1016/j.scitotenv.2017.10.125

Johnson, P., Nicholls, S., Student, J., Amelung, B., Baggio, R., Balbi, S., Boavida-Portugal, I., de Jong, E., Hofstede, G. J., Lamers, M., Pons, M., & Steiger, R. (2017). Easing the adoption of agent-based modelling (ABM) in tourism research. Current Issues in Tourism, 20(8), 801–808. https://doi.org/10.1080/13683500.2016.1209165

Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349 (6245), 255-260. https://doi.org/10.1126/science.aaa8415

Karlsson, L. & Dolnicar, S. (2016). Does eco certification sell tourism services? Evidence from a quasi-experimental observation study in Iceland. Journal of Sustainable Tourism, 24(5), 694–714. http://dx.doi.org/10.1080/09669582.2015.1088859

Kim, J., Park, J., Lee, J., Kim, S., Gonzalez-Jimenez, H., Lee, J., Choi, Y. K., Lee, J. C., Jang, S., Franklin, D., Spence, M. T., & Marshall, R. (2022). COVID-19 and extremeness aversion: The role of safety seeking in travel decision making. Journal of Travel Research, 61(4), 837–854. https://doi.org/10.1177/00472875211008252

Landers, R. N., Brusso, R. C., Cavanaugh, K. J., & Collmus, A. B. (2016). A primer on theory-driven web scraping: Automatic extraction of big data from the Internet for use in psychological research. Psychological Methods, 21(4), 475–492. https://doi.org/10.1037/met0000081

Leatherdale, S. T. (2019). Natural experiment methodology for research: a review of how different methods can support real-world research. International Journal of Social Research Methodology, 22(1), 19-35. https://doi.org/10.1080/13645579.2018.1488449

Lee, H., Koo, C., & Yang, S.-B. (2022). Spatial and social distances between U.S. domestic travelers in restaurant review assessment. Tourism Management, 93, Article 104609. https://doi.org/https://doi.org/10.1016/j.tourman.2022.104609

Li, S., Sung, B., Lin, Y., & Mitas, O. (2022). Electrodermal activity measure: A methodological review. Annals of Tourism Research, 96, Article 103460. https://doi.org/10.1016/j.annals.2022.103460

Li, S., Walters, G., Packer, J., & Scott, N. (2018). Using skin conductance and facial electromyography to measure emotional responses to tourism advertising. Current Issues in Tourism, 21(15), 1761-1783. https://doi.org/10.1080/13683500.2016.1223023

Mair, J. (2011). Exploring air travellers’ voluntary carbon-offsetting behaviour. Journal of Sustainable Tourism, 19(2), 215–230. http://dx.doi.org/10.1080/09669582.2010.517317

Mattila, A. S. (2004). Consumer behavior research in hospitality and tourism journals.
International Journal of Hospitality Management, 23(5), 449-457. https://doi.org/10.1016/j.ijhm.2004.10.001

Mazanec, J. A. (1992). Classifying tourists into market segments: A neural network approach. Journal of Travel & Tourism Marketing, 1(1), 39-60. https://doi.org/10.1300/J073v01n01_04

Mazanec, J. A. (2006). Inferred Causation Theory: Time for a Paradigm Shift in Marketing Science? In M. Spiliopoulou, R. Kruse, C. Borgelt, A. Nürnberger & W. A. Gaul (Eds.), From Data and Information Analysis to Knowledge Engineering: Proceedings of the 29th Annual Conference of the Gesellschaft für Klassifikation eV University of Magdeburg, March 9–11, 2005 (pp. 40-51). Berlin, Heidelberg: Springer Berlin Heidelberg.

Miller, C. J., Smith, S. N., & Pugatch, M. (2020). Experimental and quasi-experimental designs in implementation research. Psychiatry research, 283, Article 112452. https://doi.org/10.1016/j.psychres.2019.06.027

Mitra, S. K., Chattopadhyay, M., & Chatterjee, T. K. (2022). Can tourism development reduce gender inequality? Journal of Travel Research, Article 00472875211073975. https://doi.org/10.1177/00472875211073975

Papadopoulou, N. M., Ribeiro, M. A., & Prayag, G. (2022). Psychological determinants of tourist satisfaction and destination loyalty: The influence of perceived overcrowding and overtourism. Journal of Travel Research, Article 00472875221089049. https://doi.org/10.1177/00472875221089049

Pearl, J., & Verma, T. S. (1995). A theory of inferred causation. In Studies in Logic and the Foundations of Mathematics (Vol. 134, pp. 789-811). Elsevier.

Peng, J., Yang, X., Fu, S., & Huan, T.-C. (2023). Exploring the influence of tourists’ happiness on revisit intention in the context of Traditional Chinese Medicine cultural tourism. Tourism Management, 94, Article 104647. https://doi.org/https://doi.org/10.1016/j.tourman.2022.104647

Pickering, C., & Byrne, J. (2014). The benefits of publishing systematic quantitative literature reviews for PhD candidates and other early-career researchers. Higher Education Research & Development, 33(3), 534-548. https://doi.org/10.1080/07294360.2013.841651

Rhodes, R. E., & Dickau, L. (2012). Experimental evidence for the intention–behavior relationship in the physical activity domain: A meta-analysis. Health Psychology, 31(6), 724–727. https://doi.org/10.1037/a0027290

Roe, B. E., & Just, D. R. (2009). Internal and external validity in economics research: Tradeoffs between experiments, field experiments, natural experiments, and field data. American Journal of Agricultural Economics, 91(5), 1266-1271. https://doi.org/10.1111/j.1467-8276.2009.01295.x

Rossiter, J.R. (2001) What is marketing knowledge? Stage I: forms of marketing knowledge. Marketing Theory, 1(1), 9-26. http://dx.doi.org/10.1177/147059310100100101

Rossiter, J.R. (2002) The five forms of transmissible, usable marketing knowledge. Marketing Theory, 2(4), 369-380. http://dx.doi.org/10.1177/147059310200200407

Samarasinghe, N., & Mannan, M. (2019). Towards a global perspective on web tracking. Computers & Security, 87, Article 101569. https://doi.org/10.1016/j.cose.2019.101569

Schuckert, M., Peters, M., & Pilz, G. (2018). The co-creation of host-guest relationships via couchsurfing: a qualitative study. Tourism Recreation Research, 43(2), 220–234. https://doi.org/10.1080/02508281.2017.1384127

SCImago, (n.d.). SJR — SCImago Journal & Country Rank [Portal]. Retrieved 18 January 2022, from http://www.scimagojr.com

Slack, M. K., & Draugalis Jr, J. R. (2001). Establishing the internal and external validity of experimental studies. American journal of health-system pharmacy, 58(22), 2173- 2181. http://dx.doi.org/10.1093/ajhp/58.22.2173

Steylaerts, V., & O’ Dubhghaill, S. (2012). Couchsurfing and authenticity: Notes towards an understanding of an emerging phenomenon. Hospitality & Society, 1(3), 261–278. https://doi.org/10.1386/hosp.1.3.261_1

Stoner, J. L., Felix, R., & Stadler Blank, A. (2022). Best practices for implementing experimental research methods. International Journal of Consumer Studies. Article 12878. https://doi.org/10.1111/ijcs.12878

Su, L., Tang, B., & Nawijn, J. (2023). How destination social responsibility shapes resident emotional solidarity and quality of life: Moderating roles of disclosure tone and visual messaging. Journal of Travel Research, 62(1), 105–120. https://doi.org/10.1177/00472875211056683

Sun, S., Law, R., & Zhang, M. (2020). An updated review of tourism-related experimental design articles. Asia Pacific Journal of Tourism Research, 25(7), 710-720. http://dx.doi.org/10.1080/10941665.2019.1699128

Tam, S. M., & Kim, J. K. (2018). Big Data ethics and selection-bias: An official statistician’s perspective. Statistical Journal of the IAOS, 34(4), 577-588. http://dx.doi.org/ 10.3233/SJI-170395

Viglia, G., & Dolnicar, S. (2020). A review of experiments in tourism and hospitality. Annals of Tourism Research, 80, Article 102858. https://doi.org/10.1016/j.annals.2020.102858

Weiermair, K., & Fuchs, M. (1999). Measuring tourist judgment on service quality. Annals of tourism research, 26(4), 1004-1021. https://doi.org/10.1016/S0160-7383(99)00037-7

Yang, F. X., Li, X., & Choe, Y. (2022a). What constitutes a favorable destination brand portfolio? Through the lens of coherence. Tourism Management, 90, Article 104480. https://doi.org/10.1016/j.tourman.2021.104480

Yang, W., Zhang, Y., & Wang, Y.-C. (2022b). Would travel experiences or possessions make people happier? Journal of Travel Research, 62(2), 412-431. https://doi.org/10.1177/00472875211064631

Yi, X., Fu, X., Lin, V. S., & Xiao, H. (2022). Integrating Authenticity, Well-being, and Memorability in Heritage Tourism: A Two-Site Investigation. Journal of Travel Research, 61(2), 378–393. https://doi.org/10.1177/0047287520987624

Zhang, Z., Qiao, S., Chen, Y., & Zhang, Z. (2022). Effects of spatial distance on consumers’ review effort. Annals of Tourism Research, 94, Article 103406. https://doi.org/10.1016/j.annals.2022.103406