Replication research's disturbing trend

https://doi.org/10.1016/j.jbusres.2006.12.003Get rights and content

Abstract

Researchers express concern over a paucity of replications. In line with this, editorial policies of some leading marketing journals now encourage more replications. This article reports on an extension of a 1994 study to see whether these efforts have had an effect on the number of replication studies published in leading marketing journals. Results show that the replication rate has fallen to 1.2%, a decrease in the rate by half. As things now stand, practitioners should be skeptical about using the results published in marketing journals as hardly any of them have been successfully replicated, teachers should ignore the findings until they receive support via replications and researchers should put little stock in the outcomes of one-shot studies.

Section snippets

Definitions and method

The definitions of the central terms in this study are in line with those employed by H&A (1994, p. 236). A replication is defined as “a duplication of a previously published empirical study that is concerned with assessing whether similar findings can be obtained upon repeating the study.” Likewise, a replication with extension is “a duplication of a previously published empirical research project that serves to investigate the ability to generalize earlier research findings.” Note that this

Frequency of replications

Table 1 shows the present findings on the publication incidence of replication research in marketing, and compares them with those of H&A. Whereas H&A estimate that an average of 2.4% of empirical research papers published in JM, JMR, and JCR for 1974–1989 are replications with extensions–a figure they regarded as too low–the average for these same three journals for 1990–2004 has fallen to 1.2% (i.e., only 16 extensions out of 1389 empirical articles). This downward trend applies to each of

Practical solutions to the lack of replications

To encourage the growth of replications in marketing's empirical literature, the data and methods used in the original studies should be made available on the Internet concurrent with a paper's publication. Traditionally, supporting information of this nature has been difficult to obtain (see Hubbard and Little, 1997). This procedure may require some effort by the authors, but the effort is well worth it. Gleditsch et al. (2003), in their analysis of 416 papers published in the Journal of Peace

Conclusions

Given the favorable reaction over the past decade to calls for more replications, the initial expectation was that a greater frequency of them would be published in JM, JMR, and JCR for the period 1990–2004 than were found for 1974–1989. However, the percentage of replications was published over the latter period dropped by 50%.

A number of strategies to promote replication research can be endorsed. These include:

  • Using footnotes to direct readers to data and methods (in enough detail to permit

References (18)

There are more references available in the full text version of this article.

Cited by (210)

View all citing articles on Scopus
View full text