Replications and Reproducibility in IO-OB-HR Research: How Are We Doing, and Can We Do Better?

  • No photo
    Gilad Chen, Ph.D.

    Robert H. Smith Chair in Organizational Behavior
    Editor, Journal of Applied Psychology
    Robert H. Smith School of Business
    4538 Van Munching Hall
    University of Maryland
    College Park, MD 20742-1815
    Phone: 301-405-0923
    Email Gilad
    Gilad’s Website

Bio

Dr. Gilad Chen is the Robert H. Smith Chair in Organizational Behavior, at the University of Maryland’s Robert H. Smith School of Business. He received his bachelor degree in Psychology from the Pennsylvania State University in 1996, and his doctoral degree in Industrial/Organizational Psychology from George Mason University in 2001. Prior to joining the Smith School, Dr. Chen was on the faculty at the Georgia Institute of Technology and Texas A&M University. He has also visited and taught at the Hong Kong University of Science and Technology, Singapore Management University, Technion, and Tel-Aviv University.

Dr. Chen teaches courses on a variety of organizational behavior, human resource management, and methodological topics. His research focuses on work motivation, adaptation, teams and leadership, with particular interest in understanding the complex interface between individuals and the socio-technical organizational context. He has won several research awards, including the 2007 Distinguished Early Career Contributions Award from the Society for Industrial and Organizational Psychology, and the 2008 Cummings Scholar Award from the Organizational Behavior Division of the Academy of Management, and the 2014 Distinguished Scholar-Teacher Award from the University of Maryland. Dr. Chen is also an elected Fellow of the American Psychological Association, Association for Psychological Science, Society of Industrial-Organizational Psychology, and Society of Organizational Behavior.

Dr. Chen’s research has appeared in such journals as the Academy of Management Journal, Journal of Applied Psychology, Journal of Organizational Behavior, Personnel Psychology, Organizational Behavior & Human Decision Processes, and Research in Organizational Behavior. He is now serving as the Editor of the Journal of Applied Psychology (2014-2020), after serving as Associate Editor for the journal from 2008 through 2013. He has also been serving as an editorial board member of the Academy of Management Journal and the Academy of Management Review.

Abstract

In this talk, I’ll review some recent debates and evidence regarding the so-called “replication and reproducibility crisis” in I-O-OB-HR research, and related fields. I’ll note that evidence suggests that, as a field, we may not be doing as badly as some authors have concluded. However, we can certainly improve scientific rigor in our field, by promoting stronger climate that supports replications and reproducibility of research findings.

Digital Reader: Resources Recommended by the Speaker

Anderson, S. F., & Maxwell, S. E. (2016). There’s more than one way to conduct a replication study: Beyond statistical significance. Psychological Methods, 21, 1-12.

Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M., and Rao, S. M. (2018). Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board Task Force Report. American Psychologist, 73, 3-25.

Chen, G. (2015). Editorial. Journal of Applied Psychology, 100,1-4.

Cortina, J. M., Aguinis, H., & DeShon, R. P. (2017). Twilight of dawn or of evening? A century of research methods in the Journal of Applied Psychology. Journal of Applied Psychology, 102, 274-290.

Kozlowski, S. W. J. (2011). Comment policy. Journal of Applied Psychology, 96,231-232.

Gilbert, D. T., King, G., Pettigrew, S., & Wilson, T. D. (2016). Comment on “Estimating the reproducibility of psychological science”. Science, 351(6277),1037-c.

Maxwell, S. E., Lau, M. Y., & Howard, G. S. (2015). Is psychology suffering from a replication crisis? What does “failure to replicate” really mean? American Psychologist, 70, 487-498.

Mitchell, G. (2012). Revisiting truth or triviality: The external validity of research in the psychological laboratory. Perspectives on Psychological Science, 7,109-117.

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251),1-8.