TY - JOUR
T1 - Examining the generalizability of research findings from archival data
AU - Generalizability Tests Forecasting Collaboration
AU - Delios, Andrew
AU - Clemente, Elena Giulia
AU - Wu, Tao
AU - Tan, Hongbin
AU - Wang, Yong
AU - Gordon, Michael
AU - Viganola, Domenico
AU - Chen, Zhaowei
AU - Dreber, Anna
AU - Johannesson, Magnus
AU - Pfeiffer, Thomas
AU - Uhlmann, Eric Luis
AU - Al-Aziz, Ahmad M.Abd
AU - Abraham, Ajay T.
AU - Trojan, Jais
AU - Adamkovic, Matus
AU - Agadullina, Elena
AU - Ahn, Jungsoo
AU - Akinci, Cinla
AU - Akkas, Handan
AU - Albrecht, David
AU - Alzahawi, Shilaan
AU - Amaral-Baptista, Marcio
AU - Anand, Rahul
AU - Ang, Kevin Francis U.
AU - Anseel, Frederik
AU - Aruta, John Jamir Benzon R.
AU - Ashraf, Mujeeba
AU - Baker, Bradley J.
AU - Bao, Xueqi
AU - Baskin, Ernest
AU - Bathula, Hanoku
AU - Bauman, Christopher W.
AU - Bavolar, Jozef
AU - Bayraktar, Secil
AU - Beckman, Stephanie E.
AU - Benjamin, Aaron S.
AU - Brown, Stephanie E.V.
AU - Buckley, Jeffrey
AU - Buitrago, Ricardo E.
AU - Bution, Jefferson L.
AU - Byrd, Nick
AU - Carrera, Clara
AU - Caruso, Eugene M.
AU - Chen, Minxia
AU - Chen, Lin
AU - Cicerali, Eyyub Ensari
AU - Cohen, Eric D.
AU - Crede, Marcus
AU - Cummins, Jamie
N1 - Publisher Copyright:
© 2022 National Academy of Sciences. All rights reserved.
PY - 2022/7/26
Y1 - 2022/7/26
N2 - This initiative examined systematically the extent to which a large set of archival research findings generalizes across contexts. We repeated the key analyses for 29 original strategic management effects in the same context (direct reproduction) as well as in 52 novel time periods and geographies; 45% of the reproductions returned results matching the original reports together with 55% of tests in different spans of years and 40% of tests in novel geographies. Some original findings were associated with multiple new tests. Reproducibility was the best predictor of generalizability-for the findings that proved directly reproducible, 84% emerged in other available time periods and 57% emerged in other geographies. Overall, only limited empirical evidence emerged for context sensitivity. In a forecasting survey, independent scientists were able to anticipate which effects would find support in tests in new samples.
AB - This initiative examined systematically the extent to which a large set of archival research findings generalizes across contexts. We repeated the key analyses for 29 original strategic management effects in the same context (direct reproduction) as well as in 52 novel time periods and geographies; 45% of the reproductions returned results matching the original reports together with 55% of tests in different spans of years and 40% of tests in novel geographies. Some original findings were associated with multiple new tests. Reproducibility was the best predictor of generalizability-for the findings that proved directly reproducible, 84% emerged in other available time periods and 57% emerged in other geographies. Overall, only limited empirical evidence emerged for context sensitivity. In a forecasting survey, independent scientists were able to anticipate which effects would find support in tests in new samples.
KW - archival data
KW - context sensitivity
KW - generalizability
KW - reproducibility
KW - research reliability
UR - http://www.scopus.com/inward/record.url?scp=85134491120&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85134491120&partnerID=8YFLogxK
U2 - 10.1073/pnas.2120377119
DO - 10.1073/pnas.2120377119
M3 - Article
C2 - 35858443
AN - SCOPUS:85134491120
SN - 0027-8424
VL - 119
JO - Proceedings of the National Academy of Sciences of the United States of America
JF - Proceedings of the National Academy of Sciences of the United States of America
IS - 30
M1 - e2120377119
ER -