Questionable research practices are one contributing factor to the replication crisis that is crippling the science of psychology. This research investigated if and if so, to what extent the use of questionable research practices is prevalent in the field of IO-psychology.
Since the early 2000s awareness about failures to replicate seemingly stable effects was steadily rising, consequently leading to the question if psychology as a whole scientific field is in a replication crisis. One root of the problem is arguably questionable research practices such as HARKing (hypothesizing after results are known), only reporting significant results, stopping the data collection after significant results are achieved, citing selectively or not publishing a negative research result. Several methodological tools exist, that can help identify whether researchers might have employed QRPs such as p-hacking, namely p-curve, TIVA and R-Index.
We analyzed 234 anonymized studies that were submitted for publication to the Journal of Personnel Psychology between 2014 and 2019 in their various stages (desk-reject, reviewed-reject, revised-reject, revised-accept) for indications of p-hacking with p-curve, TIVA and R-Index.
Accepted studies showed significantly higher indications for p-hacking than rejected studies in all three detection methods. The manuscript status (from desk-reject to revised accept) was significantly and positively correlated with all three detection methods as well. Only 4% of the results in the total sample was non-significant. No manuscript was a replication study.
Our results indicate that the peer review process in itself seems to promote the usage of QRP, p-hacking precisely.
Veranstaltungsort: Hildesheim