References
Aarts, E., Dolan, C. V., Verhage, M., & Van der Sluis, S. (2015).
Multilevel analysis quantifies variation in the experimental effect
while optimizing power and preventing false positives. BMC
Neuroscience, 16(1), 1–15. https://doi.org/10.1186/s12868-015-0228-5
Aczel, B., Szaszi, B., Nilsonne, G., Akker, O. R. van den, Albers, C.
J., Assen, M. A. van, Bastiaansen, J. A., Benjamin, D., Boehm, U.,
Botvinik-Nezer, R., Bringmann, L. F., Busch, N. A., Caruyer, E.,
Cataldo, A. M., Cowan, N., Delios, A., Dongen, N. N. van, Donkin, C.,
Doorn, J. B. van, … Wagenmakers, E.-J. (2021). Consensus-based guidance
for conducting and reporting multi-analyst studies. eLife,
10, e72185. https://doi.org/10.7554/eLife.72185
Artner, R., Verliefde, T., Steegen, S., Gomes, S., Traets, F.,
Tuerlinckx, F., & Vanpaemel, W. (2021). The reproducibility of
statistical results in psychological research: An investigation using
unpublished raw data. Psychological Methods, 26(5),
527–546. https://doi.org/10.1037/met0000365
Auspurg, K., & Brüderl, J. (2021). Has the Credibility of the Social
Sciences Been Credibly Destroyed? Reanalyzing the “Many
Analysts, One Data Set” Project. Socius,
7, 23780231211024421. https://doi.org/10.1177/23780231211024421
Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008a).
Mixed-effects modeling with crossed random effects for subjects and
items. Journal of Memory and Language, 59(4), 390–412.
https://doi.org/10.1016/j.jml.2007.12.005
Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008b).
Mixed-effects modeling with crossed random effects for subjects and
items. Journal of Memory and Language, 59(4), 390–412.
https://doi.org/10.1016/j.jml.2007.12.005
Balota, D. A., Cortese, M. J., Sergent-Marshall, S. D., Spieler, D. H.,
& Yap, M. J. (2004). Visual Word Recognition of Single-Syllable
Words. Journal of Experimental Psychology: General,
133(2), 283–316. https://doi.org/10.1037/0096-3445.133.2.283
Balota, D. a., Yap, M. J., Cortese, M. J., Hutchison, K. a., Kessler,
B., Loftis, B., Neely, J. H., Nelson, D. L., Simpson, G. B., &
Treiman, R. (2007). The english lexicon project. Behavior Research
Methods, 39(3), 445–459. https://doi.org/10.3758/BF03193014
Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013a). Random
effects structure for confirmatory hypothesis testing: Keep it maximal.
Journal of Memory and Language, 68(3), 255–278. https://doi.org/10.1016/j.jml.2012.11.001
Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013b). Random
effects structure for confirmatory hypothesis testing: Keep it maximal.
Journal of Memory and Language, 68, 255–278.
Bastiaansen, J. A., Kunkels, Y. K., Blaauw, F. J., Boker, S. M.,
Ceulemans, E., Chen, M., Chow, S.-M., Jonge, P. de, Emerencia, A. C.,
Epskamp, S., Fisher, A. J., Hamaker, E. L., Kuppens, P., Lutz, W.,
Meyer, M. J., Moulder, R., Oravecz, Z., Riese, H., Rubel, J., …
Bringmann, L. F. (2020). Time to get personal? The impact of researchers
choices on the selection of treatment targets using the experience
sampling methodology. Journal of Psychosomatic Research,
137, 110211. https://doi.org/10.1016/j.jpsychores.2020.110211
Bates, D., Kliegl, R., Vasishth, S., & Baayen, H. (2015).
Parsimonious mixed models. arXiv Preprint arXiv:1506.04967.
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting
linear mixed-effects models using lme4.
Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01
Belenky, G., Wesensten, N. J., Thorne, D. R., Thomas, M. L., Sing, H.
C., Redmond, D. P., Russo, M. B., & Balkin, T. J. (2003). Patterns
of performance degradation and restoration during sleep restriction and
subsequent recovery: a sleep dose-response study. Journal of Sleep
Research, 12(1), 1–12. https://doi.org/10.1046/j.1365-2869.2003.00337.x
Bolker, B. M., Brooks, M. E., Clark, C. J., Geange, S. W., Poulsen, J.
R., Stevens, M. H. H., & White, J.-S. S. (2009). Generalized linear
mixed models: a practical guide for ecology and evolution. Trends in
Ecology & Evolution, 24(3), 127–135. https://doi.org/10.1016/j.tree.2008.10.008
Bornstein, M. H., Jager, J., & Putnick, D. L. (2013). Sampling in
developmental science: Situations, shortcomings, solutions, and
standards. Developmental Review, 33(4), 357–370. https://doi.org/10.1016/j.dr.2013.08.003
Botvinik-Nezer, R., Holzmeister, F., Camerer, C. F., Dreber, A., Huber,
J., Johannesson, M., Kirchler, M., Iwanir, R., Mumford, J. A., Adcock,
R. A., Avesani, P., Baczkowski, B. M., Bajracharya, A., Bakst, L., Ball,
S., Barilari, M., Bault, N., Beaton, D., Beitner, J., … Schonberg, T.
(2020). Variability in the analysis of a single neuroimaging dataset by
many teams. Nature, 582(7810), 84–88. https://doi.org/10.1038/s41586-020-2314-9
Bourdieu, P. (2004). Science of Science and Reflexivity.
Polity.
Box, G. E. P. (1976). Science and statistics. Journal of the
American Statistical Association, 71(356), 791–799. https://doi.org/10.2307/2286841
Breznau, N., Rinke, E. M., Wuttke, A., Nguyen, H. H. V., Adem, M.,
Adriaans, J., Alvarez-Benjumea, A., Andersen, H. K., Auer, D., Azevedo,
F., Bahnsen, O., Balzer, D., Bauer, G., Bauer, P. C., Baumann, M.,
Baute, S., Benoit, V., Bernauer, J., Berning, C., … Żółtak, T. (2022).
Observing many researchers using the same data and hypothesis reveals a
hidden universe of uncertainty. Proceedings of the National Academy
of Sciences, 119(44), e2203150119. https://doi.org/10.1073/pnas.2203150119
Brosowsky, N., Parshina, O., Locicero, A., & Crump, M. (n.d.).
Teaching undergraduate students to read empirical articles: An
evaluation and revision of the QALMRI method. https://doi.org/10.31234/osf.io/p39sc
Brysbaert, M., & New, B. (2009). Moving beyond kucera and francis: A
critical evaluation of current word frequency norms and the introduction
of a new and improved word frequency measure for american english.
Behavior Research Methods, 41(4), 977–990. https://doi.org/10.3758/BRM.41.4.977
Bürkner, P.-C., & Vuorre, M. (2019). Ordinal Regression Models in
Psychology: A Tutorial. Advances in Methods and Practices in
Psychological Science, 2(1), 77–101. https://doi.org/10.1177/2515245918823199
Burnham, K. P. (2004). Multimodel inference: Understanding AIC and BIC
in model selection. Sociological Methods & Research,
33(2), 261–304. https://doi.org/10.1177/0049124104268644
Carp, J. (2012a). On the plurality of (methodological) worlds:
Estimating the analytic flexibility of FMRI experiments. Frontiers
in Neuroscience, 6, 149.
Carp, J. (2012b). The secret lives of experiments: Methods reporting in
the fMRI literature. Neuroimage, 63(1), 289–300.
Chang, W. (2013). R graphics cookbook. o’Reilly Media.
Christensen, R. H. B. (2015). Ordinal package for r. Version
3.4.2. 1–22. http://www.cran.r-project.org/package=ordinal/
Christensen, R. H. B. (2022). Ordinal: Regression models for ordinal
data. https://CRAN.R-project.org/package=ordinal
Clark, H. (Stanford. U. (1973).
Clark_1973_LanguageAsAFixedEffectFallacy.pdf.
Cohen, J. (1962). The statistical power of abnormal-social psychological
research: A review. Journal of Abnormal and Social Psychology,
65(3), 145–153. https://doi.org/10.1037/h0045186
Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003).
Applied multiple regression/correlation analysis for the behavioural
sciences (3rd. edition). Lawrence Erlbaum Associates.
Colenbrander, D., Miles, K. P., & Ricketts, J. (2019). To See or Not
to See: How Does Seeing Spellings Support Vocabulary Learning?
Language, Speech, and Hearing Services in Schools,
50(4), 609–628. https://doi.org/10.1044/2019_lshss-voia-18-0135
Crüwell, S., Apthorp, D., Baker, B. J., Colling, L., Elson, M., Geiger,
S. J., Lobentanzer, S., Monéger, J., Patterson, A., Schwarzkopf, D. S.,
Zaneva, M., & Brown, N. J. L. (n.d.). What’s in a
badge? A computational reproducibility investigation of the open data
badge policy in one issue of psychological science. https://doi.org/10.31234/osf.io/729qt
Davies, R. A. I., Birchenough, J. M. H., Arnell, R., Grimmond, D., &
Houlson, S. (2017). Reading through the life span: Individual
differences in psycholinguistic effects. Journal of Experimental
Psychology: Learning Memory and Cognition, 43(8). https://doi.org/10.1037/xlm0000366
Davies, R., Barbon, A., & Cuetos, F. (2013). Lexical and semantic
age-of-acquisition effects on word naming in spanish. Memory and
Cognition, 41(2), 297–311.
Davies, R., Barbón, A., & Cuetos, F. (2013). Lexical and semantic
age-of-acquisition effects on word naming in spanish. Memory and
Cognition, 41(2), 297–311. https://doi.org/10.3758/s13421-012-0263-8
Del Giudice, M., & Gangestad, S. W. (2021). A
Traveler’s Guide to the Multiverse: Promises, Pitfalls, and
a Framework for the Evaluation of Analytic Decisions. Advances in
Methods and Practices in Psychological Science, 4(1),
2515245920954925. https://doi.org/10.1177/2515245920954925
Dunlosky, J., & Lipko, A. R. (2007). Metacomprehension. Current
Directions in Psychological Science, 16(4), 228–232. https://doi.org/10.1111/j.1467-8721.2007.00509.x
Dutilh, G., Annis, J., Brown, S. D., Cassey, P., Evans, N. J., Grasman,
R. P. P. P., Hawkins, G. E., Heathcote, A., Holmes, W. R., Krypotos,
A.-M., Kupitz, C. N., Leite, F. P., Lerche, V., Lin, Y.-S., Logan, G.
D., Palmeri, T. J., Starns, J. J., Trueblood, J. S., Maanen, L. van, …
Donkin, C. (2019). The Quality of Response Time Data Inference: A
Blinded, Collaborative Assessment of the Validity of Cognitive Models.
Psychonomic Bulletin & Review, 26(4), 1051–1069.
https://doi.org/10.3758/s13423-017-1417-2
Eager, C., & Roy, J. (n.d.). Mixed effects models are sometimes
terrible. https://doi.org/10.48550/arXiv.1701.04858
Federer, L. M. (2022). Long-term availability of data associated with
articles in PLOS ONE. PLOS ONE, 17(8), e0272845. https://doi.org/10.1371/journal.pone.0272845
Fillard, P., Descoteaux, M., Goh, A., Gouttard, S., Jeurissen, B.,
Malcolm, J., Ramirez-Manzanares, A., Reisert, M., Sakaie, K., Tensaouti,
F., Yo, T., Mangin, J.-F., & Poupon, C. (2011). Quantitative
evaluation of 10 tractography algorithms on a realistic diffusion MR
phantom. NeuroImage, 56(1), 220–234. https://doi.org/10.1016/j.neuroimage.2011.01.032
Flake, J. K., & Fried, E. I. (2020). Measurement Schmeasurement:
Questionable Measurement Practices and How to Avoid Them. Advances
in Methods and Practices in Psychological Science, 3(4),
456–465. https://doi.org/10.1177/2515245920952393
Forster, K. I., & Forster, J. C. (2003). DMDX: A windows display
program with millisecond accuracy. Behavior Research Methods,
Instruments, & Computers, 35, 116–124.
Franconeri, S. L., Padilla, L. M., Shah, P., Zacks, J. M., &
Hullman, J. (2021). The Science of Visual Data Communication: What
Works. Psychological Science in the Public Interest,
22(3), 110–161. https://doi.org/10.1177/15291006211051956
Frederickson, N., Frith, U., & Reason, R. (1997). Phonological
assessment battery [PhAB]: Manual and test materials. nfer Nelson
Publishing Company Ltd.
Gabelica, M., Bojčić, R., & Puljak, L. (2022). Many researchers were
not compliant with their published data sharing statement: a
mixed-methods study. Journal of Clinical Epidemiology,
150, 33–41. https://doi.org/10.1016/j.jclinepi.2022.05.019
Gelman, A. (2014). The Connection Between Varying Treatment Effects and
the Crisis of Unreplicable Research. Journal of Management,
41(2), 632–643. https://doi.org/10.1177/0149206314525208
Gelman, a. (2015a). The connection between varying treatment effects and
the crisis of unreplicable research: A bayesian perspective. Journal
of Management, 41(2), 632–643. https://doi.org/10.1177/0149206314525208
Gelman, a. (2015b). The connection between varying treatment effects and
the crisis of unreplicable research: A bayesian perspective. Journal
of Management, 41(2), 632–643. https://doi.org/10.1177/0149206314525208
Gelman, A., & Hennig, C. (2017). Beyond subjective and objective in
statistics. Journal of the Royal Statistical Society: Series A
(Statistics in Society), 180(4), 967–1033.
Gelman, A., & Hill, J. (2007a). Data analysis using regression
and multilevel/hierarchical models. Cambridge University Press.
Gelman, A., & Hill, J. (2007b). Data analysis using regression
and multilevel/hierarchical models. Cambridge University Press.
Gelman, A., & Loken, E. (2014a). The garden of forking paths: Why
multiple comparisons can be a problem, even when there is no
“fishing expedition” or
“p-hacking” and the research hypothesis was
posited ahead of time. Psychological Bulletin, 140(5),
1272–1280.
Gelman, A., & Loken, E. (2014b). The statistical crisis in science.
American Scientist, 102(6), 460–465. https://doi.org/10.1511/2014.111.460
Gelman, A., & Unwin, A. (2013). Infovis and Statistical Graphics:
Different Goals, Different Looks. Journal of Computational and
Graphical Statistics, 22(1), 2–28. https://doi.org/10.1080/10618600.2012.761137
Gelman, A., & Weakliem, D. (2009). Of beauty, sex and power.
American Scientist, 97(4), 310–316. https://doi.org/10.1511/2009.79.310
Gilmore, R. O., Diaz, M. T., Wyble, B. A., & Yarkoni, T. (2017).
Progress toward openness, transparency, and reproducibility in cognitive
neuroscience. Annals of the New York Academy of Sciences,
1396, 5–18. https://doi.org/10.1111/nyas.13325
Goldstein, H. (1995). Multilevel statistical models. Edward
Arnold.
Golino, H., & Gomes, C. (2014). Psychology data from the
“BAFACALO project: The Brazilian Intelligence Battery based
on two state-of-the-art models Carroll’s
Model and the CHC model”. Journal of Open Psychology
Data, 2(1), e6. https://doi.org/10.5334/jopd.af
Goodman, S. N., Fanelli, D., & Ioannidis, J. P. A. (2016). What does
research reproducibility mean? Science Translational Medicine,
8(341).
Grolemund, G., & Wickham, H. (n.d.). R for Data Science
[Book]. https://www.oreilly.com/library/view/r-for-data/9781491910382/
Hardwicke, T. E., Bohn, M., MacDonald, K., Hembacher, E., Nuijten, M.
B., Peloquin, B. N., deMayo, B. E., Long, B., Yoon, E. J., & Frank,
M. C. (n.d.). Analytic reproducibility in articles receiving open data
badges at the journal psychological science: An observational study.
Royal Society Open Science, 8(1), 201494. https://doi.org/10.1098/rsos.201494
Hardwicke, T. E., Mathur, M. B., MacDonald, K., Nilsonne, G., Banks, G.
C., Kidwell, M. C., Hofelich Mohr, A., Clayton, E., Yoon, E. J., Henry
Tessler, M., Lenne, R. L., Altman, S., Long, B., & Frank, M. C.
(2018). Data availability, reusability, and analytic reproducibility:
evaluating the impact of a mandatory open data policy at the journal
Cognition. Royal Society Open Science, 5(8), 180448.
https://doi.org/10.1098/rsos.180448
Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest
people in the world? The Behavioral and Brain Sciences,
33(2-3). https://doi.org/10.1017/S0140525X0999152X
Herndon, T., Ash, M., & Pollin, R. (2014). Does high public debt
consistently stifle economic growth? A critique of Reinhart and Rogoff.
Cambridge Journal of Economics, 38(2), 257–279. https://doi.org/10.1093/cje/bet075
Hoekstra, R., Morey, R. D., Rouder, J. N., & Wagenmakers, E.-J.
(2014). Robust misinterpretation of confidence intervals.
Psychonomic Bulletin & Review, 21(5), 1157–1164.
https://doi.org/10.3758/s13423-013-0572-3
Hoffmann, S., Schönbrodt, F., Elsas, R., Wilson, R., Strasser, U., &
Boulesteix, A.-L. (n.d.). The multiplicity of analysis strategies
jeopardizes replicability: Lessons learned across disciplines. Royal
Society Open Science, 8(4), 201925. https://doi.org/10.1098/rsos.201925
Howell, D. C. (2016). Fundamental statistics for the behavioral
sciences. Cengage learning.
Ioannidis, J. P. a. (2005). Why most published research findings are
false. PLoS Medicine, 2(8), 0696–0701. https://doi.org/10.1371/journal.pmed.0020124
Jaeger, T. F. (2008). Categorical data analysis: Away from ANOVAs
(transformation or not) and towards logit mixed models. Journal of
Memory and Language, 59(4), 434–446. https://doi.org/10.1016/j.jml.2007.11.007
John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the
prevalence of questionable research practices with incentives for truth
telling. Psychological Science, 23(5), 524–532. https://doi.org/10.1177/0956797611430953
Judd, C. M., Westfall, J., & Kenny, D. A. (2012). Treating
stimuli as a random factor in social psychology : A new and
comprehensive solution to a pervasive but largely ignored problem.
103(1), 54–69. https://doi.org/10.1037/a0028347
Kidwell, M. C., Lazarević, L. B., Baranski, E., Hardwicke, T. E.,
Piechowski, S., Falkenberg, L. S., Kennett, C., Slowik, A., Sonnleitner,
C., Hess-Holden, C., Errington, T. M., Fiedler, S., & Nosek, B. A.
(2016). Badges to acknowledge open practices: A simple, low-cost,
effective method for increasing transparency. PLoS Biology,
14(5), 1–15. https://doi.org/10.1371/journal.pbio.1002456
Klau, S., Hoffmann, S., Patel, C. J., Ioannidis, J. P., &
Boulesteix, A.-L. (2021). Examining the robustness of observational
associations to model, measurement and sampling uncertainty with the
vibration of effects framework. International Journal of
Epidemiology, 50(1), 266–278. https://doi.org/10.1093/ije/dyaa164
Klau, S., Schönbrodt, F., Patel, C. J., Ioannidis, J., Boulesteix,
A.-L., & Hoffmann, S. (n.d.). Comparing the vibration of effects
due to model, data pre-processing and sampling uncertainty on a large
data set in personality psychology. https://doi.org/10.31234/osf.io/c7v8b
Kosslyn, S. M., & Rosenberg, R. S. (2005). Fundamentals of
psychology: The brain, the person, the world, 2nd ed. Pearson
Education New Zealand.
Kreft, I., & Leeuw, J. de. (1998). Introducing multilevel
modeling (D. Wright, Ed.). Sage Publications.
Kuhn, T. S. (1970). The structure of scientific revolutions
([2d ed., enl). University of Chicago Press.
Landy, J. F., Jia, M. L., Ding, I. L., Viganola, D., Tierney, W.,
Dreber, A., Johannesson, M., Pfeiffer, T., Ebersole, C. R., Gronau, Q.
F., Ly, A., Bergh, D. van den, Marsman, M., Derks, K., Wagenmakers,
E.-J., Proctor, A., Bartels, D. M., Bauman, C. W., Brady, W. J., …
Uhlmann, E. L. (2020). Crowdsourcing hypothesis tests: Making
transparent how design choices shape research results. Psychological
Bulletin, 146(5), 451–479. https://doi.org/10.1037/bul0000220
Liddell, T. M., & Kruschke, J. K. (2018). Analyzing ordinal data
with metric models: What could possibly go wrong? Journal of
Experimental Social Psychology, 79, 328–348. https://doi.org/10.1016/j.jesp.2018.08.009
Loo, M. van der, Laan, J. van der, R Core Team, Logan, N., Muir, C.,
Gruber, J., & Ripley, B. (2022). Stringdist: Approximate string
matching, fuzzy text search, and string distance functions. https://CRAN.R-project.org/package=stringdist
Lorch, R. F., & Myers, J. L. (1990a). Regression analyses of
repeated measures data in cognitive research. Journal of
Experimental Psychology: Learning, Memory and Cognition,
16(1), 149–157.
Lorch, R. F., & Myers, J. L. (1990b). Regression analyses of
repeated measures data in cognitive research. Journal of
Experimental Psychology: Learning, Memory, and Cognition,
16(1), 149–157. https://doi.org/10.1037/0278-7393.16.1.149
Lubega, N., Anderson, A., & Nelson, N. (n.d.). Experience of
irreproducibility as a risk factor for poor mental health in biomedical
science doctoral students: A survey and interview-based study. https://doi.org/10.31222/osf.io/h37kw
Maier-Hein, K. H., Neher, P. F., Houde, J.-C., Côté, M.-A.,
Garyfallidis, E., Zhong, J., Chamberland, M., Yeh, F.-C., Lin, Y.-C.,
Ji, Q., Reddick, W. E., Glass, J. O., Chen, D. Q., Feng, Y., Gao, C.,
Wu, Y., Ma, J., He, R., Li, Q., … Descoteaux, M. (2017). The challenge
of mapping the human connectome based on diffusion tractography.
Nature Communications, 8(1), 1349. https://doi.org/10.1038/s41467-017-01285-x
Masterson, J., & Hayes, M. (2007). Development and data for UK
versions of an author and title recognition test for adults. Journal
of Research in Reading, 30, 212–219.
Matuschek, H., Kliegl, R., Vasishth, S., Baayen, H., & Bates, D.
(2017). Balancing type i error and power in linear mixed models.
Journal of Memory and Language, 94, 305–315. https://doi.org/10.1016/j.jml.2017.01.001
McElreath, R. (2020). : A bayesian course with examples in r and
STAN (2nd ed.). Chapman; Hall/CRC. https://doi.org/10.1201/9780429029608
Meehl, P. E. (1967). Theory-testing in psychology and physics: A
methodological paradox. Philosophy of Science, 34(2),
103–115.
Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir
karl, sir ronald, and the slow progress of soft psychology.
46(September 1976), 806–834.
Meteyard, L., & Davies, R. A. I. (2020a). Best practice guidance for
linear mixed-effects models in psychological science. Journal of
Memory and Language, 112. https://doi.org/10.1016/j.jml.2020.104092
Meteyard, L., & Davies, R. A. I. (2020b). Best practice guidance for
linear mixed-effects models in psychological science. Journal of
Memory and Language, 112, 104092. https://doi.org/10.1016/j.jml.2020.104092
Minocher, R., Atmaca, S., Bavero, C., McElreath, R., & Beheim, B.
(n.d.). Estimating the reproducibility of social learning research
published between 1955 and 2018. Royal Society Open Science,
8(9), 210450. https://doi.org/10.1098/rsos.210450
Monaghan, P., Mattock, K., Davies, R. A. I., & Smith, A. C. (2015).
Gavagai is as gavagai does: Learning nouns and verbs from
cross-situational statistics. Cognitive Science,
39(5), 1099–1112. https://doi.org/10.1111/cogs.12186
Mousikou, P., Sadat, J., Lucas, R., & Rastle, K. (2017). Moving
beyond the monosyllable in models of skilled reading: Mega-study of
disyllabic nonword reading. Journal of Memory and Language,
93, 169–192. https://doi.org/10.1016/j.jml.2016.09.003
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers,
C. D., Percie Du Sert, N., Simonsohn, U., Wagenmakers, E. J., Ware, J.
J., & Ioannidis, J. P. A. (2017). A manifesto for reproducible
science. Nature Human Behaviour, 1(1), 1–9. https://doi.org/10.1038/s41562-016-0021
Nosek, B. A., Beck, E. D., Campbell, L., Flake, J. K., Hardwicke, T. E.,
Mellor, D. T., van?t Veer, A. E., & Vazire, S. (2019).
Preregistration is hard, and worthwhile. Trends in Cognitive
Sciences, 23(10), 815–818.
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T.
(2018). The preregistration revolution. Proceedings of the National
Academy of Sciences, 115(11), 2600–2606.
Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S.,
Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M. B.,
Rohrer, J. M., Romero, F., Scheel, A. M., Scherer, L. D., Schönbrodt, F.
D., & Vazire, S. (2022). Replicability, Robustness, and
Reproducibility in Psychological Science. Annual Review of
Psychology, 73, 719–748. https://doi.org/10.1146/annurev-psych-020821-114157
Nosek, B. A., & Lakens, D. (2014). Registered reports: A method to
increase the credibility of published results. Social
Psychology, 45(3), 137–141. https://doi.org/10.1027/1864-9335/a000192
Obels, P., Lakens, D., Coles, N. A., Gottfried, J., & Green, S. A.
(2020). Analysis of open data and computational reproducibility in
registered reports in psychology. Advances in Methods and Practices
in Psychological Science, 3(2), 229–237. https://doi.org/10.1177/2515245920918872
Parsons, S. (n.d.). Exploring reliability heterogeneity with
multiverse analyses: Data processing decisions unpredictably influence
measurement reliability. https://doi.org/10.31234/osf.io/y6tcz
Pashler, H., & Harris, C. (2012). Is the replicability crisis
overblown? Three arguments examined. Perspectives on Psychological
Science, 7(6), 531–536. https://doi.org/10.1177/1745691612463401
Pashler, H., & Wagenmakers, E. J. (2012). Editors’ introduction to
the special section on replicability in psychological science: A crisis
of confidence? Perspectives on Psychological Science,
7(6), 528–530. https://doi.org/10.1177/1745691612465253
Patel, C. J., Burford, B., & Ioannidis, J. P. A. (2015). Assessment
of vibration of effects due to model specification can demonstrate the
instability of observational associations. Journal of Clinical
Epidemiology, 68(9), 1046–1058. https://doi.org/10.1016/j.jclinepi.2015.05.029
Pinheiro, J. C., & Bates, D. M. (2000a). Mixed-effects models in
s and s-plus (statistics and computing). Springer.
Pinheiro, J. C., & Bates, D. M. (2000b). Mixed-effects models in
s and s-plus (statistics and computing). Springer.
Poline, J.-B., Strother, S. C., Dehaene-Lambertz, G., Egan, G. F., &
Lancaster, J. L. (2006). Motivation and synthesis of the FIAC
experiment: Reproducibility of fMRI results across expert analyses.
Human Brain Mapping, 27(5), 351–359. https://doi.org/10.1002/hbm.20268
Preston, C. C., & Colman, A. M. (2000). Optimal number of response
categories in rating scales: Reliability, validity, discriminating
power, and respondent preferences. Acta Psychologica,
104(1), 1–15. https://doi.org/10.1016/S0001-6918(99)00050-5
Raaijmakers, J. G. W., Schrijnemakers, J. M. C., & Gremmen, F.
(1999). How to deal with "the language-as-fixed-effect
fallacy": Common misconceptions and alternative solutions.
Journal of Memory and Language, 41(3), 416–426. https://doi.org/10.1006/jmla.1999.2650
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear
models: Applications and data analysis methods (Vol. 1). sage.
Ricketts, J., Dawson, N., & Davies, R. (2021). The hidden depths of
new word knowledge: Using graded measures of orthographic and semantic
learning to measure vocabulary acquisition. Learning and
Instruction, 74, 101468. https://doi.org/10.1016/j.learninstruc.2021.101468
Roche, D. G., Kruuk, L. E. B., Lanfear, R., & Binning, S. A. (2015).
Public data archiving in ecology and evolution: How well are we doing?
PLoS Biology, 13(11), 1–12. https://doi.org/10.1371/journal.pbio.1002295
Rodríguez-Ferreiro, J., Aguilera, M., & Davies, R. (2020a). Positive
schizotypy increases the acceptance of unpresented materials in false
memory tasks in non-clinical individuals. Frontiers in
Psychology, 11. https://doi.org/10.3389/fpsyg.2020.00262
Rodríguez-Ferreiro, J., Aguilera, M., & Davies, R. (2020b). Semantic
priming and schizotypal personality: reassessing the link between
thought disorder and enhanced spreading of semantic activation.
PeerJ, 8, e9511. https://doi.org/10.7717/peerj.9511
Salganik, M. J., Lundberg, I., Kindel, A. T., Ahearn, C. E., Al-Ghoneim,
K., Almaatouq, A., Altschul, D. M., Brand, J. E., Carnegie, N. B.,
Compton, R. J., Datta, D., Davidson, T., Filippova, A., Gilroy, C.,
Goode, B. J., Jahani, E., Kashyap, R., Kirchner, A., McKay, S., …
McLanahan, S. (2020). Measuring the predictability of life outcomes with
a scientific mass collaboration. Proceedings of the National Academy
of Sciences, 117(15), 8398–8403. https://doi.org/10.1073/pnas.1915006117
Scheel, A. M. (2022). Why most psychological research findings are not
even wrong. Infant and Child Development, 31(1),
e2295. https://doi.org/10.1002/icd.2295
Scheel, A. M., Tiokhin, L., Isager, P. M., & Lakens, D. (2021). Why
Hypothesis Testers Should Spend Less Time Testing Hypotheses.
Perspectives on Psychological Science, 16(4), 744–755.
https://doi.org/10.1177/1745691620966795
Schweinsberg, M., Feldman, M., Staub, N., Akker, O. R. van den, Aert, R.
C. M. van, Assen, M. A. L. M. van, Liu, Y., Althoff, T., Heer, J., Kale,
A., Mohamed, Z., Amireh, H., Venkatesh Prasad, V., Bernstein, A.,
Robinson, E., Snellman, K., Amy Sommer, S., Otner, S. M. G., Robinson,
D., … Luis Uhlmann, E. (2021). Same data, different conclusions: Radical
dispersion in empirical results when independent analysts operationalize
and test the same hypothesis. Organizational Behavior and Human
Decision Processes, 165, 228–249. https://doi.org/10.1016/j.obhdp.2021.02.003
Sedlmeier, P., & Gigerenzer, G. (1989). Statistical power studies.
Psychological Bulletin, 105(2), 309–316.
Shipley, W. C., Gruber, C. P., Martin, T. A., & Klein, A. M. (2009).
Shipley-2 manual western psychological services. Western
Psychological Services, 65.
Silberzahn, R., & Uhlmann, E. L. (2015). Crowdsourced research: Many
hands make tight work. Nature, 526(7572), 189–191. https://doi.org/10.1038/526189a
Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F.,
Awtrey, E., Bahník, Š., Bai, F., Bannard, C., Bonnier, E., Carlsson, R.,
Cheung, F., Christensen, G., Clay, R., Craig, M., Dalla Rosa, A., Dam,
L., Evans, M., Flores Cervantes, I., … Nosek, B. (2017). Many analysts,
one dataset: Making transparent how variations in analytical choices
affect results. Advances in Methods and Practices in Psychological
Science. https://doi.org/10.31234/osf.io/qkwst
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011a).
False-positive psychology: Undisclosed flexibility in data collection
and analysis allows presenting anything as significant.
Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011b).
False-positive psychology: Undisclosed flexibility in data collection
and analysis allows presenting anything as significant.
Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632
Snijders, T. A. B., & Bosker, R. J. (2004). Multilevel analysis:
An introduction to basic and advanced multilevel modeling. Sage
Publications Ltd.
Stainthorp, R. (1997). A children’s author recognition
test: A useful tool in reading research. Journal of Research in
Reading, 20(2), 148158.
Starns, J. J., Cataldo, A. M., Rotello, C. M., Annis, J., Aschenbrenner,
A., Bröder, A., Cox, G., Criss, A., Curl, R. A., Dobbins, I. G., Dunn,
J., Enam, T., Evans, N. J., Farrell, S., Fraundorf, S. H., Gronlund, S.
D., Heathcote, A., Heck, D. W., Hicks, J. L., … Wilson, J. (2019).
Assessing Theoretical Conclusions With Blinded Inference to Investigate
a Potential Inference Crisis. Advances in Methods and Practices in
Psychological Science, 2(4), 335–349. https://doi.org/10.1177/2515245919869583
Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016a).
Increasing transparency through a multiverse analysis. Perspectives
on Psychological Science, 11(5), 702–712.
Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016b).
Increasing transparency through a multiverse analysis. Perspectives
on Psychological Science, 11(5), 702–712.
Tedersoo, L., Küngas, R., Oras, E., Köster, K., Eenmaa, H., Leijen, Ä.,
Pedaste, M., Raju, M., Astapova, A., Lukner, H., Kogermann, K., &
Sepp, T. (2021). Data sharing practices and data availability upon
request differ across scientific disciplines. Scientific Data,
8(1), 192. https://doi.org/10.1038/s41597-021-00981-0
Torgesen, J. K., Rashotte, C. A., & Wagner, R. K. (1999). TOWRE:
Test of word reading efficiency. Pro-ed Austin, TX.
Towse, J. N., Ellis, D. A., & Towse, A. S. (2021). Opening Pandora’s
Box: Peeking inside Psychology’s data sharing practices, and seven
recommendations for change. Behavior Research Methods,
53(4), 1455–1468. https://doi.org/10.3758/s13428-020-01486-1
Ulrich, R., & Miller, J. (1994). Effects of truncation on reaction
time analysis. Journal of Experimental Psychology: General,
123, 34–80.
Vankov, I., Bowers, J., & Munafò, M. R. (2014). On the persistence
of low power in psychological science. Quarterly Journal of
Experimental Psychology, 67(5), 1037–1040. https://doi.org/10.1080/17470218.2014.885986
Vasishth, S., & Gelman, A. (2021). How to embrace variation and
accept uncertainty in linguistic and psycholinguistic data analysis.
Linguistics, 59(5), 1311–1342. https://doi.org/10.1515/ling-2019-0051
Vazire, S. (2018). Implications of the Credibility Revolution for
Productivity, Creativity, and Progress. Perspectives on
Psychological Science, 13(4), 411–417. https://doi.org/10.1177/1745691617751884
Wagenmakers, E.-J., Sarafoglou, A., & Aczel, B. (2022). One
statistical analysis must not rule them all. Nature,
605(7910), 423–425. https://doi.org/10.1038/d41586-022-01332-8
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., & Maas, H. L. J. van
der. (2011). Why psychologists must change the way they analyze their
data: The case of psi: Comment on bem (2011). Journal of Personality
and Social Psychology, 100(3), 426–432. https://doi.org/10.1037/a0022790
Wessel, I., Albers, C., Zandstra, A. R. E., & Heininga, V. E.
(2020). A multiverse analysis of early attempts to replicate memory
suppression with the think/no-think task.
Wicherts, J. M., Borsboom, D., Kats, J., & Molenaar, D. (2006). The
poor availability of psychological research data for reanalysis.
American Psychologist, 61(7), 726–728. https://doi.org/10.1037/0003-066X.61.7.726
Wickham, H. (2016). ggplot2: Elegant graphics for data
analysis. Springer-Verlag New York. https://ggplot2.tidyverse.org
Wickham, H. (2017). Tidyverse: Easily install and load the
’tidyverse’. https://cran.r-project.org/package=tidyverse
Wickham, H., & Grolemund, G. (2016). R for data science: Import,
tidy, transform, visualize, and model data. " O’Reilly
Media, Inc.".
Wild, H., Kyröläinen, A.-J., & Kuperman, V. (2022). How
representative are student convenience samples? A study of literacy and
numeracy skills in 32 countries. PLOS ONE, 17(7),
e0271191. https://doi.org/10.1371/journal.pone.0271191
Wilke, C. O. (n.d.). Fundamentals of data visualization. https://clauswilke.com/dataviz/
Wilkinson, L. (2013). The Grammar of Graphics. Springer Science
& Business Media.
Yarkoni, T. (2022). The generalizability crisis. Behavioral and
Brain Sciences, 45, e1. https://doi.org/10.1017/S0140525X20001685
Yarkoni, T., Balota, D., & Yap, M. (2008). Moving beyond coltheart’s
n: A new measure of orthographic similarity. Psychonomic Bulletin
& Review, 15(5), 971–979. https://doi.org/10.3758/PBR.15.5.971
Young, C. (2018). Model uncertainty and the crisis in science.
Socius, 4, 2378023117737206.
Young, C., & Holsteen, K. (2017). Model Uncertainty and Robustness:
A Computational Framework for Multimodel Analysis. Sociological
Methods & Research, 46(1), 3–40. https://doi.org/10.1177/0049124115610347