Citation

Ashby FG, O'Brien JB. Psychon. Bull. Rev. 2008; 15(1): 16-27.

Affiliation

Department of Psychology, University of California, Santa Barbara, California 93106, USA. ashby@psych.ucsb.edu

Copyright

(Copyright © 2008, Psychonomic Society Publications)

DOI

unavailable

PMID

18605475

Abstract

In traditional statistical methodology (e.g., the ANOVA), confidence in the observed results is often assessed by computing thep value or the power of the test. In most cases, adding more participants to a study will improve these measures more than will increasing the amount of data collected from each participant. Thus, traditional statistical methods are biased in favor of experiments with large numbers of participants. This article proposes a method for computing confidence in the results of experiments in which data are collected from a few participants over many trials. In such experiments, it is common to fit a series of mathematical models to the resulting data and to conclude that the best-fitting model is superior. The probability of replicating this result (i.e., Prep) is derived for any two nested models. Simulations and empirical applications of this new statistic confirm its utility in studies in which data are collected from a few participants over many trials.

Language: en