Stats for
Menu

t-Tests · General Linear Model

Bayesian Independent-Samples t-Test

A Bayesian t-test weighs the evidence for and against two groups differing.

A Bayesian independent-samples t-test takes one continuous outcome from two separate groups and weighs how much the data favor a difference between the population means over none, or the reverse.

Example

Do students who take lecture notes by hand remember more than students who type them, or do both remember about the same? One hundred undergraduates were randomly assigned to take notes on a recorded lecture by hand or on a laptop, 50 each, then took a 30-question quiz on it the next day.

Two Independent Groups + Continuous Outcome → Bayesian Independent-Samples t-Test
Tests
Evidence for a difference between two population means versus none, and the standardized difference δ
Observations
Independent
Parametric
Yes
Headline Statistic
BF_{01}
Effect Size
Cohen's d
Groups
2

The Basics

When to Use It

Use it when you have two separate groups of different people, one continuous outcome, and you want to weigh the evidence for a difference between their averages against the evidence for none. It earns its keep when "no difference" is a real possibility you'd like to be able to support, not just fail to reject: a replication, a cheaper treatment that might work as well, or a claim that two versions of a task give the same average. For example:

  • Do students who take lecture notes by hand remember more than students who type them, or do both remember about the same?
  • Do people primed with words about old age walk more slowly down a hallway than people primed with neutral words, or at the same speed?
  • Do patients in a group-therapy format end treatment with different depression scores than patients in individual therapy, or similar ones?

The test compares two hypotheses: the null, that the population means are equal, and an alternative that says how big a difference might be. The Bayes factor says which of them predicted the data better, and by how much.

Vocabulary You Need

Definition Bayes Factor (BF_{10})

A ratio comparing how well two hypotheses predicted the observed data: each hypothesis's marginal likelihood (how well it predicted the data, averaged over its prior distribution), divided by the other's. For continuous data these are predictive densities (heights of a curve at the observed data), not probabilities of those exact observations. BF₁₀ puts the alternative on top and BF₀₁ = 1 / BF₁₀ puts the null on top, so BF₁₀ = 5 and BF₀₁ = 5 are equally strong, in opposite directions. Values near 1 mean the data can't tell the hypotheses apart. Unlike a p-value, it can show evidence for the null, not just a failure to reject it. It depends on the prior distributions, so report which ones you used. Glossary →

Definition Population Effect Size (\delta)

The difference between two population means divided by the standard deviation within each population (assumed the same in both): δ = (μ₁ − μ₂) / σ. It's what Cohen's d estimates from a sample. Bayesian t-tests put their prior on δ rather than on the raw difference, so the same default prior can be used whatever scale the outcome was measured on. Like d, it still depends on how varied the population is and how reliably the outcome is measured. Glossary →

Definition Prior Distribution

A probability distribution that says which values of a parameter, such as the size of an effect, a hypothesis considers plausible before the data are in. In a Bayes factor it's how each hypothesis makes its predictions: a narrow prior bets on small effects, a wide one spreads its bets over large effects too. The Bayes factor depends on it, so the prior is chosen before the analysis, usually a published default, and results are checked with other reasonable priors. Glossary →

Definition Posterior Distribution

The prior distribution updated by the data: each possible value of a parameter is reweighted by how well it predicted what was observed. It's usually summarized by its median and a credible interval. In a Bayesian t-test the posterior for δ is worked out under the alternative hypothesis, so it describes how big the effect might be if there is one; whether there is one is the Bayes factor's job. Glossary →

Assumptions and Inference

1. Independent Students (for the Model) Each student's score is unconnected to every other student's, within and between the groups.
What It Means
Each person's data comes from a separate, unconnected source, so knowing one person's scores tells you nothing about anyone else's. In designs that measure people more than once, their own scores are meant to be related; it's different people who must be independent. Methods built for clustered data, such as multilevel models and GEE, also allow members of the same group to be related, as long as the analysis names the grouping; then it's the groups that must be independent.
Applies To
Observations
How to Check
By design, not by a test. Did each person contribute their own data, with no clusters such as roommates, couples, therapy groups or classrooms whose members influence each other? If there are clusters, does the analysis model every one of them?
How Much It Matters
A lot. Students who took notes side by side or compared answers before the quiz score more alike than the model expects, and the Bayes factor overstates the evidence in whichever direction it points.

What to Do

  • Alternative Test Multilevel Regression (HLM). A multilevel model, Bayesian or classical, which accounts for the clustering.
  • Design Change Make sure each person contributes one score, to one group only, and is tested separately.
2. Normality in Each Group (for the Model) The default Bayesian t-test models each group's scores as normal, and its Bayes factor is conditional on that model. Moderate departures may matter little, especially with bigger groups, but strong skew, heavy tails or extreme outliers can change the Bayes factor. Look at both groups' distributions.
What It Means
Going Bayesian doesn't make a wrong model harmless. The Bayes factor compares two versions of one model, normal scores with a common SD in each group, that differ only in what they say about δ. If that model fits the scores badly, both versions predict them badly, and the ratio can mislead. When the scores clearly aren't normal, a model with a better-suited likelihood (for example, a skewed or heavy-tailed one) is the fix, not a different prior.
Applies To
Residuals
How to Check
Look at a histogram or Q–Q plot of each group's scores for strong skew or extreme outliers. Don't rely on a normality test: with small samples it misses real problems, and with large ones it flags trivial ones. In the note-taking study both groups look roughly symmetric; the lowest score, a laptop student's 7, is about 3 standard deviations below that group's mean.
How Much It Matters
The Bayes factor comes from the same t statistic as the classical test, so whatever moves t moves it: outliers pull the means and inflate the spread. With 50 per group, moderate skew is a smaller worry than a few wild scores; check whether one or two of them drive the result. Here, dropping the 7 moves BF₀₁ from 4.02 to 4.52, the same conclusion.

What to Do

  • Transformation Log-transform right-skewed outcomes such as reaction times. Results are then on the log scale.
  • Alternative Test Mann-Whitney U Test. JASP's Bayesian version of this rank test gives a Bayes factor that outliers barely move. It compares how often one group outscores the other, not the means, and has its own model and prior, so the question and the number both change.
  • Proceed with Caveat Rerun the analysis without the extreme scores and report both Bayes factors if they differ.
3. Equal Variances (for the Model) The default JZS t-test assumes the two populations share one SD. Unlike Welch's classical t-test, it doesn't allow for unequal spreads; if they differ a lot, use a Bayesian model with a separate SD for each group.
What It Means
The outcome varies about as much within each group's population. Tests that pool the groups' variances into one estimate, like Student's t-test and the classic ANOVA, rely on it.
Applies To
Variances
How to Check
Compare the groups' standard deviations and look at side-by-side box plots. In the note-taking study the SDs are 4.00 and 4.20, nearly equal.
How Much It Matters
The default Bayesian t-test pools the two groups' variances, like Student's t-test, so it shares that test's weakness: unequal spreads matter most when the groups also differ in size. With equal groups, as here, it matters much less.

What to Do

  • Design Change Keep the groups the same size, which makes pooling far less risky.
  • Alternative Test Fit a model that gives each group its own SD (in R, brms with bf(score ~ group, sigma ~ group)) and read the posterior for the difference. A Bayes factor from such a model needs its own priors and method, so its number isn't comparable with the default one.
  • Proceed with Caveat Run Welch's t-test alongside. It can flag a difference the pooled model hides, but a non-significant Welch p doesn't back up the Bayes factor's evidence for no difference.

Limitations

This Is Not Your Test If…

What It Doesn't Tell You

  • That the methods are identical. A BF₀₁ of 4.02 says "no difference" predicted the quiz scores better than the default alternative did. A difference too small for 100 students to detect, a tenth of an SD say, is still possible. To show a difference is too small to matter, set a smallest difference of interest in advance and use an equivalence test or an interval null.
  • Anything prior-free. The evidence depends on the prior. Wider priors favor no difference more strongly (BF₀₁ = 5.46 at r = 1, 7.54 at r = 1.414; see the prior-sensitivity table in How It's Reported); a prior concentrated on small differences favors it less, and at r = 0.3 the evidence is inconclusive (BF₀₁ = 2.15).
  • Other kinds of learning. The quiz asked about one recorded lecture, the next day. Studying from the notes later, longer lectures or questions that need more than recall could all turn out differently.
  • Who else. The result describes students like these 100 volunteers, taking notes the way they were told to in a lab, not in their own courses.

Common Mistakes

  • Don'tRead a non-significant p as evidence that the two groups are the same.

    InsteadReport a Bayes factor, which can show evidence for no difference, or show that the data can't tell.

  • Don'tCall a Bayes factor between 1/3 and 3 evidence for either hypothesis.

    InsteadCall it inconclusive, and say more data are needed.

  • Don'tReport a Bayes factor without its prior.

    InsteadName the prior and its scale (a Cauchy prior with r = 0.707), and show how the result changes with wider and narrower ones.

  • Don'tChoose the prior scale after seeing which one gives the result you want.

    InsteadFix it before the analysis, usually the default, and report a robustness check with other scales.

  • Don'tSwitch to a one-sided Bayes factor after seeing which way the data point.

    InsteadChoose a one-sided Bayes factor only if you specified the direction before collecting data and have a defensible reason why an effect in the opposite direction wouldn't count toward your claim. Otherwise, report the two-sided Bayes factor.

  • Don'tTest for a difference by checking whether the credible interval includes 0.

    InsteadUse the Bayes factor to compare no difference with a difference. This credible interval assumes a difference exists and only says how big it might be. Interval-based decisions, such as a region of practical equivalence, are a separate procedure you'd have to define in advance.

  • Don'tRead a Bayes factor as the probability that a hypothesis is true.

    InsteadRead it as how many times better one hypothesis predicted the data than the other. Turning it into a probability needs prior odds too.

Result Decoder

Evidence and Effect Size Guidelines

Benchmarks Strength of Evidence (Bayes Factor)
  • BF₁₀ < 1/100 Extreme for H₀

    Extreme evidence for the null hypothesis.

  • 1/100 ≤ BF₁₀ < 1/30 Very Strong for H₀

    Very strong evidence for the null hypothesis.

  • 1/30 ≤ BF₁₀ < 1/10 Strong for H₀

    Strong evidence for the null hypothesis.

  • 1/10 ≤ BF₁₀ < 1/3 Moderate for H₀

    Moderate evidence for the null hypothesis.

  • 1/3 ≤ BF₁₀ < 1 Anecdotal for H₀

    Barely worth mentioning. Treat it as inconclusive.

  • 1 ≤ BF₁₀ < 3 Anecdotal for H₁

    Barely worth mentioning. Treat it as inconclusive.

  • 3 ≤ BF₁₀ < 10 Moderate for H₁

    Moderate evidence for the alternative hypothesis.

  • 10 ≤ BF₁₀ < 30 Strong for H₁

    Strong evidence for the alternative hypothesis.

  • 30 ≤ BF₁₀ < 100 Very Strong for H₁

    Very strong evidence for the alternative hypothesis.

  • BF₁₀ ≥ 100 Extreme for H₁

    Extreme evidence for the alternative hypothesis.

Labels from Lee and Wagenmakers, adjusted from Jeffreys (1961). They are a rough heuristic, not cutoffs; Kass and Raftery (1995) proposed coarser ones. For BF₀₁, flip the scale: BF₀₁ = 5 is BF₁₀ = 1/5.

More about strength of evidence ↓

Benchmarks Effect Size (Cohen's d)
  • |d| < 0.2 Below Small

    Smaller than the conventional small benchmark. Whether an effect this size matters depends on the setting.

  • 0.2 ≤ |d| < 0.5 Small

    A relatively modest standardized difference

  • 0.5 ≤ |d| < 0.8 Medium

    A moderate standardized difference

  • |d| ≥ 0.8 Large

    A relatively substantial standardized difference

The sign shows direction. Cohen (1988) named 0.2, 0.5 and 0.8 small, medium and large; the “below small” band under 0.2 is this site's label, not Cohen's. Benchmarks are conventions, not measures of practical or clinical importance.

More about effect size ↓

How It's Reported

Equation Form

Example

Mdiff = 0.50, d = 0.12, δ = 0.11, 95% CrI [−0.26, 0.48], BF01 = 4.02 (JZS prior, r = 0.707), t(98) = 0.61, p = .544

Table 1. Prior Sensitivity of the Bayes Factor for Note-Taking Method

Example
BF_{10}BF_{01}P(H_0 \mid \text{data})95% CrI for \delta
Medium, r = \sqrt{2}/2 0.254.02.80[−0.26, 0.48]
Wide, r = 1 0.185.46.85[−0.26, 0.49]
Ultrawide, r = \sqrt{2} 0.137.54.88[−0.27, 0.50]

Note. r is the scale of the Cauchy prior on \delta; the names are R's BayesFactor presets, and “medium” is the default reported above. Posterior probabilities assume even prior odds; the credible interval assumes the alternative is true. Wider priors spread the alternative's bets over larger differences, so our small observed difference fits the alternative worse and the evidence for no difference grows. Narrower priors do the reverse: at r = 0.3, BF₀₁ falls to 2.15, inconclusive. The interval for \delta barely moves.

Full Text (APA Style)

Example

Quiz scores the next day were about the same whether students took notes by hand (M = 20.00, SD = 4.00, n = 50) or on a laptop (M = 19.50, SD = 4.20, n = 50), a difference of 0.50 points, d = 0.12. A Bayesian independent-samples t-test with the default JZS prior (Rouder et al., 2009) at scale r = 0.707, the default in R's BayesFactor (Morey & Rouder, 2026), favored no difference: the data were 4.02 times more likely if the two methods led to the same average score than if they differed, BF01 = 4.02, moderate evidence for the null hypothesis. With wider priors (r = 1 and 1.414), BF01 rose to 5.46 and 7.54. Assuming there is a difference, its posterior median was δ = 0.11, 95% CrI [−0.26, 0.48]. A classical t-test was not significant, t(98) = 0.61, p = .544.

  • Name the prior and its scale, and the software, so readers can reproduce the Bayes factor.
  • Say which way the Bayes factor points: BF₀₁ for evidence for the null, BF₁₀ for evidence for a difference. Give the evidence label too, as a rough guide.
  • Report a robustness check with at least one wider prior; conclusions that flip with the prior deserve a caveat.
  • Give each group's M, SD and n, and say which group was subtracted from which, so the sign of d and δ makes sense.
  • Say that the credible interval assumes a difference exists, and that the posterior probability assumes even prior odds (.80 here).

Reading Each Piece

The equation form above, piece by piece, in this order: how big, how certain, then how surprising.

Mdiff = 0.50 Students who took notes by hand scored half a point higher on the 30-question quiz than students who typed, about half a question.
d = 0.12 The by-hand group scored 0.12 standard deviations higher, below small by the benchmarks above. That's this sample's difference; the credible interval below says how big the population difference might be. Benchmarks ↑
95% CrI [−0.26, 0.48] If there is a difference, it is plausibly anywhere from about a quarter of an SD in favor of laptops to about half an SD in favor of writing by hand: from small in favor of laptops to near medium in favor of writing by hand, by the benchmarks above. Given the data, the prior and the alternative, there's a 95% chance δ lies in this range. Its middle, the posterior median, is δ = 0.11 in favor of writing by hand, a little smaller than the sample's d of 0.12 because the prior pulls toward 0.
t(98) = 0.61 The difference is 0.61 standard errors above zero. The 98 is the degrees of freedom: n₁ + n₂ − 2.
BF01 = 4.02 The quiz scores are 4.02 times more likely if taking notes by hand and on a laptop lead to the same average score than if they differ by the amounts the default prior expects. That's moderate evidence for the null by the conventional labels, which are rough guides, not cutoffs. It favors means that are exactly equal over the default alternative; it doesn't show the methods are interchangeable in practice, since small differences remain plausible, and showing that needs a range of unimportant differences set in advance. It's about the data, not yet about how likely the null is. Benchmarks ↑
P(H0 | data) = .80 Starting from 50–50, there's now about an 80% chance the two methods lead to the same average score, and a 20% chance they differ. A reader who started out 80% sure there was no difference would now be at 94%, so say which prior odds you used.
p = .544 If the two population means were equal, a gap at least this big, in either direction, would turn up about 54% of the time. Not significant. But the same p could come from methods that truly work alike or from a study too small to tell them apart; p can't distinguish the two.
f(\delta = 0 \mid \text{data}, H_1) The height of the posterior density curve at δ = 0, not a probability that δ is 0: under the alternative, any single value has probability 0, and a density can exceed 1.
f(\delta = 0 \mid H_1) The height of the Cauchy prior's density at δ = 0. The posterior height divided by this one, 1.81 ÷ 0.45 = 4.02, is BF₀₁ (the Savage–Dickey density ratio).

Calculations

The Running Example

Example

The note-taking study: 100 undergraduates, 50 taking notes by hand and 50 on a laptop, each with one quiz score from 0 to 30.

Null Hypothesis:
\delta = 0
Alternative (Two-Sided):
\delta \sim \text{Cauchy}(0,\ 0.707)

\delta (“delta”) is the population effect size: the difference between the two population means, hand minus laptop, in standard deviations within each population. The null hypothesis says it's exactly 0, so the two ways of taking notes leave students with the same average score. Unlike a classical test, the alternative has to say how big the difference might be, because a Bayes factor compares how well each hypothesis predicted the data. Its prior distribution on \delta is a Cauchy prior with scale r = 0.707 (√2/2), the default in R's BayesFactor and in JASP: a bell-shaped curve centered on 0 with heavy tails. With r = 0.707, the alternative expects half of all differences to lie within ±0.707 SD, but about 18% within ±0.2 SD and about 39% beyond ±1 SD. This default is called the JZS (Jeffreys–Zellner–Siow) prior. Both hypotheses are two-sided: a difference in either direction counts (Strength of Evidence covers one-sided Bayes factors).

What Your Data Should Look Like

One row per person, with a column saying which group they're in and a numeric column for the outcome. Each person appears once, in one group only. It's the same layout as a classical independent-samples t-test; the difference is in the analysis, not the data. If you measured the same people twice, the Bayesian version is a one-sample Bayes factor on the difference scores (ttestBF(x = after - before) in R, or JASP's Bayesian Paired Samples T-Test).

Example
studentnotesquiz
1laptop20
2hand27
3hand25
4hand13
5laptop19
6laptop17
… 94 more rows
nMSD
By Hand5020.004.00
Laptop5019.504.20
All10019.754.09

Simulated data.

Download all 100 rows (data.csv)

Napkin Math

The whole test, one idea at a time. Under each heading, a line says why the step matters, and the green note reads the equation in plain English with the note-taking study's numbers. Collapse all to read just the headings: they tell the whole story on their own.

  1. 1. Find the Difference between the Two Groups' Averages

    Everything starts with the gap between the two groups. A positive gap means the by-hand group scored higher.

    \bar{x}_1 - \bar{x}_2 =20.00 - 19.50

    = 0.50 points

    \bar{x}_1 is the by-hand group's average quiz score, 20.00 points, and \bar{x}_2 is the laptop group's, 19.50. 20.00 − 19.50 = 0.50 points.

  2. 2. Put Both Groups' Spread on One Scale

    To judge how big a gap is, we need one typical spread for a student's score around their own group's average. Pooling averages the two groups' variances (each SD squared). The Bayesian t-test uses this pooled spread too, because its model gives both groups the same one.

    s_p = \sqrt{\dfrac{(n_1 - 1)s_1^2 + (n_2 - 1)s_2^2}{n_1 + n_2 - 2}} =\sqrt{\dfrac{49 \times 4.00^2 + 49 \times 4.20^2}{50 + 50 - 2}}

    = 4.10 points

    s_1 = 4.00 points

    s_2 = 4.20 points

    s_1 = 4.00 and s_2 = 4.20 are the two groups' SDs, each weighted by n - 1 = 49. 784.0 + 864.4 = 1648.4, divided by 98 is 16.82, and its square root is 4.10 points.

  3. 3. Measure How Big the Difference Is in Standard Deviations

    The Bayesian t-test's hypotheses are about the difference in standard deviations, not in points, so start by measuring ours that way. Dividing by the spread among students gives a size with no units.

    d = \dfrac{\bar{x}_1 - \bar{x}_2}{s_p} =\dfrac{0.50}{4.10}

    = 0.12

    The 0.50-point gap divided by the 4.10-point pooled spread s_p is 0.12 of a standard deviation.

  4. 4. Compare the Difference with Its Uncertainty

    Group averages wobble from sample to sample, so a gap only means something next to that wobble, the standard error. t counts how many standard errors the gap is. The Bayes factor is built from t, the two group sizes and the prior, and nothing else.

    t = \dfrac{\bar{x}_1 - \bar{x}_2}{SE} =\dfrac{0.50}{0.820}

    = 0.61

    SE = 0.820 points

    df = 98

    The standard error is the pooled spread scaled by both group sizes: 4.10 × √(1/50 + 1/50) = 4.10 × 0.2 = 0.82 points. Our 0.50-point gap divided by that 0.82 puts the by-hand group 0.61 standard errors above the laptop group.

  5. 5. See What the Classical Test Can and Can't Say

    The classical t-test asks how often a gap at least this big, in either direction, would turn up if the two population means were equal and the model's assumptions held. A large p says the data aren't surprising under the null; it can't say whether they support it.

    p = 2 \times P(T_{df} \le -|t|) =2 \times P(T_{98} \le -0.61)

    = .544

    P(T_{98} \le -0.61) is the chance that a t with 98 df lands at or below −0.61. Doubling it counts both directions, .544, about 1 in 2.

  6. 6. Ask How Well Each Hypothesis Predicted the Data

    A Bayes factor compares predictions. The null predicts t values near 0, spread as an ordinary t distribution. The alternative predicts t values shifted by whatever δ is, averaged over its prior, so it spreads its bets more widely. Picture each hypothesis's predictions as sand spread along a line of possible t values: the height of the pile at our t says how strongly it predicted that value. These heights are densities, not chances (they can be above 1), but their ratio compares the two predictions fairly. Whichever is higher at our t wins, by the ratio of the two (Gronau, Ly & Wagenmakers, 2020). The averaging has no closed-form answer; software does it numerically, and we reproduced the software's value with our own integration.

    BF_{10} = \dfrac{\int T_{\nu}\!\left(t \mid \sqrt{n_\delta}\,\delta\right) \pi(\delta)\, d\delta}{T_{\nu}(t)}, \quad \pi(\delta) = \text{Cauchy}(0, r) =\dfrac{\int T_{98}(0.61 \mid 5\,\delta)\, \pi(\delta)\, d\delta}{T_{98}(0.61)} = \dfrac{0.0821}{0.3300}

    p(t \mid H_1) = 0.0821

    p(t \mid H_0) = 0.3300

    BF_{10} = 0.25

    \nu is our 98 df, and the effective sample size n_\delta = 50 \times 50 / 100 = 25, so \sqrt{n_\delta} = 5. The null's t distribution has a density (height) of 0.3300 at our t of 0.61. Averaged over the Cauchy prior \pi(\delta), the alternative's is only 0.0821 high there. 0.0821 ÷ 0.3300 = 0.25: the alternative predicted our data a quarter as well as the null did.

  7. 7. Turn It Around to Read as Evidence for No Difference

    A Bayes factor below 1 is easier to read upside down. BF₀₁ puts the null on top.

    BF_{01} = \dfrac{1}{BF_{10}} =\dfrac{1}{0.2487}

    = 4.02

    BF_{10} is our 0.2487. 1 ÷ 0.2487 = 4.02, so the data are 4.02 times more likely if the two methods lead to the same average score than if they differ by the amounts the default prior expects.

  8. 8. Turn the Evidence into How Probable No Difference Is

    If, before the study, a difference and no difference seemed equally likely, the Bayes factor converts straight into a probability.

    P(H_0 \mid \text{data}) = \dfrac{BF_{01}}{1 + BF_{01}} =\dfrac{4.02}{1 + 4.02}

    = .80

    BF_{01} is our 4.02, and with even prior odds the posterior odds are the same 4.02 to 1. 4.02 ÷ 5.02 = .80.

  9. 9. Estimate How Big the Difference Might Be, If There Is One

    The Bayes factor says whether there's a difference; the posterior distribution says how big it might be if there is one. Each possible δ is weighted by its prior and by how well it predicted our t. The middle 95% of the result is the credible interval. It's worked out under the alternative, so it isn't a test of the point null: an interval that includes 0 doesn't show there's no difference.

    f(\delta \mid \text{data}, H_1) = \dfrac{T_{\nu}\!\left(t \mid \sqrt{n_\delta}\,\delta\right) \pi(\delta)}{\int T_{\nu}\!\left(t \mid \sqrt{n_\delta}\,\delta'\right) \pi(\delta')\, d\delta'}

    = [−0.26, 0.48]

    \text{Mdn}_{\delta} = 0.11

    f(\delta = 0 \mid \text{data}, H_1) = 1.81

    f(\delta = 0 \mid H_1) = 0.45

    The posterior for \delta is centered a little below our d of 0.12, at a median of 0.11, because the prior pulls toward 0. Its middle 95% runs from −0.26 to 0.48. Its density (height) at \delta = 0 is 1.81, against the prior's 0.45: 1.81 ÷ 0.45 = 4.02, BF₀₁ again. The data made values near \delta = 0 four times more plausible than the prior had them.

Run It in Software

The code below reads the running example's data from data.csv; save it in the same folder you run the code from.

SPSS (version 30, as documented by IBM)
  1. Analyze → Bayesian Statistics → Independent Samples Normal.
  2. Move quiz into Test Variable(s) and notes into Grouping Variable, then click Define Groups and enter hand and laptop.
  3. Under Bayesian Analysis, choose Use Both Methods (Bayes factor and posterior distribution).
  4. Click Bayes Factor and check which method and prior SPSS will use, then click OK.

Reading the Output

Read the group means first, then the Bayes factor, then the posterior summary of the difference.

SPSS offers more than one way to compute this Bayes factor, and we couldn't confirm which one is its default or whether any setting reproduces the JZS prior with r = 0.707 that R and JASP use. So its Bayes factor for these data may not be exactly 4.02 (or its reciprocal, 0.25), and its posterior is for the raw difference in points, not δ. Check which way SPSS puts the ratio, since the direction is labeled in its output, and report the method you used. We couldn't run SPSS for this page, so no output is shown; for a result that matches this page exactly, use JASP or R.

Output laid out from this page's computed results, not captured from a live run; labels and layout may differ slightly in your version.

R 
library(BayesFactor)

study <- read.csv("data.csv")
study$notes <- factor(study$notes, levels = c("hand", "laptop"))

bf <- ttestBF(formula = quiz ~ notes, data = study)   # rscale = "medium" (r = sqrt(2)/2) by default
bf
1 / extractBF(bf)$bf        # BF01

# Robustness: wider priors
ttestBF(formula = quiz ~ notes, data = study, rscale = "wide")       # r = 1
ttestBF(formula = quiz ~ notes, data = study, rscale = "ultrawide")  # r = sqrt(2)

# Posterior for delta (assumes there is a difference), by MCMC
set.seed(2026)
post <- posterior(bf, iterations = 100000, progress = FALSE)
quantile(post[, "delta"], c(.025, .5, .975))

# The classical t-test, for comparison
t.test(quiz ~ notes, data = study, var.equal = TRUE)

Bayes factor analysis
--------------
[1] Alt., r=0.707 : 0.248695 ±0.02%

Against denominator:
  Null, mu1-mu2 = 0
---
Bayes factor type: BFindepSample, JZS

[1] 4.020989
Bayes factor analysis
--------------
[1] Alt., r=1 : 0.1832079 ±0.06%

Against denominator:
  Null, mu1-mu2 = 0
---
Bayes factor type: BFindepSample, JZS

Bayes factor analysis
--------------
[1] Alt., r=1.414 : 0.1326021 ±0.12%

Against denominator:
  Null, mu1-mu2 = 0
---
Bayes factor type: BFindepSample, JZS

      2.5%        50%      97.5%
-0.2596178  0.1074167  0.4775213

	Two Sample t-test

data:  quiz by notes
t = 0.60955, df = 98, p-value = 0.5436
alternative hypothesis: true difference in means between group hand and group laptop is not equal to 0
95 percent confidence interval:
 -1.127816  2.127816
sample estimates:
  mean in group hand mean in group laptop
                20.0                 19.5

The first block is BF₁₀ = 0.249 for a difference against the null, and the line after it is BF₀₁ = 1 / 0.249 = 4.02. The wider priors give BF₁₀ = 0.183 (BF₀₁ = 5.46) and 0.133 (BF₀₁ = 7.54). The ± percentage is the numerical error of the integral, negligible here. The quantiles are δ's posterior median and 95% credible interval, 0.11 [−0.26, 0.48]; they come from random draws, so they differ from Napkin Math's exact values in the third decimal and change slightly with the seed. var.equal = TRUE gives Student's t-test, the one the Bayes factor is built from: t(98) = 0.61, p = .544. notes must be a factor with hand first, or the sign of δ flips.

Output from a real run: R 4.6.1 (BayesFactor 0.9.12-4.8), run 2026-10-10.

Python 
import numpy as np
import pandas as pd
import pingouin as pg

study = pd.read_csv("data.csv")
hand = study.loc[study["notes"] == "hand", "quiz"]
laptop = study.loc[study["notes"] == "laptop", "quiz"]

# Student's t-test with the default JZS Bayes factor (r = 0.707) in the BF10 column
res = pg.ttest(hand, laptop, correction=False)
print(res[["T", "dof", "p-val", "cohen-d", "BF10"]].round(4))

bf10 = pg.bayesfactor_ttest(res["T"].iloc[0], len(hand), len(laptop), r=np.sqrt(2) / 2)
print(bf10, 1 / bf10)

# Robustness: wider priors
for r in [1, np.sqrt(2)]:
    print(r, 1 / pg.bayesfactor_ttest(res["T"].iloc[0], len(hand), len(laptop), r=r))

             T  dof   p-val  cohen-d   BF10
T-test  0.6095   98  0.5436   0.1219  0.249
0.24869503957553651 4.020988925660773
1 5.458281143562151
1.4142135623730951 7.541359290191725

correction=False gives Student's t-test; pingouin's default for unequal group sizes is Welch's, and the Bayes factor assumes equal spreads. The second line is BF₁₀ = 0.249 and BF₀₁ = 4.02; the last two are BF₀₁ for r = 1 and √2. Pass r=np.sqrt(2) / 2 rather than 0.707, which moves the fourth decimal. pingouin has no posterior for δ; use R or JASP for the credible interval.

Output from a real run: Python 3.9.6 (pingouin 0.5.5, scipy 1.13.1, pandas 2.3.3), run 2026-10-10.

JASP
  1. Open data.csv. Check that notes is marked nominal and quiz continuous (scale).
  2. Open the T-Tests menu and choose Independent Samples T-Test under the Bayesian heading.
  3. Move quiz into Dependent Variables and notes into Grouping Variable.
  4. Leave Alternative Hypothesis at Group 1 ≠ Group 2 (choose Group 1 > Group 2 only if you specified that direction before collecting the data and have a defensible reason why a difference the other way wouldn't count toward your claim; see Strength of Evidence), and under Bayes Factor choose BF₀₁ to print the evidence for no difference directly.
  5. Under Plots, check Prior and posterior (it adds δ's posterior median and 95% credible interval) and Bayes factor robustness check.
  6. Leave the prior at its default, a Cauchy with scale 0.707 (van Doorn et al., 2021).

Reading the Output

The table gives BF₀₁ (about 4.02) and its error %, the numerical error of the integral, which can be ignored when it's small. The Prior and posterior plot draws the Cauchy prior and δ's posterior. Compare their heights at δ = 0, about 0.45 and 1.81: their ratio is BF₀₁ (see Effect Size). With Additional info checked, it also prints the posterior median, about 0.11, and the 95% credible interval, about [−0.26, 0.48]. The robustness check plots the Bayes factor for Cauchy prior scales from 0 to 1.5; the result is robust if the conclusion doesn't change across reasonable scales.

JASP's Bayesian t-test uses the same default prior as R's BayesFactor, so it should give the R values above. We couldn't run JASP for this page, so no output is shown.

Output laid out from this page's computed results, not captured from a live run; labels and layout may differ slightly in your version.

Impact

Effect Size

Cohen's d expresses the difference in standard deviations. It has no units, so it can be compared across studies that used different scales, but it still depends on the design, how varied the sample is, and how reliably the outcome was measured. It's the same number a classical t-test gives, and its population counterpart, δ, is what the Bayesian t-test's hypotheses are about.

d = \dfrac{\bar{x}_1 - \bar{x}_2}{s_p} = \dfrac{20.00 - 19.50}{4.10} = 0.12

Students who took notes by hand scored half a point higher than students who typed, which is 0.12 of the usual spread among students in the same group, below small by convention.

For the population difference, the Bayesian answer is the posterior distribution of δ: a median of 0.11 and a 95% credible interval from −0.26 to 0.48, anywhere from a small difference in favor of laptops, through no difference, to a near-medium one in favor of writing by hand. Read it with one condition attached: it describes how big δ might be if there is a difference, because it's worked out under the alternative hypothesis (van Doorn et al., 2021). So an interval that includes 0 doesn't show there's no difference, and one that excludes 0 doesn't show there is one; comparing those two is the Bayes factor's job. A credible interval can feed an interval-based decision, such as checking it against a region of practical equivalence, but only when that procedure and its range are defined in advance.

The two connect at δ = 0. The posterior's density there (its height, as in step 6 of Napkin Math, not a probability) is 1.81 and the prior's only 0.45, so the data made values near δ = 0 4.02 times more plausible than the prior had them, and that ratio is exactly BF₀₁. This is the Savage–Dickey density ratio (Wagenmakers et al., 2010): when the null fixes one parameter and both hypotheses share the priors on everything else, as here, the Bayes factor is how much the data raised or lowered the alternative's belief in the null value.

Benchmarks are conventions. Whether half a quiz question matters is a question about learning, not statistics.

Benchmarks Cohen's d at a Glance
Below Small
|d| < 0.2
Small
0.2 ≤ |d| < 0.5
Medium
0.5 ≤ |d| < 0.8
Large
|d| ≥ 0.8

Full benchmarks and caveats ↑

Strength of Evidence

A Bayesian t-test doesn't test for significance. It replaces p with the Bayes factor: how many times better one hypothesis predicted the data than the other. BF₁₀ above 1 favors a difference, BF₀₁ above 1 favors none, and values near 1 mean the data can't tell.

BF₀₁ = 4.02: the quiz scores are 4.02 times more likely if taking notes by hand and on a laptop lead to the same average score than if they differ by the amounts the default prior expects. That favors means that are exactly equal over the default alternative; it doesn't establish that the methods are interchangeable in practice, because small differences remain plausible. The classical t-test on the same data gave p = .544, which only says the result isn't surprising under the null.

That's the difference that matters. A large p means "not enough evidence against the null," whether the null is true or the study was too small. A Bayes factor separates the two: it can show evidence for no difference (BF₀₁ well above 1) or show that the data are inconclusive (BF near 1) (Rouder et al., 2009). The evidence labels in the guidelines above, such as "moderate" for a Bayes factor between 3 and 10, are rough conventions (Schönbrodt & Wagenmakers, 2018), not thresholds like α = .05. Read BF₀₁ on the same scale as BF₁₀, flipped.

It works the other way too. In a study this size, a result right at p = .05 (t = 1.98) would give BF₁₀ = 1.19: the data would be barely more likely under a difference than under none. A just-significant p can be weak evidence.

The answer depends on the prior, so it comes with a robustness check: rerun it with wider and narrower priors and report how much the Bayes factor moves (van Doorn et al., 2021). Here BF₀₁ passes 3 at every scale from r = 0.5 up (3.04 at 0.5), and grows as the prior widens. If earlier studies say how big the difference is likely to be, an informed prior can replace the default (Gronau et al., 2020).

If you specified the direction before collecting the data, and have a defensible reason why a difference the other way wouldn't count toward your claim, you can put the prior on that side only: a one-sided Bayes factor (BF₊₀ in JASP, nullInterval = c(0, Inf) in R's ttestBF). If the data go the predicted way, the evidence for a difference can be up to twice as strong as the two-sided one; if they go the other way, they count for the null. Decide before seeing the data: picking the side afterwards is like switching to a one-sided p.

A Bayes factor is not the probability that a hypothesis is true. Posterior odds are the Bayes factor times the prior odds; only with even prior odds does BF₀₁ = 4.02 become an 80% chance that the means are equal.

Benchmarks Bayes Factor at a Glance
Extreme for H₀
BF₁₀ < 1/100
Very Strong for H₀
1/100 ≤ BF₁₀ < 1/30
Strong for H₀
1/30 ≤ BF₁₀ < 1/10
Moderate for H₀
1/10 ≤ BF₁₀ < 1/3
Anecdotal for H₀
1/3 ≤ BF₁₀ < 1
Anecdotal for H₁
1 ≤ BF₁₀ < 3
Moderate for H₁
3 ≤ BF₁₀ < 10
Strong for H₁
10 ≤ BF₁₀ < 30
Very Strong for H₁
30 ≤ BF₁₀ < 100
Extreme for H₁
BF₁₀ ≥ 100

How many times better one hypothesis predicted the data than the other.

Full benchmarks and caveats ↑

Power and Sample Size

Power is the chance that a study detects an effect that's really there. Bayes factors don't have power in that sense, because there's no significance cutoff. Their version is a Bayes factor design analysis: before collecting data, work out how often a planned study would reach convincing evidence, say a Bayes factor of 3 or more, in the right direction, both when the groups truly differ and when they truly don't (Schönbrodt & Wagenmakers, 2018).

With 50 students per group, if the two methods truly led to the same average score, the study would reach BF₀₁ > 3 about 69% of the time. If writing by hand truly helped by half a standard deviation (d = 0.5), it would reach BF₁₀ > 3 only about 52% of the time, and about 7% of the time it would point the wrong way, with BF₀₁ > 3. To get BF₁₀ > 3 at least 80% of the time for d = 0.5, plan for 91 students per group; to get BF₀₁ > 3 80% of the time when there's no difference, plan for 100.

Those figures are calculated numerically from the t distribution, without simulation. They're exact within the planning model (normal scores, equal SDs, the stated n, δ and threshold), not guarantees for data that break its assumptions. The default Bayes factor depends on the data only through t and the two group sizes, so each chance is the probability that t lands beyond the point where the Bayes factor crosses 3: under the ordinary t distribution if there's no difference, and under a shifted (noncentral) t distribution, as in a classical power analysis, if there is. Evidence for the null builds slowly: even with 150 per group, a true null gives BF₀₁ > 3 only about 84% of the time. Bayes factors can also be checked as the data come in, stopping once they pass a threshold set in advance, such as 10 either way (Schönbrodt & Wagenmakers, 2018).

The calculator works in Cohen's d with the same default prior, r = 0.707 on δ, so its numbers are the ones above.

Aim for a design that gives compelling evidence whichever hypothesis is true, and plan it before collecting data. The calculator shows both chances; the dashed line is the no-difference case. Try it:

Other Info

Follow-Up and Post Hoc Tests

None needed. This test answers a single question, so there are no planned contrasts or post hoc tests to run.

References

  • Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 16(2), 225–237. https://doi.org/10.3758/PBR.16.2.225
    “Here we highlight a Bayes factor alternative to the conventional t test that will allow researchers to express preference for either the null hypothesis or the alternative.” (Abstract)
  • Gronau, Q. F., Ly, A., & Wagenmakers, E.-J. (2020). Informed Bayesian t-tests. The American Statistician, 74(2), 137–143. https://doi.org/10.1080/00031305.2018.1562983
  • Wagenmakers, E.-J., Lodewyckx, T., Kuriyal, H., & Grasman, R. (2010). Bayesian hypothesis testing for psychologists: A tutorial on the Savage–Dickey method. Cognitive Psychology, 60(3), 158–189. https://doi.org/10.1016/j.cogpsych.2009.12.001
  • van Doorn, J., van den Bergh, D., Böhm, U., Dablander, F., Derks, K., Draws, T., Etz, A., Evans, N. J., Gronau, Q. F., Haaf, J. M., Hinne, M., Kucharský, Š., Ly, A., Marsman, M., Matzke, D., Komarlu Narendra Gupta, A. R., Sarafoglou, A., Stefan, A., Voelkel, J. G., & Wagenmakers, E.-J. (2021). The JASP guidelines for conducting and reporting a Bayesian analysis. Psychonomic Bulletin & Review, 28(3), 813–826. https://doi.org/10.3758/s13423-020-01798-5
    “the robustness of the result to different prior distributions can be explored and included in the report” (p. 816)
    “The Bayes factor and credible interval have different purposes and can yield different conclusions.” (p. 821)
  • Morey, R. D., & Rouder, J. N. (2026). BayesFactor: Computation of Bayes factors for common designs (R package version 0.9.12-4.8). https://doi.org/10.32614/CRAN.package.BayesFactor
  • Rouder, J. N., Morey, R. D., Speckman, P. L., & Province, J. M. (2012). Default Bayes factors for ANOVA designs. Journal of Mathematical Psychology, 56(5), 356–374. https://doi.org/10.1016/j.jmp.2012.08.001
  • Schönbrodt, F. D., & Wagenmakers, E.-J. (2018). Bayes factor design analysis: Planning for compelling evidence. Psychonomic Bulletin & Review, 25(1), 128–142. https://doi.org/10.3758/s13423-017-1230-y
    “The evidential categories serve as a rough heuristic whose main goal is to prevent researchers from overinterpreting the evidence in the data.” (Table 1 and the paragraph before it)
  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.
  • Kass, R. E., & Raftery, A. E. (1995). Bayes factors. Journal of the American Statistical Association, 90(430), 773–795. https://doi.org/10.1080/01621459.1995.10476572
    “these categories are not a calibration of the Bayes factor, but rather a rough descriptive statement about standards of evidence in scientific investigation” (Section 3.2, p. 777)
  • Lee, M. D., & Wagenmakers, E.-J. (2014). Bayesian cognitive modeling: A practical course. Cambridge University Press. https://doi.org/10.1017/CBO9781139087759
  • Jeffreys, H. (1961). Theory of probability (3rd ed.). Clarendon Press.

Spot an error? Something still confusing? Tell us at support@statsforpsychology.com.