Kruskal-Wallis Test: Interpretation and APA Reporting

The Basics

When to Use It

Use it when you have three or more separate groups of different people, one outcome that is ordinal (ratings, rankings) or numeric but badly skewed or prone to outliers, and you want to know whether some groups tend to score higher than others. When the outcome is a count, such as intrusive memories, the Kruskal-Wallis test compares the groups through their ranks. If your question is about expected numbers of events, or how many times more often they happen in one group, a count model such as negative binomial regression, the extension of Poisson regression for spread-out counts, may answer it more directly. For example:

Does what people do right after a distressing film (Tetris, a quiz or nothing) change how many intrusive memories they have that week?
Do pain ratings on a 0–10 scale differ between patients given relaxation training, hypnosis or standard care?
Does the number of days absent from school differ among students in three kinds of reading intervention?

If your groups have an expected order, such as three increasing doses, and your question is whether scores rise or fall in that order, the Jonckheere-Terpstra test is built for that and usually has more power for it (Hollander et al., 2014).

Vocabulary You Need

Definition Rank

A score's position when all the scores are lined up from lowest to highest: 1 for the lowest, 2 for the next, and so on. Tied scores share the average of the positions they fill. Ranks keep the order of the scores but throw away the distances between them, so one extreme score counts only as the highest and can't dominate a rank-based test, and taking logs or square roots of the scores leaves the ranks unchanged. Glossary →

Definition Mean Rank (

\bar{R}

)

A group's rank sum divided by the number of people in it. If the groups don't differ, every group's mean rank should sit near the middle rank, (N + 1) ÷ 2; a group that tends to score higher has a mean rank above it. It says which groups tend to be higher, not by how many points. Glossary →

Definition Kruskal-Wallis H (

H

)

A measure of how far the groups' mean ranks spread from the overall middle rank, scaled by how much ranks vary. It is near k − 1 on average when the groups don't differ and grows as some groups collect the high ranks and others the low ones. It is compared with a chi-square distribution with k − 1 degrees of freedom to get a p-value. Glossary →

Definition Omnibus Test

A single test of whether any of several groups differ, without saying which. A significant omnibus result is followed by comparisons of specific groups; a non-significant one doesn't prove the groups are equal. Glossary →

Assumptions and Inference

1. Independent Observations (for the p-Value) The people (or other units) in your study don't influence each other.

What It Means: Each person's data comes from a separate, unconnected source, so knowing one person's scores tells you nothing about anyone else's. In designs that measure people more than once, their own scores are meant to be related; it's different people who must be independent. Methods built for clustered data, such as multilevel models and GEE, also allow members of the same group to be related, as long as the analysis names the grouping; then it's the groups that must be independent.
Applies To: Observations
How to Check: By design, not by a test. Did each person contribute their own data, with no clusters such as roommates, couples, therapy groups or classrooms whose members influence each other? If there are clusters, does the analysis model every one of them?
How Much It Matters: A lot. Ranking doesn't fix dependence: if participants watched the film together or compared diaries, their counts are more alike than the test expects, and p comes out too small.

What to Do

Design Change Test and record each person separately, or analyze one score per cluster.
Alternative Test Multilevel Regression (HLM). A multilevel model accounts for the clustering; for counts, a multilevel Poisson or negative binomial model.

2. Similar Shapes (for Reading It as a Shift) The usual null is that all groups come from identical distributions; the test needs no particular shape. Reading a significant result as a shift in typical scores or medians needs similar shapes and spreads. If spreads differ a lot, choose a method suited to that and your question, such as a model for the original scores. The chi-square p-value is also a large-sample approximation (see Statistical Significance).

What It Means: Strictly, a rank test's null hypothesis is that the groups' scores come from identical distributions. It's built to detect one group tending to score higher, but if the groups have clearly different spreads or shapes, a significant result can reflect that difference instead, and p can be off even when no group tends to score higher, especially with unequal group sizes (Fagerland & Sandvik, 2009). Reading the result as a difference in medians, or as one distribution slid along the scale, also needs the same shape and spread.
Applies To: Outcome
How to Check: Put the groups side by side in dot plots, histograms or box plots. Do they have similar skew and similar spread? Don't use a test for equal variances as a gatekeeper.
How Much It Matters: For what a significant H means, and for p. The test's null is that all the groups' scores come from one population. If the spreads clearly differ, a significant H can reflect those spread differences rather than any group tending to score higher, and p can be off (Fagerland & Sandvik, 2009, show this for two groups). Reading the result as a difference in medians needs similar shapes too (Divine et al., 2018). Here the control group's counts spread wider (1 to 18) than the Tetris group's (1 to 9), as counts with a higher average usually do, so we say control participants tended to report more intrusions, not that the medians differ by exactly 2, and we show the plots.

What to Do

Proceed with Caveat Describe the result as one group tending to score higher than another, and show each group's median and range so readers can see the shapes.
Alternative Test Poisson Regression. Negative binomial regression, Poisson regression's extension for spread-out counts, suits counts like these: it models each group's average count directly and lets the spread exceed the mean, as it does here: each group's variance is about 1.5 to 3.4 times its mean. Plain Poisson regression assumes variance equals the mean, so with counts like ours it gives standard errors that are too small, unless you use robust (sandwich) standard errors.
Alternative Test To compare pairs of groups whose spreads clearly differ, the Brunner-Munzel test asks whether one group tends to score higher without assuming equal shapes or spreads (Brunner & Munzel, 2000). It's the brunnermunzel package in R and scipy.stats.brunnermunzel in Python.

Limitations

This Is Not Your Test If…

You have only two groups Mann-Whitney U Test →
You measured the same people in every condition Friedman Test →
Your outcome is continuous and roughly normal within each group One-Way Between-Subjects ANOVA →
You have a second grouping variable or a covariate Ordinal Regression →
Your outcome is unordered categories, not a scale Categorical Data and Proportions →

What It Doesn't Tell You

Which conditions. The omnibus test only says the three conditions aren't all alike. That Tetris differs from control, and the quiz from neither, comes from Dunn's follow-up tests.
How many intrusions. H and ε² work on ranks, so they say the Tetris group tended to rank lower, not that Tetris saves a particular number of intrusions. The medians (3 versus 5 a week) describe the size in counts.
Why. Random assignment lets us say the activity made a difference, but not why Tetris helped: its visual demands, distraction in general, or simply being more engaging than a quiz.
That there's no difference, for the quiz. It didn't differ reliably from either Tetris or control (Holm p = .320), but with 24 people per condition that's inconclusive, not evidence the quiz works as well as Tetris.

Common Mistakes

Don'tReport a significant H as proof that the medians differ.

InsteadSay which groups tend to score higher, and show the groups' distributions. Claim a median difference only if the groups' shapes and spreads look alike.
Don'tFollow a significant H with an uncorrected Mann-Whitney test for every pair.

InsteadUse Dunn's test on the joint ranks, with a Holm or Bonferroni correction.
Don'tReport H and p alone.

InsteadGive each group's median and mean rank, and an effect size such as ε².
Don'tSwitch to Kruskal-Wallis because a normality test came out significant.

InsteadChoose it in advance because the outcome is ordinal, badly skewed or full of outliers.
Don'tRead a p-value as the chance that the null hypothesis is true.

InsteadRead it as: if the null were true, how often data at least this extreme would turn up.

Result Decoder

Significance and Effect Size Guidelines

Benchmarks Significance (p Value)

p < .001
Very strong evidence against the null hypothesis.
.001 ≤ p < .01
Strong evidence against the null hypothesis.
.01 ≤ p < .05
Statistically significant at α = .05. Moderate evidence against the null hypothesis.
.05 ≤ p < .10
Not significant. Weak evidence; sometimes called a “trend.” Treat it as inconclusive.
p ≥ .10
Not significant. Little evidence against the null, which is not the same as evidence of no effect.

These evidence labels are one common convention for describing p in graded language rather than as significant or not; the boundaries are rough guides, not rules, and other authors draw them differently. Muff et al. (2022) argue for this kind of language of evidence over a single cutoff. α is chosen before the analysis, and p never says how big or important an effect is.

More about statistical significance ↓

Benchmarks Effect Size (Epsilon Squared)

ε² < .01 Below Small
Smaller than the conventional small benchmark. Whether an effect this size matters depends on the setting.
.01 ≤ ε² < .06 Small
A small share of rank variation associated with group membership, by convention.
.06 ≤ ε² < .14 Medium
A moderate share of rank variation associated with group membership, by convention.
ε² ≥ .14 Large
A large share of rank variation associated with group membership, by convention.

Borrowed from η², because ε² here is exactly the η² of a one-way ANOVA run on the ranks. They are conventions (Cohen, 1988; Lakens, 2013), not Kruskal-Wallis-specific standards or measures of practical importance, and they describe a share of variation in ranks, not in the original counts. With no differences at all, ε² averages (k − 1)/(N − 1), about .03 for our 72 people, so small values can be noise.

More about effect size ↓

How It's Reported

Equation Form

Example

ε² = .10, 95% CI [.02, .28], H(2) = 7.44, p = .024

Full Text (APA Style)

Example

Participants who played Tetris after the film reported the fewest intrusive memories over the following week (Mdn = 3, mean rank = 28.42), followed by the quiz condition (Mdn = 4, mean rank = 36.33) and the control condition (Mdn = 5, mean rank = 44.75). Condition was associated with about 10% of the variation in participants' pooled ranks, ε² = .10, 95% CI [.02, .28] (percentile bootstrap, 20,000 resamples within each condition, seeded). A Kruskal-Wallis test showed the differences were statistically significant, H(2) = 7.44, p = .024. Dunn's tests with a Holm correction showed that participants in the Tetris condition tended to report fewer intrusions than those in the control condition, z = 2.73, p = .019; the quiz condition did not differ reliably from either (both p = .320).

Give every group's median and mean rank, so readers can see the pattern H summarizes. Means and SDs mislead for skewed counts.
Report H with its degrees of freedom (k − 1). Some journals write it as χ²(2, N = 72) = 7.44; it's the same number.
Follow a significant H with Dunn's test, and name the correction. The sign of Dunn's z depends only on which group's mean rank is subtracted from which: SPSS prints −2.73 for Tetris versus control, R's rstatix prints 2.73, and this page reports 2.73 (control minus Tetris). Say which way round yours is, or just name the group that ranked higher.
ε² has no formula for its interval, so give a bootstrap CI and say how it was made: the method (percentile), the number of resamples and that it was seeded. Another seed or program moves the limits slightly.
Italicize Latin symbols (Mdn, H, z, p) but not Greek ones (ε²), and report exact p to three decimals.

Reading Each Piece

The equation form above, piece by piece, in this order: how big, then how surprising.

ε² = .10	Condition accounts for about 10% of the variation in where people fall in the ranking. That's a medium effect by the benchmarks above. ε² averages about .03 even when conditions don't differ; subtracting that leaves .08 ( $\eta^2_H$ , in Effect Size), still medium. Whether it matters is a question about intrusive memories, not statistics. Benchmarks ↑
95% CI [.02, .28]	A seeded percentile bootstrap with 20,000 resamples. The effect could plausibly be anything from small (2% of the ranking) to large (28%): with 24 people per condition, the size is poorly pinned down. Because ε² can't go below 0, the interval can't show whether there's an effect; p does that. R's rcompanion with seed 1 and 2,000 resamples gives a slightly different [.02, .29], because R's random numbers differ and rcompanion resamples all 72 people together rather than within each condition.
H(2) = 7.44	The conditions' average ranks spread out enough to give an H of 7.44, where about 2 is typical if the activity doesn't matter. The 2 in parentheses is the degrees of freedom: k − 1.
p = .024	If the activity made no difference, average places at least this far apart would turn up about 2% of the time. That's below .05, so the result is statistically significant. Benchmarks ↑

Calculations

The Running Example

Example

The Tetris study: 72 adults, 24 per condition (Tetris, quiz, control), each with one count of intrusive memories over a week. The counts are skewed and full of ties.

Null Hypothesis:: All three conditions' counts come from the same distribution
Alternative:: At least one condition tends to report more or fewer intrusions

The null says that which condition a person was in tells you nothing about where their count falls in the combined ranking. Our medians of 3, 4 and 5 intrusions suggest otherwise. The alternative doesn't say which conditions differ or in what direction.

What Your Data Should Look Like

One row per person, with one column naming their group and one column for their score, either a number or an ordered rating coded as a number. Each person appears once, in exactly one group. You don't rank anything yourself; the software does it.

Example

participant	condition	intrusions
1	tetris	2
2	quiz	4
3	quiz	8
4	tetris	3
5	tetris	1
6	tetris	2
… 66 more rows

	n	Mdn	Q₁–Q₃	Mean Rank
Tetris	24	3	1.75–4	28.42
Quiz	24	4	2–6.5	36.33
Control	24	5	3–9	44.75

Q₁–Q₃ is where the middle half of the scores lies, from the 25th to the 75th percentile (R's default quantile definition).

Simulated data.

Download all 72 rows (data.csv)

Napkin Math

The whole test, one idea at a time. Under each heading, a line says why the step matters, and the green note reads the equation in plain English with the Tetris study's numbers. Collapse all to read just the headings: they tell the whole story on their own.

1. Put Everyone in One Line, from Fewest Intrusions to Most

Ranking replaces each count with its place in line, so a person with 18 intrusions counts as “last in line,” not as someone far beyond everyone else. Tied counts share the average of the places they fill.

$r_i = \text{position of } x_i \text{ among all } N \text{ scores}$ = $\dfrac{21 + 36}{2}$

$r_{\text{3 intrusions}}$ = 28.5

$r_i$ is each participant's place among all 72. Two people reported 0 intrusions (places 1–2), ten reported 1 (3–12) and eight reported 2 (13–20), so the sixteen who reported 3 fill places 21 to 36 and each gets (21 + 36) ÷ 2 = 28.5.
2. Add Up the Places Each Group Holds

A condition whose members tend to report more intrusions collects the later places in line. Adding up each group's ranks shows where each one landed.

$R_1 = \sum \text{ranks in group 1}$

$R_{\text{Tetris}}$ = 682
$R_{\text{quiz}}$ = 872
$R_{\text{control}}$ = 1074

$R_j$ adds up the ranks of everyone in condition $j$ . The Tetris group's ranks total 682, the quiz group's 872 and the control group's 1074. Together they make 2628, every rank from 1 to 72.
3. Find Each Group's Average Place in Line

If condition made no difference, every group would average the middle place, (72 + 1) ÷ 2 = 36.5. Averaging puts groups of any size on the same footing.

$\bar{R}_j = \dfrac{R_j}{n_j}$ = $\dfrac{682}{24}$

$\bar{R}_{\text{Tetris}}$ = 28.42
$\bar{R}_{\text{quiz}}$ = 36.33
$\bar{R}_{\text{control}}$ = 44.75

$R_j$ is each group's rank total and $n_j$ its 24 people. 682 ÷ 24 = 28.42 for Tetris, 872 ÷ 24 = 36.33 for the quiz, 1074 ÷ 24 = 44.75 for control. Tetris sits about 8 places before the middle, control about 8 after.
4. Measure How Far the Groups' Average Places Stray from the Middle

This squares how far each group's average place strayed from 36.5 (so misses above and below don't cancel), counts it once for every person in the group, adds them up, then scales the total by how much ranks vary. The result would hover around 2 if the conditions didn't matter.

$H = \dfrac{12}{N(N+1)} \sum n_j \left(\bar{R}_j - \dfrac{N+1}{2}\right)^2$ = $\dfrac{12}{72 \times 73}\left[24(28.42 - 36.5)^2 + 24(36.33 - 36.5)^2 + 24(44.75 - 36.5)^2\right]$

$H_{\text{no ties}}$ = 7.31

$n_j$ is 24 per group and $N$ is our 72 people. The three pieces are 1568.17 + 0.67 + 1633.50 = 3202.33, and 12 ÷ (72 × 73) = 12 ÷ 5256, so 3202.33 × 12 ÷ 5256 = 7.31. Nearly all of it comes from Tetris and control.
5. Allow for All the Tied Counts

Tied people share a rank, so the ranks vary a little less than 1 to 72 would. Dividing by a correction factor scales H to the variation the ranks really have, which nudges H up (Hollander et al., 2014).

$H = \dfrac{H_{\text{no ties}}}{1 - \dfrac{\sum (t^3 - t)}{N^3 - N}}$ = $\dfrac{7.31}{1 - \dfrac{6612}{373{,}176}}$

= 7.44

$\text{divisor}$ = 0.982

$t$ is the size of each group of tied counts, such as the 16 people at 3 and the 10 at 1. Their $t^3 - t$ values add up to 6612, and $N^3 - N$ is 373,176, so the divisor is 1 − 0.018 = 0.982. 7.31 ÷ 0.982 = 7.44.
6. Account for How Many Groups We Compared

Three average places around a fixed middle leave two free to vary. The degrees of freedom pick which chi-square distribution to judge H against.

$df = k - 1$ = $3 - 1$

= 2

$k$ is our 3 conditions. 3 − 1 = 2.
7. Measure How Much of the Ordering the Conditions Explain

H grows with sample size; ε² doesn't. It asks what share of the variation in people's places in line lines up with which condition they were in.

$\varepsilon^2 = \dfrac{H}{(N^2 - 1)/(N + 1)} = \dfrac{H}{N - 1}$ = $\dfrac{7.44}{72 - 1}$

= .10

$95\%\ \text{CI}$ = [.02, .28]

$H$ is our 7.44 and $N - 1$ is 71. 7.44 ÷ 71 = .10, so condition accounts for about 10% of the variation in the ranks. There's no single, widely preferred simple formula for its interval, so we bootstrap it, redrawing 24 people with replacement within each condition 20,000 times (seeded); the middle 95% of those ε² values runs from .02 to .28.
8. Ask How Surprising the Result Would Be If the Activity Made No Difference

Finally, ask how often chance alone would spread three groups' average places at least this far apart if all three conditions came from the same population.

$p = P(\chi^2_{k-1} \ge H)$ = $P(\chi^2_{2} \ge 7.44)$

= .024

$\chi^2_{2}$ is a chi-square distribution with 2 df: the spread of H values you'd get if you reran a no-difference study again and again. They average 2 and reach 5.99, the .05 cutoff, only 5% of the time. The chance of 7.44 or more is .024, about 1 in 41. Only large H values count against the null, so there's one tail, but it catches differences in any direction.
9. Find Which Pairs of Conditions Actually Differ

A significant H says the conditions aren't all alike, not which ones differ. Dunn's test compares two groups' average places from the same combined ranking, and the Holm correction keeps the overall chance of a false positive at or below .05.

$z_{ij} = \dfrac{\bar{R}_i - \bar{R}_j}{\sqrt{\left(\dfrac{N(N+1)}{12} - \dfrac{\sum (t^3 - t)}{12(N - 1)}\right)\left(\dfrac{1}{n_i} + \dfrac{1}{n_j}\right)}}$ = $\dfrac{44.75 - 28.42}{5.99}$

$z$ = 2.73
$p_{\text{Holm}}$ = .019

$\bar{R}_i - \bar{R}_j$ is control's 44.75 minus Tetris's 28.42, 16.33 places. The square root works out to 5.99 places, so z = 2.73, unadjusted p = .006. Holm multiplies the smallest of the three p-values by 3, giving .019. Quiz versus the other two gives z = 1.41 (p = .160, against control) and 1.32 (p = .186, against Tetris). Holm multiplies the next smallest, .160, by 2, giving .320, and the largest, .186, by 1; an adjusted p can't be smaller than the one before it, so that one becomes .320 too.

Run It in Software

The code below reads the running example's data from data.csv; save it in the same folder you run the code from.

SPSS (version 30, as documented by IBM)

SPSS's nonparametric procedures need a numeric group, so first recode condition: Transform → Recode into Different Variables, into a new variable cond with 1 = tetris, 2 = quiz, 3 = control, and give it those value labels. Set its measurement level to Nominal.
Analyze → Nonparametric Tests → Legacy Dialogs → K Independent Samples.
Move intrusions into Test Variable List and cond into Grouping Variable. Click Define Range and enter 1 and 3. Leave Kruskal-Wallis H checked, and click OK.
For the pairwise comparisons, use Analyze → Nonparametric Tests → Independent Samples. On the Fields tab move intrusions to Test Fields and cond to Groups; on the Settings tab choose Customize tests, check Kruskal-Wallis 1-way ANOVA (k samples), and set Multiple comparisons to All pairwise. Click Run, then double-click the output to open the pairwise table.

Or paste this syntax:

RECODE condition ('tetris'=1) ('quiz'=2) ('control'=3) INTO cond.
VALUE LABELS cond 1 'tetris' 2 'quiz' 3 'control'.
VARIABLE LEVEL cond (NOMINAL).
EXECUTE.

NPAR TESTS
  /K-W=intrusions BY cond(1 3)
  /MISSING ANALYSIS.

NPTESTS
  /INDEPENDENT TEST (intrusions) GROUP (cond) KRUSKAL_WALLIS(COMPARE=PAIRWISE)
  /MISSING SCOPE=ANALYSIS USERMISSING=EXCLUDE
  /CRITERIA ALPHA=0.05 CILEVEL=95.

Reading the Output

Read the tables in this order: the Mean Rank for each group, then Kruskal-Wallis H, df and Asymp. Sig., then the pairwise comparisons. Asymp. Sig. is p from the chi-square approximation. SPSS doesn't print ε²; divide H by N − 1 yourself (7.443 ÷ 71 = .105). It doesn't print medians here either; get them from Analyze → Descriptive Statistics → Explore.

Ranks

cond	N	Mean Rank
tetris	24	28.42
quiz	24	36.33
control	24	44.75
Total	72

Test Statistics (Grouping Variable: cond)

	intrusions
Kruskal-Wallis H	7.443
df	2
Asymp. Sig.	.024

Pairwise Comparisons of cond

Sample 1-Sample 2	Test Statistic	Std. Error	Std. Test Statistic	Sig.	Adj. Sig.
tetris-quiz	-7.917	5.988	-1.322	.186	.558
tetris-control	-16.333	5.988	-2.728	.006	.019
quiz-control	-8.417	5.988	-1.406	.160	.479

This is Dunn's test. The Test Statistic is the difference in mean ranks, and the Std. Test Statistic is Dunn's z. SPSS adjusts with Bonferroni, which here gives the same .019 for Tetris versus control as the Holm correction on the page; for the other pairs Holm gives .320.

Output laid out from this page's computed results, not captured from a live run; labels and layout may differ slightly in your version.

R

study <- read.csv("data.csv")
study$condition <- factor(study$condition, levels = c("tetris", "quiz", "control"))

tapply(study$intrusions, study$condition, median)

result <- kruskal.test(intrusions ~ condition, data = study)
result

# Epsilon squared: H / (N - 1)
unname(result$statistic) / (nrow(study) - 1)

# 95% CI for epsilon squared: percentile bootstrap, 2,000 resamples, seeded
# (install.packages("rcompanion") once)
set.seed(1)
rcompanion::epsilonSquared(x = study$intrusions, g = study$condition,
                           ci = TRUE, R = 2000)

# Dunn's test with a Holm correction (install.packages("rstatix") once)
rstatix::dunn_test(study, intrusions ~ condition, p.adjust.method = "holm")

 tetris    quiz control
      3       4       5

	Kruskal-Wallis rank sum test

data:  intrusions by condition
Kruskal-Wallis chi-squared = 7.4431, df = 2, p-value = 0.0242

[1] 0.104833
  epsilon.squared lower.ci upper.ci
1           0.105   0.0186    0.289

# A tibble: 3 × 9
  .y.        group1 group2     n1    n2 statistic       p  p.adj p.adj.signif
* <chr>      <chr>  <chr>   <int> <int>     <dbl>   <dbl>  <dbl> <chr>       
1 intrusions tetris quiz       24    24      1.32 0.186   0.320  ns          
2 intrusions tetris control    24    24      2.73 0.00638 0.0191 *           
3 intrusions quiz   control    24    24      1.41 0.160   0.320  ns

R calls H “Kruskal-Wallis chi-squared,” because it's judged against a chi-square distribution. Its p is already corrected for ties. rcompanion::epsilonSquared rounds ε² to .105 and gives a percentile bootstrap 95% CI of [.019, .289]. That's close to this page's [.02, .28] but not the same: R's random numbers differ from the ones behind this page's 20,000 resamples, and rcompanion redraws all 72 people together, so a resample's conditions needn't have 24 people each. Always set a seed, or the interval changes every run; with rcompanion's default of 1,000 resamples and the same seed it's [.021, .277]. Each Dunn statistic is a z: group2's mean rank minus group1's, over its standard error, so the positive values mean the second group ranked later (more intrusions). p is unadjusted and p.adj is Holm's.

Output from a real run: R 4.6.1 (rcompanion 2.5.4, boot 1.3.32, rstatix 1.1.0), run 2026-10-03.

Python

import pandas as pd
from scipy import stats
import scikit_posthocs as sp

study = pd.read_csv("data.csv")
order = ["tetris", "quiz", "control"]
groups = [study.loc[study["condition"] == g, "intrusions"] for g in order]

result = stats.kruskal(*groups)
result.statistic                      # 7.4431
result.pvalue                         # 0.02420
result.statistic / (len(study) - 1)   # epsilon squared, 0.1048

print(sp.posthoc_dunn(study, val_col="intrusions", group_col="condition",
                      p_adjust="holm").round(3))

         control  quiz  tetris
control    1.000  0.32   0.019
quiz       0.320  1.00   0.320
tetris     0.019  0.32   1.000

scipy.stats.kruskal corrects H for ties automatically. scikit_posthocs.posthoc_dunn (install it with pip install scikit-posthocs) prints Holm-adjusted p-values for every pair as a table; only Tetris versus control is below .05. It doesn't print the z values; the R and SPSS output above have them.

Output laid out from this page's computed results, not captured from a live run; labels and layout may differ slightly in your version.

Impact

Effect Size

Epsilon squared (ε²) expresses the result as the share of all the variation in the ranks that lines up with the groups. It's H divided by its largest possible value, N − 1, so it doesn't grow with sample size the way H does (Tomczak & Tomczak, 2014). It's a proportion of variation in ranks, not a proportion of variation in the original number of intrusive memories.

\varepsilon^2 = \dfrac{H}{N - 1} = \dfrac{7.44}{72 - 1} = .10

Of all the variation in where people fell in the line, from fewest intrusions to most, about 10% lines up with which activity they did after the film, a medium effect by convention.

There's no single, widely preferred simple formula for a confidence interval for this ε², so this page uses a percentile bootstrap. Our percentile bootstrap redrew 24 people with replacement within each condition, recomputed ε², repeated that 20,000 times from a fixed seed, and kept the middle 95%: 95% CI [.02, .28]. The interval is wide: anything from a small to a large share of the ranking fits the data. Because this interval is built for an estimate that can't go below 0, and isn't calibrated as an inversion of the Kruskal-Wallis test, don't use whether it contains 0 to decide significance; it sits above 0 even when the groups don't differ. H's p-value answers that. A bootstrap depends on its random draws, so report the method, the number of resamples and the seed. With 20,000 resamples the rounded limits held at .02 and .28 across 20 different seeds. R's rcompanion (epsilonSquared(..., ci = TRUE)) with set.seed(1) and 2,000 resamples gives [.019, .289]; it uses different random numbers and resamples all 72 people together instead of within each condition.

This ε² is exactly the eta squared you'd get from a one-way ANOVA run on the ranks, which is why the η² benchmarks are borrowed for it. Like η², it runs high: H averages k − 1 when the groups don't differ, so ε² averages (k − 1)/(N − 1), about .03 for our 72 people, already past the small benchmark. Another rank-based option, $\eta^2_H = (H - k + 1)/(N - k)$ , subtracts that expected amount, so it's smaller: .08 for our data (Tomczak & Tomczak, 2014). With small samples, report $\eta^2_H$ too, or read ε² against .03 rather than 0. This ε² isn't the ANOVA effect size also called epsilon squared, which corrects η² for bias, so say which one you report.

ε² covers all three conditions at once. For the comparison readers care about, give the size in the outcome's own terms: the Tetris group's median was 3 intrusions a week against the control group's 5, and the Tetris group's average place in line was about 16 places earlier. Or say how often one group beats the other: in 73% of the 576 pairings of a control participant with a Tetris participant, the control participant reported more intrusions (counting ties as half; counted from the raw scores, since the rank sums alone don't determine it). Benchmarks are conventions. Whether two fewer intrusions a week matters is a question about distress and recovery, not statistics.

Benchmarks Epsilon Squared at a Glance

Below Small

ε² < .01

Small

.01 ≤ ε² < .06

Medium

.06 ≤ ε² < .14

Large

ε² ≥ .14

Full benchmarks and caveats ↑

Statistical Significance

p answers one narrow question: if all three conditions' counts came from the same population, how often would the groups' average ranks spread at least this far apart? Only a large H counts against the null, so p comes from the upper tail of a chi-square distribution with k − 1 degrees of freedom, but it responds to differences in any direction. Below the conventional α = .05, the result is called statistically significant.

If the activity made no difference, average ranks at least as spread out as 28.42, 36.33 and 44.75 would turn up about 2.4% of the time, roughly 1 in 41. That's below .05, so it's significant.

The chi-square distribution is an approximation. It works well for groups the size of ours; Kruskal and Wallis (1952) gave exact tables for three groups of five or fewer. For small or heavily tied samples, get a permutation p instead. R's kruskal.test doesn't offer one, but coin::kruskal_test(intrusions ~ factor(condition), data = study, distribution = approximate(nresample = 10000)) does; in SPSS, use the Exact button, which needs the Exact Tests module.

p is not the probability that the null hypothesis is true, and it says nothing about how large the differences are or which conditions differ. That's what ε² and Dunn's tests are for.

Benchmarks p at a Glance

Very Strong

p < .001

Strong

.001 ≤ p < .01

Significant

.01 ≤ p < .05

Weak

.05 ≤ p < .10

Little

p ≥ .10

Strength of evidence against the null hypothesis.

Full benchmarks and caveats ↑

Power and Sample Size

Power is the chance that a study detects an effect that's really there. A real effect doesn't guarantee a significant result: with too few people, it can easily come out non-significant. Power depends on how big the true differences are, how many people are in each group, how many groups there are, α, and, for a rank test, the shape of the scores.

There's no simple formula for the Kruskal-Wallis test's power. A safe rough plan is to size the study as a one-way ANOVA with Cohen's f, then allow for the worst case: relative to the F test, the Kruskal-Wallis test's large-sample efficiency never falls below 0.864 when the groups differ only by a shift (Hodges & Lehmann, 1956), so about 16% more people covers that case. The calculator below does exactly that. With many ties, small groups, or groups that differ in spread, as counts like ours do, that allowance can fall short, so simulate: write down what you expect each group's scores to look like, generate thousands of pretend studies, and count how often the test comes out significant.

Suppose, before collecting any data, we expected skewed counts like the ones that generated our data: negative binomial counts averaging 3.2, 4.6 and 5.8 intrusions a week, with dispersion (size) 2, so the spread grows faster than the average. In 5,000 simulated studies with 24 people per condition (seed 99, chi-square p at α = .05), only about 48% came out significant. It takes about 48 per condition, 144 in all, to reach 80%. These are simulation estimates, so they move by a point or two between runs; treat 48 as approximate, not an exact minimum.

The answer depends heavily on how spread out you expect the counts to be. Less spread-out counts with the same averages need fewer people, so base the simulation on pilot data or earlier studies, and try a few plausible values. Under our model, the significant result came from a study with about a coin flip's chance of finding it. That's common, and it's one more reason the comparisons involving the quiz are inconclusive rather than negative.

Researchers usually aim for at least 80% power. Plan it before collecting data, using the smallest differences that would matter, and simulate with scores shaped like the ones you expect. Try it:

Other Info

Follow-Up and Post Hoc Tests

Post Hoc Tests

Dunn's Test: The standard follow-up to a significant Kruskal-Wallis test, and step 9 of the Napkin Math. For our data only Tetris versus control differs: z = 2.73, Holm-adjusted p = .019. It is not the same as running a separate Mann-Whitney U test for each pair: Dunn's test keeps the ranks from the combined ranking, so every comparison uses the same yardstick. The trade-off is that one pair's result can shift with the other groups' data. Pairwise Mann-Whitney tests with a Holm correction avoid that and are also defensible; for our data they agree that only Tetris versus control differs. Dunn's comparisons use a normal approximation, and Holm controls the familywise error rate across the three comparisons, so each pair's verdict is separate from the omnibus test's.
Holm Correction: Apply it to Dunn's p-values. It holds the familywise error rate at or below α with more power than Bonferroni. For our data it multiplies the smallest of the three p-values (.006) by 3, giving .019, the next (.160) by 2, giving .320, and the largest (.186) by 1; an adjusted p can't be smaller than the one before it, so that one becomes .320 too.
Bonferroni Correction: Multiplies every Dunn p-value by the number of pairs. SPSS applies it by default. Here it gives the same .019 for Tetris versus control, and .558 and .479 for the other two pairs.

Related Methods

Related DesignFriedman Test: Asks the same question of ranks when the same people are measured in every condition, ranking within each person instead of across everyone.
Alternative MeasureBayesian ANOVA: A Bayesian comparison of the group means, which can give evidence for equal means as well as against them. It assumes normal scores within each group, and it asks about means rather than how the groups rank.
Special CaseMann-Whitney U Test: The two-group case of this test. With just two groups, the Mann-Whitney U test's z squared is H (both with the tie correction), with the same p.
Alternative MeasureOne-Way Between-Subjects ANOVA: The comparison of three or more group means. It asks whether the population means differ, under normal residuals (and, for the classic F, equal variances); Welch's ANOVA drops the equal-variance assumption.
AlternativeMultilevel Regression (HLM): An alternative if “Independent Observations” doesn't hold. A multilevel model accounts for the clustering; for counts, a multilevel Poisson or negative binomial model.
AlternativePoisson Regression: An alternative if “Similar Shapes” doesn't hold. Negative binomial regression, Poisson regression's extension for spread-out counts, suits counts like these: it models each group's average count directly and lets the spread exceed the mean, as it does here: each group's variance is about 1.5 to 3.4 times its mean. Plain Poisson regression assumes variance equals the mean, so with counts like ours it gives standard errors that are too small, unless you use robust (sandwich) standard errors.

References

Kruskal, W. H., & Wallis, W. A. (1952). Use of ranks in one-criterion variance analysis. Journal of the American Statistical Association, 47(260), 583–621. https://doi.org/10.1080/01621459.1952.10483441
“Under the stated hypothesis, H is distributed approximately as χ²(C−1), unless the samples are too small, in which case special approximations or exact tables are provided.” (p. 584)

“The only assumptions underlying the use of ranks made in this paper are that the observations are all independent, that all those within a given sample come from a single population, and that the C populations are of approximately the same form.” (p. 585)
Hollander, M., Wolfe, D. A., & Chicken, E. (2014). Nonparametric statistical methods (3rd ed.). Wiley. https://doi.org/10.1002/9781119196037
Dunn, O. J. (1964). Multiple comparisons using rank sums. Technometrics, 6(3), 241–252. https://doi.org/10.1080/00401706.1964.10490181
Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2), 65–70. https://www.jstor.org/stable/4615733
Divine, G. W., Norton, H. J., Barón, A. E., & Juarez-Colunga, E. (2018). The Wilcoxon–Mann–Whitney procedure fails as a test of medians. The American Statistician, 72(3), 278–286. https://doi.org/10.1080/00031305.2017.1305291
Fagerland, M. W., & Sandvik, L. (2009). The Wilcoxon–Mann–Whitney test under scrutiny. Statistics in Medicine, 28(10), 1487–1497. https://doi.org/10.1002/sim.3561
“small differences in variances and moderate degrees of skewness can produce large deviations from the nominal type I error rate” (Abstract)
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.
Brunner, E., & Munzel, U. (2000). The nonparametric Behrens-Fisher problem: Asymptotic theory and a small-sample approximation. Biometrical Journal, 42(1), 17–25. https://doi.org/10.1002/(SICI)1521-4036(200001)42:1<17::AID-BIMJ17>3.0.CO;2-U
Hodges, J. L., Jr., & Lehmann, E. L. (1956). The efficiency of some nonparametric competitors of the t-test. The Annals of Mathematical Statistics, 27(2), 324–335. https://doi.org/10.1214/aoms/1177728261
“This result also holds for the Kruskal-Wallis test compared with the F-test” (Abstract)
Tomczak, M., & Tomczak, E. (2014). The need to report effect size estimates revisited. An overview of some recommended measures of effect size. Trends in Sport Sciences, 21(1), 19–25. https://tss.awf.poznan.pl/The-need-to-report-effect-size-estimates-revisited-An-overview-of-some-recommended,188960,0,2.html
“once the Kruskal-Wallis H-test has been computed, the epsilon-squared estimate of effect size can be calculated” (p. 24)
Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, 863. https://doi.org/10.3389/fpsyg.2013.00863
“Cohen (1988) has provided benchmarks to define small (η2 = 0.01), medium (η2 = 0.06), and large (η2 = 0.14) effects.” (Eta-squared in between and within-subjects comparisons)
Muff, S., Nilsen, E. B., O'Hara, R. B., & Nater, C. R. (2022). Rewriting results sections in the language of evidence. Trends in Ecology & Evolution, 37(3), 203–210. https://doi.org/10.1016/j.tree.2021.10.009
“we suggest language of evidence that allows for a more nuanced approach to communicate scientific findings as a simple and intuitive alternative to statistical significance testing” (Abstract)