Factorial Between-Subjects ANOVA

The Basics

When to Use It

Use it when you have two (or more) grouping variables, called factors, crossed so that separate groups of people fill every combination, one continuous outcome, and you want to know each factor's effect and whether one factor's effect depends on the other. For example:

Does how students spend a 20-minute break (scrolling or walking) change their mood, and does it matter whether a friend is with them?
Does a mindfulness app reduce anxiety scores more for first-year students than for final-year students?
Does sleep deprivation slow reaction time more on a hard task than on an easy one, with each person tested in one condition?

Vocabulary You Need

Definition Main Effect

How much the outcome differs across the levels of one factor, averaging over the levels of the other factors. It is read from the marginal means: for each level, the average of its cell means, each cell counted equally. When the factors interact, a main effect is an average of effects that differ, so it can describe no actual condition well. Glossary →

Definition Interaction Effect

The part of a factorial design's result that neither factor explains on its own: the effect of one factor changes depending on the level of the other. In a 2 × 2 design it is a difference of differences, the effect of one factor at one level of the other minus its effect at the other level. On a plot of the cell means, clearly non-parallel lines suggest one; the interaction test says whether the departure is bigger than noise. An interaction is symmetric: if A's effect depends on B, B's effect depends on A. Glossary →

Definition Simple Effect

The effect of one factor within one level of another factor, such as the effect of a treatment for women only. Comparing simple effects is how an interaction is unpacked: the interaction says the effect changes, and the simple effects say what it is at each level. With between-subjects factors it's usually tested against the whole study's within-groups variance, which keeps the full degrees of freedom; with within-subjects factors, usually against an error term from its own cells only (for two levels, a paired t-test). Glossary →

Definition Partial Eta Squared (

\eta^2_p

)

An effect's sum of squares divided by that sum of squares plus its own error term's. It leaves out the other effects and, in designs with repeated measures, the stable differences between people, so each effect gets its own denominator. It equals eta squared in a one-way between-subjects design; otherwise it runs larger, and it doesn't add up across effects. It is what SPSS prints for ANOVAs. In MANOVA and MANCOVA there's no single sum of squares; SPSS's multivariate version is Pillai's trace divided by s, the number of dimensions the groups can differ along (V ÷ s). Don't convert a repeated-measures partial η² into Cohen's f for a power calculator that also asks for the correlation between measures: the correlation would be counted twice. Glossary →

Definition F Statistic (

F

)

The variance due to an effect divided by the error variance it is tested against: in a between-subjects ANOVA, the between-groups variance over the within-groups variance; in a repeated-measures ANOVA, the conditions' variance over the inconsistency in how people change across them. It is near 1 when there's no real effect and grows as the effect stands out from the error. It is compared with an F distribution, which has two degrees of freedom, to get a p-value. Glossary →

Assumptions and Inference

1. Independent Students (for the p-Values and Intervals) Each student's score is independent of everyone else's, within and across the four cells.

What It Means: Each person's data comes from a separate, unconnected source, so knowing one person's scores tells you nothing about anyone else's. In designs that measure people more than once, their own scores are meant to be related; it's different people who must be independent. Methods built for clustered data, such as multilevel models and GEE, also allow members of the same group to be related, as long as the analysis names the grouping; then it's the groups that must be independent.
Applies To: Observations
How to Check: By design, not by a test. Did each person contribute their own data, with no clusters such as roommates, couples, therapy groups or classrooms whose members influence each other? If there are clusters, does the analysis model every one of them?
How Much It Matters: A lot. Random assignment doesn't prevent clustering: students assigned one by one can still share a classroom, a team or a session. The friends came along but weren't measured, which keeps each score independent. If both friends had been measured, pairs would share a break and a mood, their scores would be more alike than the test assumes, and the F tests would typically be too liberal.

What to Do

Alternative Test Multilevel Regression (HLM). A multilevel model with pairs (or groups, or classrooms) as clusters, which accounts for the shared variation.
Design Change Measure one person per unit, or analyze each cluster's average.

2. Normality Within Cells (for the p-Values and Intervals) The scores within each of the four cells (the residuals) are roughly normal; the 96 scores pooled together needn't be. Each F is exact under normal residuals and equal variances. Moderate departures are usually tolerated, especially with bigger, equal-sized cells, but strong skew, heavy tails or extreme outliers can still matter.

What It Means: The test's model assumes the scores it relies on are roughly normal: the scores within each group (or the scores themselves, for a one-sample test), the difference scores in a paired design, or the residuals (the errors left around a model's predictions) in a regression. For p-values and confidence intervals, what really matters is that the estimates (such as a mean, a difference between means or a model's coefficients) are roughly normal across repeated samples. A large enough sample usually provides that; how large depends on how skewed the scores are and how extreme their outliers.
Applies To: Residuals
How to Check: Here a residual is each student's score minus their cell's average. Plot those in a histogram or Q–Q plot, not the raw scores pooled across cells, which mix four different centers. Look for strong skew or extreme outliers. Don't rely on a normality test: with small samples it misses real problems, and with large ones it flags trivial ones.
How Much It Matters: Little here: within each cell the residuals' skewness runs from −0.33 to 0.79, and no residual is more than 2.7 SDs from its cell's mean. With 24 people in every cell, that matters little: each F is the same kind of ratio as in a one-way ANOVA, where simulations found the F-test held its false-positive rate across many non-normal shapes (Blanca et al., 2017). Extreme outliers matter more: they pull a cell mean around, which can create or hide an interaction.

What to Do

Robust Estimation Compare trimmed means with a robust two-way ANOVA (in R, WRS2::t2way), which resists outliers and heavy tails. The general remedy when residuals aren't normal.
Transformation Log-transform right-skewed outcomes such as reaction times. Interactions can appear or vanish on the log scale, so interpret on the scale you analyzed.
Alternative Test Ordinal Regression. When the outcome is a few ordered categories rather than a score, model category probabilities with both factors and their interaction. That's a different question from the means, so use it for the outcome's type, not because a continuous score is skewed.

3. Equal Variances (for the Classic F-Tests) The classic F-tests assume every cell's population is equally spread out. Equal cell sizes make them fairly robust to moderate differences but don't remove the assumption. When cell variances differ substantially, use a heteroscedasticity-robust factorial analysis that keeps the main-effect and interaction questions; Welch's one-way ANOVA doesn't replace a factorial model.

What It Means: The outcome varies about as much within each group's population. Tests that pool the groups' variances into one estimate, like Student's t-test and the classic ANOVA, rely on it.
Applies To: Variances
How to Check: Compare the groups' standard deviations and look at side-by-side box plots. Don't use Levene's test as a gatekeeper: it misses real differences in small samples and flags trivial ones in large samples.
How Much It Matters: Less with equal cells, like our 24 per cell with SDs from 5.93 to 6.06. When cells differ in both size and spread, error rates drift, as they do for one-way ANOVA (Delacre et al., 2019). Welch's one-way ANOVA is no fix here: it compares the four cells as one factor and can't test the main effects and interaction. Use a method that keeps the factorial structure.

What to Do

Correction Fit the ANOVA as a regression and test each main effect and the interaction with heteroscedasticity-consistent (HC3) standard errors (in R, car::Anova(model, type = 3, white.adjust = "hc3")). Here it gives the interaction F = 5.75, p = .019, close to the classic 6.00 and .016.
Design Change Plan equal cell sizes. It protects against unequal spreads and keeps the sums of squares from depending on their type.

Limitations

This Is Not Your Test If…

You have only one grouping variable One-Way Between-Subjects ANOVA →
You measured the same people in every condition Factorial Repeated-Measures ANOVA →
One factor is between people, the other within them Mixed Factorial ANOVA →
You want to adjust for a covariate, like a baseline score ANCOVA →
You have several outcomes to analyze together MANOVA →
Your outcome is yes/no rather than a score Logistic Regression →
One factor is a continuous score you'd have to split Multiple Linear Regression →

What It Doesn't Tell You

Why. Random assignment lets us say the break made a difference, but not why a friend helped on a walk: the conversation, the company, or simply walking longer or faster. For a factor people weren't assigned to, like year of study or gender, its “main effect” is only a difference between groups that differ in many ways, and an interaction says only that the assigned factor's effect differs between those groups.
Whether it matters. Significance doesn't make 7 points on a 40-point mood scale important for how students feel the rest of the day.
Who else. The result describes undergraduates who brought a friend to a lab study, for one 20-minute break; it may not hold for daily habits, other ages or scrolling at home.
That a friend doesn't help while scrolling. That simple effect wasn't significant, but its interval runs from −2.44 to 4.44 points. That's inconclusive, not evidence the friend makes no difference.

Common Mistakes

Don'tAsk for Type III sums of squares in R with its default treatment coding.

InsteadSet sum-to-zero contrasts first, options(contrasts = c("contr.sum", "contr.poly")). Otherwise the “main effects” are simple effects at the reference level.
Don'tDeclare an interaction because two lines on a plot look different.

InsteadTest it, and give its size. Lines that look non-parallel can come from noise, especially with small cells.
Don'tRun a separate t-test for each simple effect with only that cell pair's data.

InsteadTest simple effects against the whole design's within-cells variance, and correct across the set.
Don'tInterpret a main effect without first looking at the interaction.

InsteadCheck the interaction first. If it's there, describe the main effect as an average over the other factor, and let simple effects say where the effect holds.
Don'tConclude that two effects differ because one is significant and the other isn't.

InsteadTest the interaction, which compares the two effects directly. The difference between significant and not significant isn't itself significant.
Don'tAdd up the partial η² values as if they split the variation between them.

InsteadRemember each has its own denominator, so they can sum past 1. Report η² if you want shares of the total.
Don'tReport partial η² under the label η².

InsteadSay which one you report. SPSS prints partial η², which is never smaller than η² and is usually larger, especially with other effects or repeated measures.
Don'tRead a p-value as the chance that the null hypothesis is true.

InsteadRead it as: if the null were true, how often data at least this extreme would turn up.

Result Decoder

Significance and Effect Size Guidelines

Benchmarks Significance (p Value)

p < .001
Very strong evidence against the null hypothesis.
.001 ≤ p < .01
Strong evidence against the null hypothesis.
.01 ≤ p < .05
Statistically significant at α = .05. Moderate evidence against the null hypothesis.
.05 ≤ p < .10
Not significant. Weak evidence; sometimes called a “trend.” Treat it as inconclusive.
p ≥ .10
Not significant. Little evidence against the null, which is not the same as evidence of no effect.

These evidence labels are one common convention for describing p in graded language rather than as significant or not; the boundaries are rough guides, not rules, and other authors draw them differently. Muff et al. (2022) argue for this kind of language of evidence over a single cutoff. α is chosen before the analysis, and p never says how big or important an effect is.

More about statistical significance ↓

Benchmarks Effect Size (Partial Eta Squared)

η²ₚ < .01 Below Small
Smaller than the conventional small benchmark. Whether an effect this size matters depends on the setting.
.01 ≤ η²ₚ < .06 Small
A sliver of the remaining variation lines up with the effect. Hard to see without measuring it.
.06 ≤ η²ₚ < .14 Medium
A noticeable share of the remaining variation lines up with the effect.
η²ₚ ≥ .14 Large
A large share of the remaining variation lines up with the effect.

Report one for each effect (both main effects and the interaction). They are borrowed from Cohen's (1988) conventions for eta squared and are not measures of practical importance. Partial η² is larger than η² whenever the other effects explain some of the variance, so the same cutoffs label it more generously.

More about effect size ↓

How It's Reported

Equation Form

Example

Interaction: 6.00 points, 95% CI [1.14, 10.86], η²_p = .06, F(1, 92) = 6.00, p = .016. Activity: η²_p = .21, F(1, 92) = 24.00, p < .001. Company: η²_p = .10, F(1, 92) = 10.67, p = .002.

Table 1. Two-Way ANOVA for Positive Mood

Example

	$SS$	$df$	$MS$	$F$	$p$	$\eta^2_p$
Activity	864.00	1	864.00	24.00	< .001	.21
Company	384.00	1	384.00	10.67	.002	.10
Activity × Company	216.00	1	216.00	6.00	.016	.06
Error (within cells)	3312.00	92	36.00

Note. η²_p = partial eta squared. Each F is the effect's mean square divided by the error mean square, 36. Read the interaction first: it says whether the friend's effect depends on the activity, and so whether the two main effects are fair summaries.

Full Text (APA Style)

Example

Having a friend along lifted mood more on a walk than while scrolling. Walking with a friend gave the highest mood (M = 38.00, SD = 6.06), 7 points above walking alone (M = 31.00, SD = 6.01), while scrolling with a friend (M = 29.00, SD = 6.00) was only 1 point above scrolling alone (M = 28.00, SD = 5.93). A 2 × 2 between-subjects ANOVA showed a significant activity × company interaction: the friend's effect was 6 points larger on a walk, 95% CI [1.14, 10.86], η²_p = .06, F(1, 92) = 6.00, p = .016. Simple-effect tests, with Holm-adjusted p-values across the two comparisons and unadjusted 95% confidence intervals, showed a friend raised mood on a walk by 7 points, 95% CI [3.56, 10.44], F(1, 92) = 16.33, p < .001, but not while scrolling, 1 point, 95% CI [−2.44, 4.44], F(1, 92) = 0.33, p = .565. Averaged over company, walking beat scrolling by 6 points, η²_p = .21, F(1, 92) = 24.00, p < .001; averaged over activity, company raised mood by 4 points, η²_p = .10, F(1, 92) = 10.67, p = .002, though the simple effects show that gain came almost entirely from the walk.

Report every cell's M and SD, or show them in a table or an interaction plot, so readers can see the pattern the Fs summarize.
Lead with the interaction when it's significant, and unpack it with simple effects before you describe the main effects.
Give an effect size and both degrees of freedom for every effect, and say it's partial η² if it is.
With unequal cell sizes, say which type of sums of squares you used; with equal cells, like these, the types agree. Type II adjusts each main effect for the other main effect but not for the interaction, which gives it more power when there's no interaction (Langsrud, 2003); Type III tests main effects averaged equally over the cells and is SPSS's default. Pick one in advance and name it. With a meaningful interaction, read the main effects cautiously under either type.

Reading Each Piece

The equation form above, piece by piece, in this order: how big, then how surprising.

Interaction: 6.00 points, 95% CI [1.14, 10.86]	A friend lifted mood 6 points more on a walk (7 points) than while scrolling (1 point). This is the pattern the interaction's F tests. The true difference could plausibly be anywhere from about 1 to 11 points.
η²_p = .06	The interaction accounts for about 6% of the variation that isn't already explained by activity or company, a medium effect by the benchmarks above. Activity's .21 is large and company's .10 medium; whether a friend on a walk matters is a question about wellbeing, not statistics. Benchmarks ↑
F(1, 92) = 6.00	The interaction's mean square is 6 times the within-cell variance, which is about the size it would be, on average, if there were no interaction. The 1 and 92 are the degrees of freedom: (a − 1)(b − 1) for the interaction and N − ab within cells. Activity's F is 24.00 and company's 10.67, each on 1 and 92.
p = .016	If a friend's effect were really the same on a walk as while scrolling, an interaction F of 6.00 or more would turn up about 1.6% of the time, under the test's assumptions. That's below .05, so the interaction is statistically significant, as are both main effects (p = .002 and p < .001). Benchmarks ↑

Calculations

The Running Example

Example

The break study: 96 undergraduates, 24 in each of four cells (scroll or walk × alone or with a friend), each with one positive-mood score from 10 to 50.

	Null Hypothesis	Alternative
Activity	$\mu_{\text{walk}} = \mu_{\text{scroll}}$	$\mu_{\text{walk}} \neq \mu_{\text{scroll}}$
Company	$\mu_{\text{friend}} = \mu_{\text{alone}}$	$\mu_{\text{friend}} \neq \mu_{\text{alone}}$
Interaction	$\mu_{\text{walk, friend}} - \mu_{\text{walk, alone}} = \mu_{\text{scroll, friend}} - \mu_{\text{scroll, alone}}$	$\mu_{\text{walk, friend}} - \mu_{\text{walk, alone}} \neq \mu_{\text{scroll, friend}} - \mu_{\text{scroll, alone}}$

Three pairs of hypotheses, one per effect. For the interaction, the null says the friend's effect $\mu_{\text{walk, friend}} - \mu_{\text{walk, alone}}$ is the same as $\mu_{\text{scroll, friend}} - \mu_{\text{scroll, alone}}$ , where each $\mu$ (“mu”) is a population mean mood. For activity, the null says walkers and scrollers average the same once we average over company; for company, the same for alone versus with a friend. None of the alternatives has a direction.

What Your Data Should Look Like

One row per person, one column for each factor naming that person's level, and one numeric column for their score. Each person appears once, in exactly one cell. A column naming the combined cell (like walk_friend) is optional; it's handy for plotting and for software that runs simple effects as a one-way analysis.

Example

participant	activity	company	cell	mood
1	scroll	friend	scroll_friend	27
2	scroll	alone	scroll_alone	35
3	walk	friend	walk_friend	38
4	walk	alone	walk_alone	37
5	walk	alone	walk_alone	27
6	scroll	alone	scroll_alone	26
… 90 more rows

	Alone	With a Friend	All
Scroll	28.00 (5.93)	29.00 (6.00)	28.50
Walk	31.00 (6.01)	38.00 (6.06)	34.50
All	29.50	33.50	31.50

Cell M (SD), activity by company. All gives the marginal means; the grand mean is 31.50.

Between subjects: n = 24 per cell, 96 in all.

Simulated data.

Download all 96 rows (data.csv)

Napkin Math

The whole test, one idea at a time. Under each heading, a line says why the step matters, and the green note reads the equation in plain English with the break study's numbers. Collapse all to read just the headings: they tell the whole story on their own.

1. Find the Average in Each Cell and Overall

A factorial design is a grid of groups, one for every combination of the factors. Everything starts from the grid's cell averages and the average of everyone.

$\bar{x}_{\text{grand}} = \dfrac{\sum x_i}{N}$ = $\dfrac{3024}{96}$

= 31.50 points

$\bar{x}_{\text{scroll, alone}}$ = 28.00
$\bar{x}_{\text{scroll, friend}}$ = 29.00
$\bar{x}_{\text{walk, alone}}$ = 31.00
$\bar{x}_{\text{walk, friend}}$ = 38.00

$\sum x_i$ is all 96 mood scores added up, 3024, and $N$ is our 96 students, so the overall average is 31.50 points. The four cell averages are on the right: scrolling alone 28, scrolling with a friend 29, walking alone 31, walking with a friend 38.
2. Check Whether the Friend's Effect Depends on the Activity

Read the interaction before anything else. If a friend changes mood by the same amount on a walk as while scrolling, each factor can be summarized on its own; if not, those summaries hide what happened.

$(\bar{x}_{22} - \bar{x}_{21}) - (\bar{x}_{12} - \bar{x}_{11})$ = $(38 - 31) - (29 - 28)$

= 6.00 points

$95\%\ \text{CI}$ = [1.14, 10.86] points

The first subscript is the activity (1 scroll, 2 walk) and the second the company (1 alone, 2 friend), so $\bar{x}_{22}$ is walking with a friend. $(\bar{x}_{22} - \bar{x}_{21})$ is the friend's effect on a walk, 38 − 31 = 7 points, and $(\bar{x}_{12} - \bar{x}_{11})$ is its effect while scrolling, 29 − 28 = 1 point. 7 − 1 = 6: a friend helped 6 points more on the walk.
3. Average over the Other Factor to Get Each Factor's Overall Effect

A main effect compares the levels of one factor after averaging over the other. It's a fair summary only if the other factor doesn't change the story, which step 2 says it partly does.

$\bar{x}_{2\cdot} - \bar{x}_{1\cdot}$ = $\dfrac{31 + 38}{2} - \dfrac{28 + 29}{2}$

= 6.00 points

$\text{company}$ = 4.00 points

The dot means averaged over company: $\bar{x}_{2\cdot}$ is the walkers' average over both kinds of company, 34.5, and $\bar{x}_{1\cdot}$ the scrollers', 28.5, so walking comes out 6 points ahead. The same move for company, 33.5 − 29.5, gives a friend 4 points, but that 4 averages a 7 and a 1.
4. Split the Spread between Cells into Activity, Company and Their Combination

The four cell averages spread around the overall average. When every cell has the same number of people, that spread breaks cleanly into three pieces, one per effect, each measured the same way as in a one-way ANOVA. With unequal cells the pieces overlap, and software has to choose how to divide the overlap (the types of sums of squares in How It's Reported).

$SS_{\text{cells}} = SS_A + SS_B + SS_{A \times B}$ = $864 + 384 + 216$

= 1464

$SS_{\text{activity}}$ = 864
$SS_{\text{company}}$ = 384
$SS_{\text{interaction}}$ = 216

A sum of squares adds up squared distances from the overall average, one for each student. Each activity level holds 48 students (two cells of 24), so $SS_A$ uses the activity averages: 48 × [(28.5 − 31.5)² + (34.5 − 31.5)²] = 864. $SS_B$ does the same for company: 48 × [(29.5 − 31.5)² + (33.5 − 31.5)²] = 384. The four cell averages sit 3.5, 2.5, 0.5 and 6.5 points from 31.5; squared, those are 12.25, 6.25, 0.25 and 42.25, and with 24 students per cell they give 24 × (12.25 + 6.25 + 0.25 + 42.25) = 1464. The interaction $SS_{A \times B}$ is what's left, 1464 − 864 − 384 = 216.
5. Measure How Much Students Vary within Their Own Cell

Students who had the same break still feel differently afterward. That spread is the noise every effect has to stand out from, and all three effects share it.

$MS_{\text{within}} = \dfrac{\sum (n_{jk} - 1)\, s_{jk}^2}{N - ab}$ = $\dfrac{23(5.93^2 + 6.00^2 + 6.01^2 + 6.06^2)}{96 - 4}$

= 36.00

$SS_{\text{within}}$ = 3312

$s_{jk}$ is each cell's standard deviation and $n_{jk} - 1 = 23$ . The top adds to 3312, and $N - ab$ = 96 − 4 = 92, since each of the 4 cells used up one piece of information on its own average. 3312 ÷ 92 = 36.00, a pooled SD of 6 points.
6. Compare Each Effect with the Noise

Each effect gets its own F, its spread per degree of freedom divided by the shared noise. With two levels per factor, every effect has 1 degree of freedom, so its sum of squares is already its mean square.

$F = \dfrac{SS_{\text{effect}} / df_{\text{effect}}}{MS_{\text{within}}}$ = $\dfrac{216 / 1}{36}$

= 6.00

$F_{\text{activity}}$ = 24.00
$F_{\text{company}}$ = 10.67

For the interaction, $SS_{\text{effect}}$ is 216 and $df_{\text{effect}} = (2 - 1)(2 - 1) = 1$ , over $MS_{\text{within}}$ = 36: F = 6.00. Activity gives 864 ÷ 36 = 24.00 and company 384 ÷ 36 = 10.67. The interval in step 2 uses the same noise. The interaction combines four cell averages of 24 students, so its standard error is √(36 × 4 ÷ 24) = 2.45 points, and 1.99 is the t value that cuts off the top 2.5% with 92 df: 6 ± 1.99 × 2.45 runs from 1.14 to 10.86. The 6 points divided by 2.45 is t = 2.45, and t² = 6.00 is the interaction's F.
7. Measure How Much of the Variation Each Effect Explains

F grows with sample size; partial η² doesn't. For each effect it asks what share of the variation the effect accounts for, setting the other effects aside.

$\eta^2_p = \dfrac{SS_{\text{effect}}}{SS_{\text{effect}} + SS_{\text{within}}}$ = $\dfrac{216}{216 + 3312}$

= .06

$\eta^2_{p,\,\text{activity}}$ = .21
$\eta^2_{p,\,\text{company}}$ = .10

$SS_{\text{effect}}$ is the interaction's 216 and $SS_{\text{within}}$ is 3312. 216 ÷ 3528 = .06. Activity gives 864 ÷ 4176 = .21 and company 384 ÷ 3696 = .10.
8. Ask How Surprising Each Result Would Be If There Were No Real Effect

For each effect, ask how often chance alone would produce an F at least this large if that effect were truly zero.

$p = P(F_{df_1,\,df_2} \ge F)$ = $P(F_{1,\,92} \ge 6.00)$

= .016

$p_{\text{company}}$ = .002

$F_{1,\,92}$ is an F distribution with 1 and 92 df. For the interaction, the chance it lands at or above 6.00 is .016, about 1 in 62. Company's 10.67 gives .002, and activity's 24.00 gives less than .001.
9. Unpack the Interaction by Testing the Friend's Effect within Each Activity

A significant interaction says the friend's effect differs between activities, not what it is in each. Simple effects test it at each level, using the same shared noise and its 92 degrees of freedom.

$\bar{x}_{\text{walk, friend}} - \bar{x}_{\text{walk, alone}}$ = $38 - 31$

= 7.00 points

$F_{\text{walk}}$ = 16.33
$\text{CI}_{\text{walk}}$ = [3.56, 10.44] points
$\text{scrolling}$ = 1.00 points
$F_{\text{scroll}}$ = 0.33
$p_{\text{scroll}}$ = .565
$\text{CI}_{\text{scroll}}$ = [−2.44, 4.44] points

On a walk, the friend's effect is 38 − 31 = 7 points. Its test is a one-way F within the walkers, $F = n \sum_k (\bar{x}_{jk} - \bar{x}_{j\cdot})^2 / MS_{\text{within}}$ , where $j$ marks the activity, $k$ the company and $\bar{x}_{j\cdot}$ that activity's average. Each walk cell sits 3.5 points from the walkers' 34.5 (38 − 34.5 and 31 − 34.5), so 24 × (3.5² + 3.5²) = 588, over 36, gives F = 16.33, p < .001. Its interval is 7 ± 1.99 × √(36 × 2 ÷ 24) = 7 ± 3.44, from 3.56 to 10.44. While scrolling it's 29 − 28 = 1 point: 24 × (0.5² + 0.5²) = 12, over 36, is F = 0.33, p = .565.

Run It in Software

The code below reads the running example's data from data.csv; save it in the same folder you run the code from.

SPSS (version 30, as documented by IBM)

Analyze → General Linear Model → Univariate.
Move mood into Dependent Variable, and activity and company into Fixed Factor(s).
Plots: put company on Separate Lines and activity on Horizontal Axis, click Add, then Continue.
EM Means: move activity*company into Display Means for, then Continue. (Simple effects need the syntax below.)
Options: check Descriptive statistics, Estimates of effect size and Homogeneity tests, and click OK. Model stays at Full factorial with Type III sums of squares.

Or paste this syntax:

UNIANOVA mood BY activity company
  /METHOD=SSTYPE(3)
  /INTERCEPT=INCLUDE
  /PLOT=PROFILE(activity*company) TYPE=LINE
  /EMMEANS=TABLES(activity*company) COMPARE(company) ADJ(BONFERRONI)
  /PRINT=DESCRIPTIVE ETASQ HOMOGENEITY
  /CRITERIA=ALPHA(.05)
  /DESIGN=activity company activity*company.

Reading the Output

Read the tables in this order: the cell Means in Descriptive Statistics (or the profile plot), then the activity * company row of Tests of Between-Subjects Effects (its Partial Eta Squared, F and Sig.), then the simple effects in Univariate Tests, and only then the two main-effect rows.

Tests of Between-Subjects Effects (Dependent Variable: mood)

Source	Type III Sum of Squares	df	Mean Square	F	Sig.	Partial Eta Squared
Corrected Model	1464.000ᵃ	3	488.000	13.556	<.001	.307
Intercept	95256.000	1	95256.000	2646.000	<.001	.966
activity	864.000	1	864.000	24.000	<.001	.207
company	384.000	1	384.000	10.667	.002	.104
activity * company	216.000	1	216.000	6.000	.016	.061
Error	3312.000	92	36.000
Total	100032.000	96
Corrected Total	4776.000	95

a. R Squared = .307 (Adjusted R Squared = .284)

The Corrected Model row tests all three effects at once, like a one-way ANOVA on the four cells; the Intercept row tests whether the overall mean is zero, which nobody asked. Skip both.

Univariate Tests (the effect of company within each activity)

activity		Sum of Squares	df	Mean Square	F	Sig.	Partial Eta Squared
scroll	Contrast	12.000	1	12.000	.333	.565	.004
	Error	3312.000	92	36.000
walk	Contrast	588.000	1	588.000	16.333	<.001	.151
	Error	3312.000	92	36.000

Each test uses the full design's error term, with 92 df. With two levels of company there's one comparison per activity, so the Bonferroni adjustment in the Pairwise Comparisons table changes nothing; to correct across the two simple effects, apply Holm to these two p-values yourself.

Output laid out from this page's computed results, not captured from a live run; labels and layout may differ slightly in your version.

R

breaks <- read.csv("data.csv", stringsAsFactors = TRUE)

# Sum-to-zero coding, so Type III tests the main effects you mean
options(contrasts = c("contr.sum", "contr.poly"))
fit <- lm(mood ~ activity * company, data = breaks)
car::Anova(fit, type = 3)

# Simple effects of company within each activity, using the full error term
library(emmeans)
joint_tests(fit, by = "activity")

# Interaction plot
with(breaks, interaction.plot(activity, company, mood))

Anova Table (Type III tests)

Response: mood
                 Sum Sq Df  F value    Pr(>F)
(Intercept)       95256  1 2646.000 < 2.2e-16 ***
activity            864  1   24.000 4.111e-06 ***
company             384  1   10.667  0.001533 **
activity:company    216  1    6.000  0.016199 *
Residuals          3312 92
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

activity = scroll:
 model term df1 df2 F.ratio p.value
 company      1  92   0.333  0.5651

activity = walk:
 model term df1 df2 F.ratio p.value
 company      1  92  16.333  0.0001

R doesn't print partial η²; compute it from the table as each effect's Sum Sq over itself plus the Residuals' 3312, or use effectsize::eta_squared(fit, partial = TRUE). Without the options(contrasts = …) line, car::Anova(type = 3) gives activity F = 3.00 and company F = 0.33: those are the simple effects at the first level of the other factor, not main effects. summary(aov(...)) gives the right answer here only because the cells are equal.

Output laid out from this page's computed results, not captured from a live run; labels and layout may differ slightly in your version.

Python

import pandas as pd
import pingouin as pg
from scipy import stats
import statsmodels.formula.api as smf
from statsmodels.stats.anova import anova_lm

breaks = pd.read_csv("data.csv")

# Sum-to-zero coding, so Type III tests the main effects you mean
fit = smf.ols("mood ~ C(activity, Sum) * C(company, Sum)", data=breaks).fit()
print(anova_lm(fit, typ=3))

# The same table with partial eta squared
print(pg.anova(data=breaks, dv="mood", between=["activity", "company"],
               ss_type=3, effsize="np2").round(4))

# Simple effects of company within each activity, using the pooled MS_within
ms_within, df_within = fit.mse_resid, fit.df_resid
for activity, rows in breaks.groupby("activity"):
    means = rows.groupby("company")["mood"].agg(["mean", "size"])
    diff = means.loc["friend", "mean"] - means.loc["alone", "mean"]
    F = diff**2 / (ms_within * (1 / means.loc["friend", "size"] + 1 / means.loc["alone", "size"]))
    print(activity, round(diff, 2), round(F, 3), round(stats.f.sf(F, 1, df_within), 4))

                                   sum_sq    df            F        PR(>F)
Intercept                         95256.0   1.0  2646.000000  1.375400e-69
C(activity, Sum)                    864.0   1.0    24.000000  4.110959e-06
C(company, Sum)                     384.0   1.0    10.666667  1.533351e-03
C(activity, Sum):C(company, Sum)    216.0   1.0     6.000000  1.619920e-02
Residual                           3312.0  92.0          NaN           NaN
               Source      SS  DF     MS        F   p-unc     np2
0            activity   864.0   1  864.0  24.0000  0.0000  0.2069
1             company   384.0   1  384.0  10.6667  0.0015  0.1039
2  activity * company   216.0   1  216.0   6.0000  0.0162  0.0612
3            Residual  3312.0  92   36.0      NaN     NaN     NaN
scroll 1.0 0.333 0.5651
walk 7.0 16.333 0.0001

np2 is partial η². pingouin's 0.0000 for activity means p < .001 (it's 0.000004). Each simple-effect line gives the friend's effect in points, its F on 1 and 92 df, and p.

Output laid out from this page's computed results, not captured from a live run; labels and layout may differ slightly in your version.

Impact

Effect Size

Partial eta squared (η²_p) expresses each effect as the share of variation it accounts for, out of the variation the design's other effects leave unexplained. It has no units, so it can be compared across scales, but it depends on the design: a factor that was measured rather than assigned, such as gender, takes its variation out of the denominator and makes the other effects' partials larger. It compares well only across studies with the same design. To compare with other designs, use generalized eta squared (Olejnik & Algina, 2003; Lakens, 2013); with both factors assigned, as here, it equals partial η².

\eta^2_p = \dfrac{SS_{\text{effect}}}{SS_{\text{effect}} + SS_{\text{within}}} = \dfrac{216}{216 + 3312} = .06

Setting aside what activity and company explain on their own, the change in the friend's effect accounts for about 6% of the remaining differences in mood, a medium effect by convention.

Each effect gets its own partial η²: .21 for activity, .10 for company and .06 for the interaction. They don't add up to anything, because each has its own denominator. Plain eta squared, which divides by the total sum of squares (4776), gives smaller values, .18, .08 and .05, which do add up to the model's share, .31. Report partial η² if your field expects it, which most psychology journals do, and say which one you report (Pierce et al., 2004).

For the interaction, the clearest size is in points. A friend lifted mood by 7 points on a walk, about 1.17 pooled standard deviations, and by 1 point while scrolling. Benchmarks are conventions. Whether a better mood after one break is worth planning a walk with a friend is a question about wellbeing, not statistics.

Benchmarks Partial Eta Squared at a Glance

Below Small

η²ₚ < .01

Small

.01 ≤ η²ₚ < .06

Medium

.06 ≤ η²ₚ < .14

Large

η²ₚ ≥ .14

Full benchmarks and caveats ↑

Statistical Significance

A factorial ANOVA gives three p-values, one per effect, and each answers its own narrow question: if that effect were zero in the population, how often would an F at least this large turn up? For the interaction, the null is that a friend changes mood by the same amount whatever the activity. Only a large F counts against the null, so p comes from the upper tail of the F distribution. Below the conventional α = .05, an effect is called statistically significant.

If a friend's effect were really the same on a walk as while scrolling, an interaction F of 6.00 or more would turn up about 1.6% of the time, roughly 1 in 62, under the test's assumptions. F measures the gap between the friend effects (7 points and 1 point) relative to the variation within cells. That's below .05, so the interaction is significant.

p is not the probability that the null hypothesis is true, and it says nothing about how large an effect is or where it comes from. A significant interaction doesn't say which simple effects are real; that's what the simple effects analysis is for. And one significant simple effect next to one non-significant one is not, by itself, evidence of an interaction (Gelman & Stern, 2006). Each extra effect is another chance for a false positive: a two-way design runs three tests and a three-way design seven. If you didn't predict an effect, read its p with that in mind.

Benchmarks p at a Glance

Very Strong

p < .001

Strong

.001 ≤ p < .01

Significant

.01 ≤ p < .05

Weak

.05 ≤ p < .10

Little

p ≥ .10

Strength of evidence against the null hypothesis.

Full benchmarks and caveats ↑

Power and Sample Size

Power is the chance that a study detects an effect that's really there. A real effect doesn't guarantee a significant result: with too few people, it can easily come out non-significant. In a factorial design, each effect has its own power, which depends on its size, the total number of people, its degrees of freedom and α.

Power for one effect uses Cohen's f for that effect, $f = \sqrt{\eta^2_p / (1 - \eta^2_p)}$ , with the total sample size rather than the number per cell: the noncentral F distribution with λ = f² × N, the effect's degrees of freedom and N minus the number of cells (the convention G*Power uses). Each effect in a 2 × 2 design has 1 degree of freedom, so a medium effect, f = 0.25, needs 128 people in all, 32 per cell, for 80% power.

Our interaction's η²_p of .06 works out to f = 0.26. With our 96 students, 24 per cell, a true interaction of f = 0.25 would come out significant only about 68% of the time. Planning for 80% would have taken 32 students per cell, 128 in all.

Interactions are usually the hard part. They are often smaller than the main effects in the same study, and an effect that shrinks rather than reverses (an ordinal interaction), like the friend's 7 points versus 1, makes a smaller interaction than it looks. A shrinking interaction can also come and go with the scale, for example after a log transform; a crossover, where the effect reverses, usually can't be removed that way. Plan the sample around the interaction if the interaction is your question, and don't plan from an observed effect size: with 24 per cell, it's a noisy estimate.

Researchers usually aim for at least 80% power. Plan it before collecting data, using the smallest effect that would matter. Try it:

Other Info

Follow-Up and Post Hoc Tests

Planned Contrasts

Planned Contrasts: Use when theory predicted a specific pattern before you collected data, such as “a friend helps more on a walk than while scrolling.” In a 2 × 2 design the interaction is that contrast. A prediction about a specific pattern of cells, like one cell standing out, needs its own contrast weights. In larger designs the interaction has (a − 1)(b − 1) degrees of freedom and no single difference of differences, so test the contrasts you planned directly.

Post Hoc Tests

Simple Effects Analysis: The usual next step after a significant interaction: test one factor's effect at each level of the other. In the break study, a friend lifted mood on a walk by 7 points, F(1, 92) = 16.33, p < .001, but not while scrolling, 1 point, F(1, 92) = 0.33, p = .565. When the factor being tested has three or more levels, a significant simple effect needs pairwise comparisons within it.
Holm Correction: Correct a set of simple effects so the chance of any false positive stays at α. Here the two simple effects give Holm-adjusted p < .001 for the walk and p = .565 for scrolling.
Bonferroni Correction: Works for any set of simple effects or contrasts, but it's conservative; Holm gives the same protection with more power.
Tukey's HSD: Needed only for a factor with three or more levels, to compare its levels pairwise after a significant main effect. Both factors in the break study have two levels, so the F already says which level is higher.

Related Methods

Alternative MeasureOrdinal Regression: For an outcome made of a few ordered categories (a 1–5 rating, say) rather than a meaningfully continuous score. It can include both factors and their interaction, but it models the chances of each category, not the means, so it answers a different question. A skewed continuous score calls for a method that still compares means, not this.
General CaseMultiple Linear Regression: Code each two-level factor as −1 and +1 and add their product as a third predictor: the regression's tests of the three coefficients are these three F tests (F = t²). ANOVA is regression with categorical predictors and their interactions.
Related DesignFactorial Repeated-Measures ANOVA: The same main effects and interaction, when the same people go through every combination of conditions.
Related DesignMixed Factorial ANOVA: The same questions when one factor varies between people and the other within them, such as treatment group by time.
Related DesignBayesian ANOVA: The Bayesian approach, shown there for one grouping variable. With two factors, the same default Bayes factors compare models with and without each main effect and the interaction, weighing the evidence for each effect instead of giving a p-value.
Related DesignModeration Analysis: the same interaction question when one or both factors are continuous scores. Its product term's coefficient is the slope version of a difference of differences.
Special CaseOne-Way Between-Subjects ANOVA: With a single grouping factor, a factorial ANOVA is a one-way ANOVA. There's one main effect and no interaction.
AlternativeMultilevel Regression (HLM): An alternative if “Independent Students” doesn't hold. A multilevel model with pairs (or groups, or classrooms) as clusters, which accounts for the shared variation.

References

Fisher, R. A. (1925). Statistical methods for research workers. Oliver and Boyd.
Yates, F. (1934). The analysis of multiple classifications with unequal numbers in the different classes. Journal of the American Statistical Association, 29(185), 51–66. https://doi.org/10.1080/01621459.1934.10502686
Cohen, J. (1973). Eta-squared and partial eta-squared in fixed factor ANOVA designs. Educational and Psychological Measurement, 33(1), 107–112. https://doi.org/10.1177/001316447303300111
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.
Rosnow, R. L., & Rosenthal, R. (1989). Definition and interpretation of interaction effects. Psychological Bulletin, 105(1), 143–146. https://doi.org/10.1037/0033-2909.105.1.143
Gelman, A., & Stern, H. (2006). The difference between “significant” and “not significant” is not itself statistically significant. The American Statistician, 60(4), 328–331. https://doi.org/10.1198/000313006X152649
“In making a comparison between two treatments, one should look at the statistical significance of the difference rather than the difference between their significance levels.” (p. 329)
Nieuwenhuis, S., Forstmann, B. U., & Wagenmakers, E.-J. (2011). Erroneous analyses of interactions in neuroscience: A problem of significance. Nature Neuroscience, 14(9), 1105–1107. https://doi.org/10.1038/nn.2886
“In theory, a comparison of two experimental effects requires a statistical test on their difference.” (Abstract, p. 1105)
Langsrud, Ø. (2003). ANOVA for unbalanced data: Use Type II instead of Type III sums of squares. Statistics and Computing, 13(2), 163–167. https://doi.org/10.1023/A:1023260610025
Olejnik, S., & Algina, J. (2003). Generalized eta and omega squared statistics: Measures of effect size for some common research designs. Psychological Methods, 8(4), 434–447. https://doi.org/10.1037/1082-989X.8.4.434
Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, 863. https://doi.org/10.3389/fpsyg.2013.00863
“using these benchmarks when interpreting the η2p effect size in designs that include covariates or repeated measures is not consistent with the considerations upon which the benchmarks were based” (Eta-squared in between and within-subjects comparisons)
Blanca, M. J., Alarcón, R., Arnau, J., Bono, R., & Bendayan, R. (2017). Non-normal data: Is ANOVA still a valid option? Psicothema, 29(4), 552–557. https://doi.org/10.7334/psicothema2016.383
“The results showed that in terms of Type I error the F-test was robust in 100% of the cases studied, independently of the manipulated conditions.” (Abstract, Results)
Delacre, M., Leys, C., Mora, Y. L., & Lakens, D. (2019). Taking parametric assumptions seriously: Arguments for the use of Welch's F-test instead of the classical F-test in one-way ANOVA. International Review of Social Psychology, 32(1), 13. https://doi.org/10.5334/irsp.198
Maxwell, S. E., Delaney, H. D., & Kelley, K. (2018). Designing experiments and analyzing data: A model comparison perspective (3rd ed.). Routledge. https://doi.org/10.4324/9781315642956
G*Power 3.1 manual. (2023, June 1). Heinrich-Heine-Universität Düsseldorf. https://www.psychologie.hhu.de/fileadmin/redaktion/Fakultaeten/Mathematisch-Naturwissenschaftliche_Fakultaet/Psychologie/AAP/gpower/GPowerManual.pdf
“noncentrality parameter λ = f 2 N” (§11.5)
Pierce, C. A., Block, R. A., & Aguinis, H. (2004). Cautionary note on reporting eta-squared values from multifactor ANOVA designs. Educational and Psychological Measurement, 64(6), 916–924. https://doi.org/10.1177/0013164404264848
Muff, S., Nilsen, E. B., O'Hara, R. B., & Nater, C. R. (2022). Rewriting results sections in the language of evidence. Trends in Ecology & Evolution, 37(3), 203–210. https://doi.org/10.1016/j.tree.2021.10.009
“we suggest language of evidence that allows for a more nuanced approach to communicate scientific findings as a simple and intuitive alternative to statistical significance testing” (Abstract)