Stepwise Regression: Interpretation and APA Reporting

The Basics

When to Use It

Stepwise regression is for one continuous outcome and a list of candidate predictors, when you have no theory about which of them matter and want the data to suggest a shortlist. Treat it as exploration: whatever it picks is a hypothesis to test in new data, not a finding. If you already know which predictors you care about, fit them all at once with multiple regression, or in an order you choose with hierarchical regression. If your real goal is the best prediction, cross-validated penalized regression such as the lasso (Tibshirani, 1996) usually does better.

There are three versions. Forward selection starts with no predictors and only adds. Backward elimination starts with every candidate and drops the weakest one at a time. Stepwise proper, introduced by Efroymson (1960) and shown on this page, adds like forward selection but after each step checks whether anything already in should come out. SPSS offers all three in its Method menu. Backward is less likely to miss a predictor that only helps alongside another (a suppressor), because it starts with everything in. The three can pick different models from the same data (Derksen & Keselman, 1992, compared them); in the GPA study all three happen to choose the same four. Every problem on this page applies to all of them.

Questions people bring to stepwise:

Which everyday habits predict first-year students' GPA?
Which of twelve intake questionnaire scores predict how many therapy sessions a client attends?
Which of ten personality and mood measures predict daily step counts in older adults?

Stepwise will produce an answer to each, and the answer will look confident. The rest of this page is about how much that confidence is worth.

Vocabulary You Need

Definition Entry and Removal Criteria

The rules an automated selection procedure uses to decide what goes in the model. A candidate enters if its p to enter (from the F test for adding it to the predictors already in) is below the entry cutoff; a predictor already in is removed if its p to remove rises above the removal cutoff. The removal cutoff is set looser than the entry cutoff so the same predictor can't cycle in and out forever. These p-values steer the search; they are not tests of pre-specified hypotheses. Glossary →

Definition Overfitting

Fitting a model so closely to one sample that it captures that sample's chance quirks along with the real pattern. An overfitted model looks better on the data it was built from than it will on anyone else: its R² is too high, its coefficients too far from zero, its p-values too small. The risk grows with the number of predictors or choices tried relative to the sample size, and it's worst when the data themselves were used to decide what goes in the model. The honest check is performance on data the model hasn't seen. Glossary →

Definition Cross-Validation

Judging a model by how well it predicts cases that weren't used to build it. The simplest version holds out part of the sample: build the model on one part, then apply that exact equation, unchanged, to the other. K-fold cross-validation splits the data into K parts and takes turns, so every case is used for testing once. Fitting the same model again in new data is a replication, not cross-validation; the test is whether the original equation's predictions hold up. Any selection of predictors has to be repeated inside each training part, or the test is contaminated. Glossary →

Assumptions

1. Independent Observations The people (or other units) in your study don't influence each other.

What It Means: Each person's data comes from a separate, unconnected source, so knowing one person's scores tells you nothing about anyone else's. In designs that measure people more than once, their own scores are meant to be related; it's different people who must be independent. Methods built for clustered data, such as multilevel models and GEE, also allow members of the same group to be related, as long as the analysis names the grouping; then it's the groups that must be independent.
Applies To: Observations
How to Check: By design, not by a test. Did each person contribute their own data, with no clusters such as roommates, couples, therapy groups or classrooms whose members influence each other? If there are clusters, does the analysis model every one of them?
How Much It Matters: A lot. Students who share a class or a household make the standard errors too small, so every p to enter comes out too small and more predictors get in.

If It's Violated

Alternative Test Multilevel Regression (HLM). If students are clustered in classes or schools, a multilevel model accounts for the clustering.
Design Change Collect one row per independent student.

2. Linear Relationship The relationship between your variables follows a straight line, not a curve.

What It Means: As one variable goes up, the other tends to change by about the same amount across its whole range. A curved relationship, one that levels off or a U shape, breaks this.
Applies To: Relationship
How to Check: Look at a scatterplot of each pair of variables the analysis relates: a roughly oval cloud tilted one way is fine; a clear bend, a ceiling or a U is not. In a regression with several predictors, plot the residuals against the predicted values instead: a flat, even band is fine; a curve means the model is missing a bend. A component-plus-residual plot for each predictor (R: car::crPlots) shows whether one of them bends once the others are held constant.
How Much It Matters: A lot, for which predictors get chosen. Each p to enter only measures a straight-line contribution, so a predictor with a curved relationship (sleep that helps up to a point, then doesn't) can be passed over. In the GPA study, plots of GPA against each candidate show no clear bends.

If It's Violated

Transformation Add a squared term or transform the predictor, decided before the search, not after seeing which version enters.
Proceed with Caveat Plot the outcome against every candidate before running anything, and say what you saw.

3. Normality The scores the test relies on are roughly bell-shaped, or your sample is big enough that it doesn't matter.

What It Means: The test's model assumes the scores it relies on are roughly normal: the scores within each group (or the scores themselves, for a one-sample test), the difference scores in a paired design, or the residuals (the errors left around a model's predictions) in a regression. For p-values and confidence intervals, what really matters is that the estimates (such as a mean, a difference between means or a model's coefficients) are roughly normal across repeated samples. A large enough sample usually provides that; how large depends on how skewed the scores are and how extreme their outliers.
Applies To: Residuals
How to Check: Look at a histogram or Q–Q plot for strong skew or extreme outliers. Don't rely on a normality test: with small samples it misses real problems, and with large ones it flags trivial ones.
How Much It Matters: Less than the selection itself. With 120 students, moderately non-normal residuals barely affect the F tests; the search does far more damage to them than skew does. The GPA study's residuals are mildly left-skewed (skewness −0.40).

If It's Violated

Robust Estimation Bootstrap the coefficients of a model specified in advance, rather than relying on the normal-theory standard errors.
Transformation Transform a strongly skewed outcome before the search. Results are then on the transformed scale.

4. Equal Spread of Residuals Your scores scatter about equally around the predictions, whatever the prediction.

What It Means: The residuals have about the same spread at every predicted value, so the model misses by about as much for people predicted to score low as for people predicted to score high. The opposite, heteroscedasticity, often looks like a funnel. If the model's form is right, unequal spread doesn't bias the coefficients, though it makes them less precise than they could be; it does make their standard errors, and so the p-values and confidence intervals, unreliable.
Applies To: Residuals
How to Check: Plot the residuals against the predicted values (in the SPSS REGRESSION procedure, *ZRESID against *ZPRED; for MIXED, save them with /SAVE=PRED RESID and plot those). A band of even width is fine; a funnel or fan that widens or narrows is not. Don't use a test such as Breusch–Pagan as a gatekeeper: with small samples it misses real problems, and with large ones it flags trivial ones.
How Much It Matters: Moderate. Uneven spread distorts each p to enter and so can change which predictors are chosen, on top of the search's own bias.

If It's Violated

Robust Estimation Use heteroscedasticity-consistent (robust) standard errors in a pre-specified model.
Transformation A log or square-root transformation can even out a fan that widens with the predicted value.

5. No Severe Multicollinearity Your predictors aren't so closely related that their effects can't be told apart.

What It Means: Each coefficient is estimated from the part of its predictor that doesn't overlap with the others. When predictors are highly correlated, that part is small, so the coefficients become unstable and their standard errors large, even though the model as a whole may predict well. Perfect overlap, such as including a total score and all of its parts, makes the model impossible to estimate.
Applies To: Predictors
How to Check: Look at the correlations among the candidates and at each one's variance inflation factor (VIF) with all of them entered together. A VIF near 1 means little overlap. SPSS reports tolerance, which is 1 ÷ VIF, so near 1 is good there too. Common cutoffs (VIF of 5 or 10) are rules of thumb, not tests (O'Brien, 2007).
How Much It Matters: A lot, and in a way specific to stepwise. When two candidates overlap (study hours and social media time, say), whichever edges ahead in this sample enters and the other is shut out, so the choice between them is close to a coin flip and can reverse in a new sample. With all eight candidates entered together, the GPA study's tolerances ranged from .73 (study hours) to .98; study hours and social media time correlate −.44.

If It's Violated

Design Change Combine strongly related candidates into one score, or choose between them on theoretical grounds, before any selection.
Alternative Test Multiple Linear Regression. Enter the predictors together as a block and interpret the set rather than any one coefficient.

Limitations

This Is Not Your Test If…

Theory tells you which predictors belong in the model Multiple Linear Regression →
You want to test what one set of predictors adds to another Hierarchical Regression →
You're choosing which covariates to control for Multiple Linear Regression →
Your outcome is yes/no rather than a score Logistic Regression →
Your students are grouped in classes or schools Multilevel Regression (HLM) →

What It Doesn't Tell You

Which predictors are real. Stepwise let caffeine in with p = .019, and in the simulation caffeine does nothing; in the holdout half its coefficient was 0.008 (p = .82). A selected predictor is a candidate for testing, not a discovery (Smith, 2018).
That the others don't matter. Work hours really do lower GPA a little in the simulation, and stepwise left them out. "Not selected" means "didn't win this round," often because a correlated rival or too few people kept it out.
How well it predicts. The 35% of variance the model claims is inflated, by the search and by ordinary overfitting. On the 120 held-out students, the same equation predicted 14%.
Why. Even the predictors that hold up are correlations. Nothing here shows that attending more classes raises GPA; motivated students may simply do both.
Which predictor matters most. The order of entry depends on the overlap among candidates in this particular sample, and a new sample can reorder it (Thompson, 1995).

Common Mistakes

Don'tTreat the predictors stepwise selected as confirmed findings.

InsteadTreat them as hypotheses from an exploratory search, and test them in new data.
Don'tReport stepwise p-values, confidence intervals or R² as if the model were chosen in advance.

InsteadSay the model came from stepwise selection, list every candidate, and report a holdout or cross-validated R².
Don'tConclude that a predictor stepwise left out has no effect.

InsteadSay the search didn't pick it. A correlated rival or low power can keep a real predictor out.
Don'tRank predictors' importance by the order they entered.

InsteadCompare standardized coefficients or R² change in a model you specified in advance.
Don'tUse stepwise to decide which confounders or covariates to control for.

InsteadChoose control variables from theory and prior research before looking at the outcome.
Don'tRead a p-value as the chance that the null hypothesis is true.

InsteadRead it as: if the null were true, how often data at least this extreme would turn up.

Result Decoder

Significance Guidelines

Benchmarks Significance (p Value)

p < .001
Very strong evidence against the null hypothesis.
.001 ≤ p < .01
Strong evidence against the null hypothesis.
.01 ≤ p < .05
Statistically significant at α = .05. Moderate evidence against the null hypothesis.
.05 ≤ p < .10
Not significant. Weak evidence; sometimes called a “trend.” Treat it as inconclusive.
p ≥ .10
Not significant. Little evidence against the null, which is not the same as evidence of no effect.

These evidence labels are one common convention for describing p in graded language rather than as significant or not; the boundaries are rough guides, not rules, and other authors draw them differently. Muff et al. (2022) argue for this kind of language of evidence over a single cutoff. α is chosen before the analysis, and p never says how big or important an effect is.

More about statistical significance ↓

How It's Reported

Equation Form

Example

R² = .35, adjusted R² = .33; holdout R² = .14; F(4, 115) = 15.47, p < .001, not adjusted for selection

Table 1. Stepwise Selection of Predictors of First-Year GPA (Selection Half, n = 120)

Example

	$F$ to Enter	$p$	$R^2$	$R^2_{\text{adj}}$
Step 1: Attendance	23.51	< .001	.166	.159
Step 2: Study hours	13.45	< .001	.252	.239
Step 3: Sleep	11.19	.001	.318	.300
Step 4: Caffeine (pure noise)	5.65	.019	.350	.327

Note. Entry criterion p < .05, removal criterion p > .10 (SPSS defaults); no predictor was removed. F to enter has 1 and 120 − k − 1 degrees of freedom, where k is the number of predictors after the step. None of these p-values accounts for the search. Caffeine has no effect on GPA in the simulation; its correlation with GPA in this half (the data box's .21) is chance, which is exactly what stepwise rewards. It entered as the best of five remaining candidates. The search stopped when the best remaining candidate, commute time, had p = .131.

Table 2. How the Selected Models Hold Up on Students They Never Saw

Example

	Predictors	$R^2$ , Selection Half	$R^2$ , Holdout Half
SPSS stepwise (p to enter .05)	4	.350	.136
R step() (AIC)	5	.363	.114
Fixed in advance (as if from prior research): attendance, study hours, sleep	3	.318	.149
All eight candidates, no selection	8	.378	.123

Note. Each model was chosen on the selection half (n = 120) and its equation applied, unchanged, to the holdout half (n = 120). SPSS stepwise chooses by p-value; R's step() chooses by AIC, a looser bar, and also let in commute time, a second predictor with no real effect. The last two rows use no search at all, and they drop too: part of the gap is ordinary overfitting and a holdout half that is harder to predict for every model, not the search.

Full Text (APA Style)

Example

In a random half of the sample (n = 120; GPA M = 2.98, SD = 0.49), stepwise regression (entry p < .05, removal p > .10) chose four of eight candidate predictors of first-year GPA: class attendance (b = 0.021 points per percentage point), weekly study hours (b = 0.026), nightly sleep (b = 0.105) and daily caffeinated drinks (b = 0.068). The selected model predicted 35% of the variance in GPA in the half it was chosen from (R² = .35, adjusted R² = .33). Applied unchanged to the other 120 students, the same equation predicted 14% (out-of-sample R² = .14). Refit in that half, caffeine's coefficient was close to zero (b = 0.008, 95% CI [−0.060, 0.076], p = .82), though an interval that wide can't rule out the original 0.068. The overall test of the selected model, F(4, 115) = 15.47, p < .001, does not account for the selection, so we treat these predictors as hypotheses for future work rather than confirmed effects.

Say the model came from stepwise selection, give the entry and removal criteria, and list every candidate, not only the winners.
Report how the equation does on data it wasn't chosen from: a holdout half or cross-validation.
Flag p-values, confidence intervals and R² from the selected model as unadjusted for the search, or leave the p-values out.
Better still, report a model specified in advance, and present any stepwise search as exploratory.

Reading Each Piece

The equation form above, piece by piece, in this order: how big, then how surprising.

R² = .35, adjusted R² = .33	The chosen model predicts 35% of the differences in GPA among the 120 students it was chosen from (33% after the usual adjustment). Both are overestimates: the predictors were picked because they fit this sample well.
holdout R² = .14	On students who played no part in choosing it, the equation predicts 14% of the differences in GPA, well under the 35% it claimed. Refitting the model in the holdout half, a separate check, shows where the selection half overreached: caffeine's coefficient falls to near zero (0.008, from 0.068), though one holdout half is too small to rule out the original value; attendance, study and sleep stay clear of zero, with smaller coefficients.
F(4, 115) = 15.47, p < .001	Taken at face value, the model predicts GPA far better than chance would. But the 4 and 115 degrees of freedom count only the predictors kept, not the eight tried, so this p is too small. It can't tell you which predictors are real.

Calculations

The Running Example

Example

The GPA study: 240 first-year students and eight candidate predictors of GPA. Stepwise selection runs on a random half of 120; the other 120 are held out to check what it chose.

Null Hypothesis:: $\beta_j = 0$
Alternative:: $\beta_j \neq 0$

At every step, stepwise tests this for each candidate $j$ not yet in the model: $\beta_j$ is the candidate's regression coefficient in the population, holding the predictors already in constant. Because the same data pick the candidates and then test them, none of these tests keeps its advertised 5% false-positive rate.

What Your Data Should Look Like

One row per person, one numeric column for the outcome and one for each candidate predictor. Add a column that splits people at random into a selection half and a holdout half, made before any analysis, so the chosen model can be checked on people who played no part in choosing it.

Example

student	half	gpa	attendance	study_hours	sleep_hours	work_hours	commute_min	social_media_hours	exercise_hours	caffeine_drinks
1	selection	2.52	74	6.8	7.3	0	20	2.6	5.7	0
2	selection	2.56	79	6.9	5.6	0	35	1.5	7.3	1
3	selection	3.07	80	13.0	7.3	9	16	3.7	7.0	2
4	selection	2.95	92	8.8	5.7	29	14	2.8	5.0	1
5	selection	3.37	79	20.1	6.9	0	24	4.2	3.5	3
6	selection	3.72	94	15.4	6.1	11	16	1.7	5.4	2
… 234 more rows

	M	SD	r with GPA
GPA	2.98	0.49	—
Attendance	85.79	8.22	.41
Study Hours	11.85	4.84	.39
Sleep Hours	6.94	1.06	.28
Work Hours	8.28	9.06	−.17
Commute Minutes	22.85	13.27	−.15
Social Media Hours	2.88	1.12	−.18
Exercise Hours	4.13	2.21	.08
Caffeine Drinks	1.43	1.35	.21

The selection half only: n = 120 of 240. The rest are held back to check the result.

With 9 variables, only each one's Pearson r with GPA is shown.

Simulated data.

Download all 240 rows (data.csv)

Napkin Math

The whole test, one idea at a time. Under each heading, a line says why the step matters, and the green note reads the equation in plain English with the GPA study's numbers. Collapse all to read just the headings: they tell the whole story on their own.

1. Try Every Candidate Alone and Let the Strongest In

Stepwise starts with no predictors. It tries each of the eight candidates on its own and asks how much of the variation in GPA it predicts. The best one enters if its p to enter is under .05.

$F = \dfrac{R^2_{\text{new}} - R^2_{\text{old}}}{(1 - R^2_{\text{new}})\,/\,(n - k - 1)}$ = $\dfrac{.166 - 0}{(1 - .166)\,/\,118}$

$F_{\text{attendance}}$ = 23.51
$R^2$ = .166

Attendance does best on its own: $R^2_{\text{new}}$ = .166 against $R^2_{\text{old}}$ = 0, with $n - k - 1$ = 120 − 1 − 1 = 118. That's $F$ = 23.51, $p$ < .001, far under .05, so attendance enters.
2. Add Whichever Candidate Adds the Most Now

With attendance in, each remaining candidate is judged by what it adds beyond attendance, not by what it does alone. That can change the ranking, because candidates that overlap with attendance have less left to add.

$F = \dfrac{R^2_{\text{new}} - R^2_{\text{old}}}{(1 - R^2_{\text{new}})\,/\,(n - k - 1)}$ = $\dfrac{.252 - .166}{(1 - .252)\,/\,117}$

$F_{\text{study}}$ = 13.45
$R^2$ = .252

Study hours add the most: $R^2$ climbs from .166 to .252, and with 117 degrees of freedom left, $F$ = 13.45, $p$ < .001. Study hours enter.
3. Keep Adding While Something Clears the Bar

The loop repeats. After each entry, stepwise also checks whether anything already in has become useless (p to remove above .10) and drops it. Nothing has so far.

$F = \dfrac{R^2_{\text{new}} - R^2_{\text{old}}}{(1 - R^2_{\text{new}})\,/\,(n - k - 1)}$ = $\dfrac{.318 - .252}{(1 - .318)\,/\,116}$

$F_{\text{sleep}}$ = 11.19
$R^2$ = .318

Sleep adds the most next: $R^2$ goes from .252 to .318, $F$ = 11.19 (from the unrounded $R^2$ 's; the rounded ones give 11.23), $p$ = .001. Sleep enters, and all three predictors still have $p$ to remove well under .10.
4. Let In One More That Only Just Clears It

The fourth predictor gets in with p = .019. In a model chosen in advance, that would be modest evidence. Here it's the best of five tries, which is a different thing.

$F = \dfrac{R^2_{\text{new}} - R^2_{\text{old}}}{(1 - R^2_{\text{new}})\,/\,(n - k - 1)}$ = $\dfrac{.350 - .318}{(1 - .350)\,/\,115}$

$F_{\text{caffeine}}$ = 5.65
$p$ = .019
$R^2$ = .350

Caffeine edges out the other four: $R^2$ goes from .318 to .350, $F$ = 5.65, $p$ = .019. It's under .05, so caffeine enters. In the simulation, caffeine has no effect on GPA at all.
5. Stop When No Remaining Candidate Clears the Bar

The search ends when the best remaining candidate's p to enter is .05 or more. Everything left out is simply what didn't win this round, not what was shown not to matter.

$p_{\text{enter}} = P(F_{1,\,114} \ge F_{\text{observed}})$ = $P(F_{1,\,114} \ge 2.32)$

$p_{\text{commute}}$ = .131

The best of the four left is commute time, with $F$ = 2.32 on 1 and 120 − 5 − 1 = 114 degrees of freedom (five predictors if commute got in), so $p$ = .131: the share of $F$ values at least this big that chance alone would produce with these degrees of freedom. That's over .05, so the search stops at four predictors. Work hours, which really do lower GPA a little in the simulation, stayed out ( $p$ = .193).
6. Measure How Much of GPA the Chosen Model Predicts

R² compares the model's misses with the total variation in GPA. Adjusted R² charges for each predictor kept. Neither charges for the candidates the search tried and threw away.

$R^2 = 1 - \dfrac{SS_{\text{residual}}}{SS_{\text{total}}}$ = $1 - \dfrac{18.63}{28.66}$

= .350

$R^2_{\text{adj}}$ = .327
$R^2_{\text{adj}}\ (k = 8)$ = .303

$SS_{\text{residual}}$ is the four-predictor model's 18.63 squared GPA points of miss, and $SS_{\text{total}}$ is 28.66. 1 − 18.63 ÷ 28.66 = .350. Adjusted for four predictors, 1 − .650 × 119 ÷ 115 = .327; counting all eight candidates the search looked at, 1 − .650 × 119 ÷ 111 = .303.
7. Test the Final Model as If It Had Been Chosen in Advance

Software then tests the final model with the usual F test. The test assumes the four predictors were picked before anyone looked at GPA. They weren't, so the F is too big and its p too small.

$F = \dfrac{R^2 / k}{(1 - R^2)\,/\,(n - k - 1)}$ = $\dfrac{.350 / 4}{(1 - .350)\,/\,115}$

= 15.47

$R^2$ is .350 and $k$ is the 4 predictors kept. .350 ÷ 4 = .0875, and .650 ÷ 115 = .00565, so $F$ = 15.47 and $p$ < .001. The search looked at 30 $p$ -values to get here, and this $F$ doesn't know about any of them.
8. Count the Chances Noise Had to Sneak In

At the step where caffeine entered, five candidates were competing. If all five were pure noise, the chance that at least one would clear .05 anyway is much bigger than .05.

$1 - (1 - \alpha)^m$ = $1 - (1 - .05)^5$

= 0.226

$\alpha$ is the .05 bar and $m$ is the 5 candidates tried at that step. .95 multiplied by itself five times is .774, so 1 − .774 = .226: better than a 1-in-5 chance that some noise predictor gets in at that step alone (treating the five tests as independent).
9. Check the Equation on Students It Never Saw

The honest test is to take the chosen equation, unchanged, and see how well it predicts the 120 held-out students. That's a simple form of cross-validation.

$R^2_{\text{new}} = 1 - \dfrac{\sum (y_i - \hat{y}_i)^2}{\sum (y_i - \bar{y})^2}$ = $1 - \dfrac{22.72}{26.30}$

= .136

$b_{\text{caffeine}}$ = 0.068
$b_{\text{caffeine, holdout}}$ = 0.008
$p_{\text{caffeine, holdout}}$ = .817

The selection half's equation misses the held-out students by 22.72 squared GPA points in all, against their own total of 26.30. 1 − 22.72 ÷ 26.30 = .136. Refit in the holdout half, caffeine's coefficient falls from 0.068 to 0.008 GPA points per drink, $p$ = .817.

Run It in Software

The code below reads the running example's data from data.csv; save it in the same folder you run the code from.

SPSS (version 30, as documented by IBM)

Data → Select Cases → If condition is satisfied, and enter half = "selection".
Analyze → Regression → Linear.
Move gpa into Dependent and all eight candidates into Independent(s).
Set Method to Stepwise. (Forward and Backward are the add-only and drop-only versions; Enter fits all predictors at once.)
Under Statistics, check R squared change. Under Options, leave Use probability of F at Entry .05 and Removal .10 (the defaults). Click OK.

Or paste this syntax:

TEMPORARY.
SELECT IF (half = "selection").
REGRESSION
  /MISSING LISTWISE
  /STATISTICS COEFF OUTS R ANOVA CHANGE
  /CRITERIA=PIN(.05) POUT(.10)
  /NOORIGIN
  /DEPENDENT gpa
  /METHOD=STEPWISE attendance study_hours sleep_hours work_hours commute_min
    social_media_hours exercise_hours caffeine_drinks.

Reading the Output

SPSS prints one row per step. Variables Entered/Removed lists attendance, study_hours, sleep_hours and caffeine_drinks, in that order, with nothing removed. Read the last row of the Model Summary for R² and adjusted R², then the final model's Coefficients. Beta is the standardized coefficient (in standard deviations of GPA per standard deviation of the predictor). In Excluded Variables, Beta In and Partial Correlation show what each left-out candidate would add if it entered next, and Tolerance (1 ÷ VIF) shows how little it overlaps with the predictors already in. The F change column at each step is the F to enter from the Napkin Math.

Model Summary

Model	R	R Square	Adjusted R Square	Std. Error of the Estimate	R Square Change	F Change	df1	df2	Sig. F Change
1	.408	.166	.159	.45001	.166	23.506	1	118	<.001
2	.502	.252	.239	.42799	.086	13.452	1	117	<.001
3	.564	.318	.300	.41049	.066	11.190	1	116	.001
4	.591	.350	.327	.40249	.032	5.654	1	115	.019

Coefficients (Model 4)

	B	Std. Error	Beta	t	Sig.
(Constant)	.025	.452		.055	.956
attendance	.021	.005	.356	4.504	<.001
study_hours	.026	.008	.260	3.293	.001
sleep_hours	.105	.035	.226	2.959	.004
caffeine_drinks	.068	.028	.186	2.378	.019

Excluded Variables (Model 4)

	Beta In	t	Sig.	Partial Correlation	Tolerance
work_hours	−.101	−1.309	.193	−.122	.945
commute_min	−.115	−1.523	.131	−.141	.981
social_media_hours	.004	.052	.959	.005	.791
exercise_hours	.084	1.114	.267	.104	.982

The ANOVA table's last row gives F(4, 115) = 15.472, Sig. <.001. Every Sig. value in these tables treats the model as chosen in advance, so none of them accounts for the search. The Excluded Variables table is the list of predictors that lost, not predictors shown to have no effect.

SPSS won't check the model on the holdout half for you. Use Transform → Compute Variable to build each held-out student's predicted GPA from the Model 4 B column, then compare the predictions with actual GPA as in the last Napkin Math step.

Output laid out from this page's computed results, not captured from a live run; labels and layout may differ slightly in your version.

R

students <- read.csv("data.csv")
sel  <- subset(students, half == "selection")
hold <- subset(students, half == "holdout")

empty  <- lm(gpa ~ 1, data = sel)
chosen <- step(empty, direction = "both", trace = 0,
               scope = ~ attendance + study_hours + sleep_hours + work_hours +
                 commute_min + social_media_hours + exercise_hours + caffeine_drinks)
formula(chosen)
summary(chosen)$r.squared

# Check the chosen equation on the held-out half
pred <- predict(chosen, newdata = hold)
1 - sum((hold$gpa - pred)^2) / sum((hold$gpa - mean(hold$gpa))^2)

gpa ~ attendance + study_hours + sleep_hours + caffeine_drinks + 
    commute_min
[1] 0.3628304
[1] 0.1144094

R's step() doesn't use p-values. It chooses by AIC, which rewards fit and charges 2 points per coefficient, and at each step makes whichever single addition or removal lowers AIC most. That bar is looser than SPSS's p < .05: for adding one predictor it's about the same as entering at p < .16, so here it goes one step further and adds commute time too (p = .13), another predictor with no real effect. R² rises to .36 in the selection half, and the holdout R² falls to .11. The AIC at each step:

Step	AIC
Start (no predictors)	−169.86
+ attendance	−189.66
+ study_hours	−200.71
+ sleep_hours	−209.77
+ caffeine_drinks	−213.53
+ commute_min	−213.94

Two programs, two defaults, two different "best" models from the same data: this is one of the drawbacks Whittingham et al. (2006) describe. For SPSS-style p-value selection in R, the olsrr package has ols_step_both_p().

Output laid out from this page's computed results, not captured from a live run; labels and layout may differ slightly in your version.

Python

import pandas as pd
import statsmodels.formula.api as smf

students = pd.read_csv("data.csv")
sel = students[students["half"] == "selection"]
hold = students[students["half"] == "holdout"]
candidates = ["attendance", "study_hours", "sleep_hours", "work_hours",
              "commute_min", "social_media_hours", "exercise_hours", "caffeine_drinks"]

def stepwise(data, candidates, p_enter=0.05, p_remove=0.10):
    chosen = []
    while True:
        remaining = [c for c in candidates if c not in chosen]
        p = {c: smf.ols(f"gpa ~ {' + '.join(chosen + [c])}", data).fit().pvalues[c]
             for c in remaining}
        if not p or min(p.values()) >= p_enter:
            return chosen
        best = min(p, key=p.get)
        chosen.append(best)
        print(f"enter {best:<16} p = {p[best]:.4f}")
        fit = smf.ols(f"gpa ~ {' + '.join(chosen)}", data).fit()
        worst = fit.pvalues[chosen].idxmax()
        if fit.pvalues[worst] > p_remove:
            chosen.remove(worst)
            print(f"remove {worst}")

chosen = stepwise(sel, candidates)
final = smf.ols(f"gpa ~ {' + '.join(chosen)}", sel).fit()
final.rsquared, final.rsquared_adj, final.fvalue

# Check the chosen equation on the held-out half
pred = final.predict(hold)
1 - ((hold["gpa"] - pred) ** 2).sum() / ((hold["gpa"] - hold["gpa"].mean()) ** 2).sum()

enter attendance       p = 0.0000
enter study_hours      p = 0.0004
enter sleep_hours      p = 0.0011
enter caffeine_drinks  p = 0.0191
(0.3498657808884186, 0.32725241674540706, 15.471637863158973)
0.13614895429490337

Neither statsmodels nor scikit-learn has a p-value stepwise function, so this short loop does what SPSS does: the p for a new predictor's coefficient in the bigger model is the same as its p to enter. For prediction, scikit-learn's LassoCV fits the lasso with its penalty chosen by cross-validation.

Output laid out from this page's computed results, not captured from a live run; labels and layout may differ slightly in your version.

Impact

Effect Size

R² is the share of the variation in the outcome that the model predicts ("explained variance", where "explain" means predict statistically). After a stepwise search, it has to be read alongside the out-of-sample R²: the same equation's performance on people who played no part in choosing it. The gap between them includes the optimism the search added, along with the ordinary overfitting of any fitted equation and chance differences between the two samples.

R^2 = 1 - \dfrac{18.63}{28.66} = .35 \qquad R^2_{\text{adj}} = .33 \qquad R^2_{\text{new}} = 1 - \dfrac{22.72}{26.30} = .14

On the 120 students it was chosen from, the model predicted 35% of the differences in GPA. On the 120 it never saw, it predicted 14%. Only part of that drop is the search. Equations fixed in advance fell too (Table 2): the three strongest real predictors went from .32 to .15, and all eight candidates from .38 to .12. Every fitted equation overfits its own sample, and this holdout half is simply harder to predict. Stepwise bought nothing over a sensible model chosen in advance (the three a researcher might pick from prior research): caffeine got in, and its equation predicted new students no better than the three-predictor one (.14 against .15). It did slightly better than entering all eight (.12), a gap one 120-person holdout can't distinguish from chance. A single split can't separate these sources; repeating the whole stepwise search inside K-fold cross-validation, and judging a pre-specified model the same way, can.

Adjusted R² doesn't close the gap, because it only charges for the four predictors kept. Derksen and Keselman (1992) found that adjusting for the total number of candidates, eight here, estimated the population value more faithfully; that gives .30, still well above the holdout's .14 in these data. With a single 120-person holdout the .14 is itself uncertain, so a K-fold cross-validation that repeats the whole stepwise search inside each fold gives a steadier estimate.

The coefficients shrink too. Predictors get picked partly because their sample coefficients came out large, so the selected coefficients tend to be too far from zero; Whittingham et al. (2006) list "bias in parameter estimation" first among stepwise's drawbacks. In the GPA study, sleep's coefficient fell from 0.105 to 0.078 GPA points per hour in the holdout half, and caffeine's from 0.068 to 0.008. There are no conventional benchmarks for R² after selection; whether predicting 14% of the variation in GPA is useful depends on what you'd do with the prediction.

After stepwise selection, R² overstates how well the model will predict new data; report the out-of-sample R² alongside it.

Statistical Significance

In a model chosen in advance, p answers one narrow question: if the predictors really had no relationship with the outcome, how often would a sample produce an F at least this large? After stepwise selection, it no longer answers even that. The search tried many candidates and kept the ones with the smallest p-values, so the reported p-values are the winners of a contest, and the F and t statistics "do not have the claimed distribution" (Harrell, 2026, Section 4.3). The p-values come out too small, and the confidence intervals too narrow.

Caffeine entered with p = .019. But five candidates were competing at that step, and if all five were pure noise, the chance that at least one would clear .05 is about .23. Across the whole search, 30 p-values were looked at. The final model's F(4, 115) = 15.47, p < .001 tests only the four winners, as if the other four had never been considered.

Selection also makes the entry and removal cutoffs misleading as evidence. Entering at p < .05 is not the same as being significant at .05; it's being the best of the remaining candidates and happening to fall under .05. Nor does staying out at p > .05 show that a predictor doesn't matter.

What to do instead depends on the goal. To test predictors, name them in advance and fit them together, or in a theory-driven order with hierarchical regression; then p means what it says. To explore, run the search, call it exploratory, and test what it finds in a holdout half or a new sample. To see how fragile the choice is, bootstrap the whole search and count how often each predictor is picked. To predict, use cross-validation to judge the model, and consider penalized regression such as the lasso, which shrinks every coefficient toward zero and sets some exactly to zero (Tibshirani, 1996), with its penalty chosen by cross-validation (Yarkoni & Westfall, 2017).

p is not the probability that a selected predictor has no effect, and after stepwise it isn't a trustworthy false-positive rate either.

Benchmarks p at a Glance

Very Strong

p < .001

Strong

.001 ≤ p < .01

Significant

.01 ≤ p < .05

Weak

.05 ≤ p < .10

Little

p ≥ .10

Strength of evidence against the null hypothesis.

Full benchmarks and caveats ↑

Power and Sample Size

Power is the chance that a study detects an effect that's really there. For stepwise, the question splits in two: how likely is each real predictor to be selected, and how many noise predictors will get in alongside it? Neither has a formula. Simulation studies show that the number of noise predictors that sneak in rises with the number of candidates, that correlations among the candidates affect how often real ones make it in, and that, in their simulations, sample size made little practical difference to how many real ones did (Derksen & Keselman, 1992).

In this sample of 120, stepwise found the three strongest real predictors but missed work hours, whose true effect is small, and it let caffeine in alongside them. Run on the holdout half alone, stepwise chose attendance, study hours and sleep, and nothing else: a different model from the same population.

Plan power for the model you intend to test, fitted as a multiple regression specified in advance, and keep the number of candidates small relative to the sample. A rough rule is at least 15 people per candidate examined, not per predictor kept (Harrell, 2026, Section 4.4); the GPA study's 120 students for eight candidates sit right at that limit. For an exploratory search, plan for a holdout sample or cross-validation instead. There's no calculator for stepwise, because its error rates depend on the candidates, their correlations and the search itself.

Other Info

Follow-Up and Post Hoc Tests

None needed. This test answers a single question, so there are no planned contrasts or post hoc tests to run.

Related Methods

General CaseMultiple Linear Regression: every model stepwise fits is an ordinary multiple regression; only the choice of predictors is automated. Fit the model you'd have chosen from theory, with all its predictors at once, and the p-values and R² mean what they say.
Related DesignHierarchical Regression: the same step-by-step build, with R² change and an F test at each step, but you choose the order in advance from theory instead of letting the data choose it. That's the version whose tests can be taken at face value.
AlternativeMultilevel Regression (HLM): An alternative if “Independent Observations” doesn't hold. If students are clustered in classes or schools, a multilevel model accounts for the clustering.

References

Efroymson, M. A. (1960). Multiple regression analysis. In A. Ralston & H. S. Wilf (Eds.), Mathematical methods for digital computers. Wiley.
Whittingham, M. J., Stephens, P. A., Bradbury, R. B., & Freckleton, R. P. (2006). Why do we still use stepwise modelling in ecology and behaviour? Journal of Animal Ecology, 75(5), 1182–1189. https://doi.org/10.1111/j.1365-2656.2006.01141.x
“The principal drawbacks of stepwise multiple regression include bias in parameter estimation, inconsistencies among model selection algorithms, an inherent (but often overlooked) problem of multiple hypothesis testing, and an inappropriate focus or reliance on a single best model.” (Abstract)
Thompson, B. (1995). Stepwise regression and stepwise discriminant analysis need not apply here: A guidelines editorial. Educational and Psychological Measurement, 55(4), 525–534. https://doi.org/10.1177/0013164495055004001
“Third, stepwise methods tend to capitalize on sampling error and thus tend to yield results that are not replicable.” (Abstract)
Derksen, S., & Keselman, H. J. (1992). Backward, forward and stepwise automated subset selection algorithms: Frequency of obtaining authentic and noise variables. British Journal of Mathematical and Statistical Psychology, 45(2), 265–282. https://doi.org/10.1111/j.2044-8317.1992.tb00992.x
“the number of candidate predictor variables affected the number of noise variables that gained entry to the model” (Abstract)
Smith, G. (2018). Step away from stepwise. Journal of Big Data, 5, Article 32. https://doi.org/10.1186/s40537-018-0143-6
“some real explanatory variables that have causal effects on the dependent variable may happen to not be statistically significant, while nuisance variables may be coincidentally significant.” (Abstract)
Harrell, F. E., Jr. (2026). Regression modeling strategies [Course notes]. Department of Biostatistics, Vanderbilt University. https://hbiostat.org/rmsc/
“F and χ² statistics do not have the claimed distribution” (Section 4.3)

“Model specification is preferred to model selection.” (Section 4.3.1)
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1), 267–288. https://doi.org/10.1111/j.2517-6161.1996.tb02080.x
“Because of the nature of this constraint it tends to produce some coefficients that are exactly 0 and hence gives interpretable models.” (Summary)
Yarkoni, T., & Westfall, J. (2017). Choosing prediction over explanation in psychology: Lessons from machine learning. Perspectives on Psychological Science, 12(6), 1100–1122. https://doi.org/10.1177/1745691617693393
“the overall cost function that the estimation seeks to minimize now includes an additional penalty term that is proportional to the sum of the absolute values of the coefficients.” (Regularization)

“The tendency for statistical models to mistakenly fit sample-specific noise as if it were signal is commonly referred to as overfitting.” (To fit is to overfit)

“The latter approach is not cross-validation, and does little to mitigate overfitting.” (Note 4)
Babyak, M. A. (2004). What you see may not be what you get: A brief, nontechnical introduction to overfitting in regression-type models. Psychosomatic Medicine, 66(3), 411–421. https://doi.org/10.1097/01.psy.0000127692.23278.a9
“Overfitted models will fail to replicate in future samples, thus creating considerable uncertainty about the scientific merit of the finding.” (Abstract)
Muff, S., Nilsen, E. B., O'Hara, R. B., & Nater, C. R. (2022). Rewriting results sections in the language of evidence. Trends in Ecology & Evolution, 37(3), 203–210. https://doi.org/10.1016/j.tree.2021.10.009
“we suggest language of evidence that allows for a more nuanced approach to communicate scientific findings as a simple and intuitive alternative to statistical significance testing” (Abstract)
Williams, M. N., Gómez Grajales, C. A., & Kurkiewicz, D. (2013). Assumptions of multiple regression: Correcting two misconceptions. Practical Assessment, Research & Evaluation, 18(11), 1–14. https://doi.org/10.7275/55hn-wk47
“ordinary least squares estimates will be unbiased and consistent as long as the errors are independent, but will not be efficient” (p. 9)
Hayes, A. F., & Cai, L. (2007). Using heteroskedasticity-consistent standard error estimators in OLS regression: An introduction and software implementation. Behavior Research Methods, 39(4), 709–722. https://doi.org/10.3758/BF03192961
O'Brien, R. M. (2007). A caution regarding rules of thumb for variance inflation factors. Quality & Quantity, 41(5), 673–690. https://doi.org/10.1007/s11135-006-9018-6