Linear Regression · General Linear Model
Simple Linear Regression
Simple linear regression predicts one variable from another with a straight line.
Simple linear regression takes one continuous predictor and one continuous outcome, finds the straight line with the smallest total squared prediction error, and tests whether its slope differs from zero.
Do people who walk more in a day feel better by evening, and by how much? Eighty adults wore a step counter for two weeks and rated their mood each evening from 0 to 10; each person's two-week averages are one row.
- Tests
- Population slope
- Observations
- Independent
- Parametric
- Yes
- Headline Statistic
- b
- Effect Size
- Standardized Slope (\beta)
- Groups
- 1
The Basics
When to Use It
Use it when you've measured two continuous variables on the same people, one of them is the outcome you want to predict, and you want to know how much higher or lower it is, in its own units, per unit of the other. For example:
- Do people who walk more in a day feel better by evening, and by how much?
- How much higher are anxiety scores among people who spend one more hour a night on their phone?
- How many milliseconds slower is reaction time for each additional year of age?
Vocabulary You Need
Definition Regression Coefficient (b)
The slope of a regression line: how much higher (or lower) the predicted outcome is, in the outcome's own units, for people one unit higher on a predictor. A positive coefficient means people higher on the predictor tend to score higher on the outcome; a negative one, lower. With several predictors, each coefficient is the difference in predicted outcome between people one unit apart on that predictor and equal on the others. Also called the unstandardized coefficient, B, or simply the slope. Glossary →
Definition Intercept (b_0)
The value a regression model predicts for the outcome when every predictor equals zero, which is where a regression line crosses the vertical axis. It anchors the line, but it only means something on its own when zero is a real, observed value of the predictors; otherwise it's a prediction for people who don't exist in the data. Centering a predictor (subtracting its mean) makes the intercept the prediction for someone at the average. Also called the constant. Glossary →
Definition Predicted Value (\hat{y})
The outcome value a regression model gives for a particular set of predictor values: the intercept plus each coefficient times its predictor. It's the model's best estimate of the average outcome among people with those predictor values, not a promise about any one person, who will usually land some distance away (the residual). Written ŷ (“y-hat”) and also called the fitted value. Glossary →
Definition Residual
A score minus the value the model predicts for it. In an ANOVA the prediction is the person's group mean, so the residual is how far each person sits from their own group's average; in a regression it's the point on the fitted line (or plane) for that person's predictor values. Residuals are what the model leaves unexplained, and several assumptions, such as normality and equal spread, are really about them. Glossary →
Assumptions
1. Linear Relationship The relationship between your variables follows a straight line, not a curve.
- What It Means
- As one variable goes up, the other tends to change by about the same amount across its whole range. A curved relationship, one that levels off or a U shape, breaks this.
- Applies To
- Relationship
- How to Check
- Look at a scatterplot of each pair of variables the analysis relates: a roughly oval cloud tilted one way is fine; a clear bend, a ceiling or a U is not. In a regression with several predictors, plot the residuals against the predicted values instead: a flat, even band is fine; a curve means the model is missing a bend. A component-plus-residual plot for each predictor (R: car::crPlots) shows whether one of them bends once the others are held constant.
- How Much It Matters
- A lot. The slope describes a straight line. If the real relationship curves, the slope averages over the bend, predictions miss in a systematic way at the ends, and the residuals show a pattern. The step-count data follow a roughly straight trend with no bend.
If It's Violated
- Transformation If the outcome rises quickly and then levels off, log-transform the predictor. The slope then describes the transformed scale.
- Alternative Test Multiple Linear Regression. Add the predictor's square as a second predictor (polynomial regression), which lets the line bend.
- Proceed with Caveat If a range was chosen in advance, or theory says the relationship is straight only over part of it, you can fit the line there. Report it as a restriction chosen for that reason and show the full-range fit too. Don't pick the range by eye after seeing the curve.
2. Independent Observations The people (or other units) in your study don't influence each other.
- What It Means
- Each person's data comes from a separate, unconnected source, so knowing one person's scores tells you nothing about anyone else's. In designs that measure people more than once, their own scores are meant to be related; it's different people who must be independent. Methods built for clustered data, such as multilevel models and GEE, also allow members of the same group to be related, as long as the analysis names the grouping; then it's the groups that must be independent.
- Applies To
- Observations
- How to Check
- By design, not by a test. Did each person contribute their own data, with no clusters such as roommates, couples, therapy groups or classrooms whose members influence each other? If there are clusters, does the analysis model every one of them?
- How Much It Matters
- A lot. People who influence each other, such as friends who walk together or members of the same household, make the slope's standard error too small, so p comes out too small.
If It's Violated
- Alternative Test Multilevel Regression (HLM). A multilevel model estimates the slope while accounting for clusters such as households, classrooms or repeated days.
- Design Change Recruit one person per household or friendship group, and average each person's repeated measurements into one score.
3. Equal Spread of Residuals Your scores scatter about equally around the predictions, whatever the prediction.
- What It Means
- The residuals have about the same spread at every predicted value, so the model misses by about as much for people predicted to score low as for people predicted to score high. The opposite, heteroscedasticity, often looks like a funnel. If the model's form is right, unequal spread doesn't bias the coefficients, though it makes them less precise than they could be; it does make their standard errors, and so the p-values and confidence intervals, unreliable.
- Applies To
- Residuals
- How to Check
- Plot the residuals against the predicted values (in the SPSS REGRESSION procedure, *ZRESID against *ZPRED; for MIXED, save them with /SAVE=PRED RESID and plot those). A band of even width is fine; a funnel or fan that widens or narrows is not. Don't use a test such as Breusch–Pagan as a gatekeeper: with small samples it misses real problems, and with large ones it flags trivial ones.
- How Much It Matters
- Moderately, and only for the standard error, p and the interval; the slope itself stays unbiased as long as the line is the right shape, though it's less precise than it could be (Hayes & Cai, 2007). A funnel matters most in small samples and when the widest spread sits at one end of the predictor. In the step-count data, residuals spread about as widely below the median prediction (SD 1.63) as above it (SD 1.49).
If It's Violated
- Robust Estimation Use heteroscedasticity-consistent (HC3) standard errors, which don't assume an even spread. Hayes and Cai (2007) suggest them at least as a double-check.
- Transformation Log-transform a right-skewed outcome, such as reaction time, which often evens out the spread. The slope is then on the log scale.
- Robust Estimation Bootstrap a confidence interval for the slope by resampling people.
4. Normality The scores the test relies on are roughly bell-shaped, or your sample is big enough that it doesn't matter.
- What It Means
- The test's model assumes the scores it relies on are roughly normal: the scores within each group (or the scores themselves, for a one-sample test), the difference scores in a paired design, or the residuals (the errors left around a model's predictions) in a regression. For p-values and confidence intervals, what really matters is that the estimates (such as a mean, a difference between means or a model's coefficients) are roughly normal across repeated samples. A large enough sample usually provides that; how large depends on how skewed the scores are and how extreme their outliers.
- Applies To
- Residuals
- How to Check
- Look at a histogram or Q–Q plot for strong skew or extreme outliers. Don't rely on a normality test: with small samples it misses real problems, and with large ones it flags trivial ones.
- How Much It Matters
- Little with 80 people. Only the residuals need to be roughly normal, not the predictor or the outcome (Williams et al., 2013), and as samples grow the slope's t-test holds up even when the residuals aren't. The step-count residuals are nearly symmetric (skewness −0.07).
If It's Violated
- Robust Estimation Bootstrap a confidence interval for the slope, which doesn't assume a shape for the residuals.
- Transformation Log-transform a strongly right-skewed outcome. The slope then describes the log scale.
5. No Extreme Outliers No single case is far enough from the rest to drive the result.
- What It Means
- A few points far from the rest can pull a statistic around on their own, so it describes those few points more than everyone else. In a test, that can create an effect that isn't there or hide one that is.
- Applies To
- Observations
- How to Check
- Plot the data and look for points far from the rest. With two or more variables, flag cases with large squared Mahalanobis distances across them: points can be ordinary on each variable but unusual in combination. Check each against the raw records, and compare the result with and without it.
- How Much It Matters
- A lot, especially in small samples. A person far from the others on the predictor who also sits off the line can tilt the slope alone, and because the line bends toward them, their residual can look small. So check influence, not just residuals: Cook's distance combines how far a person sits from the line with how unusual their predictor value is (Cohen et al., 2003). In SPSS, check Cook's under Save; in R, use
cooks.distance(model). Values near 1 are a common flag. In the step-count data, no residual is more than 2.45 standard errors of the estimate from the line, and the largest Cook's distance is 0.10.
If It's Violated
- Correction Check each extreme point against the raw data, and fix or remove data-entry errors.
- Proceed with Caveat If the outlier is a real person, report the slope with and without them and say which you trust.
- Robust Estimation Fit a robust regression, which gives less weight to points far from the line.
Limitations
This Is Not Your Test If…
- You have two or more predictors Multiple Linear Regression →
- You only want how strongly two scores go together Pearson Correlation →
- Your outcome is yes/no, not a score Logistic Regression →
- Your predictor is a set of groups, not a score ANOVA Family →
- You measured people repeatedly, or in clusters Multilevel Regression (HLM) →
- Your scores are ranks, or you only need a monotonic association, not a slope Spearman Correlation →
What It Doesn't Tell You
- Why. The slope says mood is higher among people who walk more, not that walking lifts mood; people in a good mood may simply walk more, or health and free time may drive both.
- Whether it matters. Significance doesn't make 0.20 points per 1,000 steps a useful prediction: steps predict only 6% of the differences in mood, individual predictions miss by about 1.56 points, and one person walking 10,000 steps could plausibly rate anywhere from 3.4 to 9.8.
- Who else, and where else. The line describes adults like these 80, between about 2,500 and 12,000 steps a day. It says nothing about what happens at 20,000.
- That there's no relationship, if p had been above .05. A non-significant slope would have meant the data were inconclusive about a straight-line trend, not that steps and mood are unrelated. A curved relationship can even have a slope near zero.
Common Mistakes
-
Don'tPredict outcomes for predictor values far outside the range you measured.
InsteadKeep predictions inside your data's range. Nothing shows the line continues beyond it.
-
Don'tInterpret the intercept when a predictor value of zero never occurs.
InsteadCenter the predictor (subtract its mean) so the intercept is the prediction for an average person.
-
Don'tSwap the predictor and outcome and expect the same line.
InsteadDecide which variable is the outcome first. Swapping them changes the slope; only r, t and p stay the same.
-
Don'tCheck the predictor and outcome for normality.
InsteadCheck the residuals. Regression assumes nothing about the predictor's distribution.
-
Don'tDescribe a regression coefficient as the effect of raising the predictor.
InsteadSay the prediction is that much higher per unit of the predictor, on the model's scale (the outcome itself, or its log-odds or log rate in a logistic, ordinal or Poisson model). Causal language needs a design that supports it, such as an experiment.
-
Don'tRead a p-value as the chance that the null hypothesis is true.
InsteadRead it as: if the null were true, how often data at least this extreme would turn up.
Result Decoder
Significance and Effect Size Guidelines
Benchmarks Significance (p Value)
-
p < .001
Very strong evidence against the null hypothesis.
-
.001 ≤ p < .01
Strong evidence against the null hypothesis.
-
.01 ≤ p < .05
Statistically significant at α = .05. Moderate evidence against the null hypothesis.
-
.05 ≤ p < .10
Not significant. Weak evidence; sometimes called a “trend.” Treat it as inconclusive.
-
p ≥ .10
Not significant. Little evidence against the null, which is not the same as evidence of no effect.
These evidence labels are one common convention for describing p in graded language rather than as significant or not; the boundaries are rough guides, not rules, and other authors draw them differently. Muff et al. (2022) argue for this kind of language of evidence over a single cutoff. α is chosen before the analysis, and p never says how big or important an effect is.
Benchmarks Effect Size (Standardized Slope (\beta))
-
|r| < .10
Below Small
Smaller than the conventional small benchmark. Whether an effect this size matters depends on the setting.
-
.10 ≤ |r| < .30
Small
A weak straight-line trend, often hard to see in a scatterplot without computing it.
-
.30 ≤ |r| < .50
Medium
Cohen described a medium effect as likely visible to the naked eye of a careful observer; whether it shows in a given scatterplot depends on the data and the sample size.
-
|r| ≥ .50
Large
A strong straight-line trend, usually clear in a scatterplot unless outliers or a small sample muddy it.
With one predictor the standardized slope β equals Pearson's r, so these are r's benchmarks, applied to its size. The cutoffs .10, .30 and .50 are Cohen's (1988, 1992); the band below .10 and the descriptions are this site's glosses. They are conventions, not measures of practical importance.
How It's Reported
Equation Form
b = 0.20, 95% CI [0.03, 0.37], β = .25, t(78) = 2.28, p = .025
Table 1. Regression of Evening Mood on Daily Steps
| b | SE | 95% CI | \beta | t | p | |
|---|---|---|---|---|---|---|
| Intercept | 4.60 | 0.639 | [3.33, 5.87] | 7.20 | < .001 | |
| Daily steps (thousands) | 0.20 | 0.088 | [0.03, 0.37] | .25 | 2.28 | .025 |
Note. N = 80. Steps are in thousands per day; mood is on a 0–10 scale. R² = .06, F(1, 78) = 5.20, p = .025; standard error of the estimate = 1.56 points.
Full Text (APA Style)
Participants averaged 7,000 steps a day (M = 7.00 thousand, SD = 2.00) and an evening mood of 6.00 out of 10 (SD = 1.60). People who walked more tended to report better evening moods: people 1,000 daily steps apart differed by 0.20 points in mood on average, b = 0.20, 95% CI [0.03, 0.37], β = .25, and steps predicted 6% of the variation in mood, R2 = .06. The slope was statistically significant, t(78) = 2.28, p = .025.
- Give the slope in the outcome's units, with the predictor's unit spelled out (“per 1,000 steps”).
- Report b with its interval before the test, and add β or R² for readers comparing studies on different scales (β still depends on how spread out each sample is).
- Write “went with” or “predicted,” not “raised” or “improved.” The design can't support a causal claim.
- F(1, 78) = 5.20 from the ANOVA table is the same test as t(78) = 2.28 (5.20 = 2.28²). Report one.
Reading Each Piece
The equation form above, piece by piece, in this order: how big, how certain, then how surprising.
| b = 0.20 | People who walked 1,000 more steps a day than others rated their evening mood 0.20 points higher on the 0–10 scale, on average. For two people 5,000 steps a day apart, the line predicts a 1-point difference in mood. |
| β = .25 | People one standard deviation (2,000 steps) apart in daily steps differed by a quarter of a standard deviation in mood. That's a small effect by the benchmarks above, and R² = .06 means a straight line in steps predicts only 6% of the variation in mood; the other 94% it can't predict. Benchmarks ↑ |
| 95% CI [0.03, 0.37] | The true slope is plausibly anywhere from a trivial 0.03 to a more noticeable 0.37 points per 1,000 steps. The whole range is above zero, but 80 people can't say whether the link is tiny or modest. |
| t(78) = 2.28 | The slope is 2.28 standard errors above zero. The 78 is the degrees of freedom, n − 2. It's the same t a Pearson correlation of these two variables would give. |
| p = .025 | If the true slope were zero, a sample of 80 would give a slope at least this far from zero, in either direction, about 2.5% of the time. That's below .05, so the slope is statistically significant. Benchmarks ↑ |
Calculations
The Running Example
The step-count study: 80 adults, each with their average daily steps and average evening mood over two weeks, predicting mood from steps.
- Null Hypothesis:
- \beta_1 = 0
- Alternative (Two-Sided):
- \beta_1 \neq 0
The model behind the test: among all adults like these, mood = \beta_0 + \beta_1 \times steps + error, where the errors are independent, about equally spread at every step count and roughly normal. \beta_1 is the population slope: how much higher evening mood is, on average, per 1,000 more daily steps, which the sample's b = 0.20 estimates. (Statistics reuses the letter: the standardized slope β further down is a different quantity.)
What Your Data Should Look Like
One row per person, with a numeric column for the predictor and one for the outcome. Both scores on a row must come from the same person, and you decide which column is the outcome before analyzing.
| participant | thousand_steps | evening_mood |
|---|---|---|
| 1 | 4.2 | 6.0 |
| 2 | 8.1 | 7.2 |
| 3 | 4.6 | 7.4 |
| 4 | 8.0 | 7.4 |
| 5 | 8.1 | 8.1 |
| 6 | 11.1 | 5.8 |
| … 74 more rows | ||
| n | M | SD | |
|---|---|---|---|
| Thousand Steps | 80 | 7.00 | 2.00 |
| Evening Mood | 80 | 6.00 | 1.60 |
Pearson correlation between the two: r = .25
Simulated data.
Napkin Math
The whole test, one idea at a time. Under each heading, a line says why the step matters, and the green note reads the equation in plain English with the step-count study's numbers. Collapse all to read just the headings: they tell the whole story on their own.
-
1. Find How Much Higher Mood Is per 1,000 More Steps
Of all straight lines, regression picks the one that makes the squared misses (the residuals) as small as possible in total, which is called least squares (Cohen et al., 2003). That line's slope adds up each person's two deviations from the averages multiplied together, and divides by the sum of the squared step deviations. It's the same as the correlation rescaled into mood points per 1,000 steps.
b_1 = \dfrac{\sum (x_i - \bar{x})(y_i - \bar{y})}{\sum (x_i - \bar{x})^2} = r\,\dfrac{s_y}{s_x} =\dfrac{63.29}{316.18}= 0.20 points per 1,000 steps
63.29 adds up each person's (steps − 7.00) × (mood − 6.00), and 316.18 adds up (steps − 7.00)². 63.29 ÷ 316.18 = 0.20 mood points per 1,000 steps. Equivalently, from the .25 correlation and the two standard deviations, .25 × 1.60 ÷ 2.00 = 0.20.
-
2. Anchor the Line at the Two Averages
Because of how least squares works, the line always passes through the point where both variables are at their averages. The intercept is wherever that line crosses zero steps.
b_0 = \bar{y} - b_1\bar{x} =6.00 - 0.20 \times 7.00= 4.60 points
\bar{y} is the 6.00 average mood and \bar{x} the 7.00 thousand average steps. 0.20 × 7.00 = 1.40, and 6.00 − 1.40 = 4.60 points. No one here walked fewer than 2,500 steps, so read 4.60 as the line's anchor, not a real prediction for someone who never moves.
-
3. Predict Mood for Someone Who Walks 10,000 Steps
With a slope and an intercept, the line gives a predicted mood for any step count in the range we measured. The gap between a person's actual mood and their prediction is their residual.
\hat{y} = b_0 + b_1 x =4.60 + 0.20 \times 10= 6.60 points
b_0 is our 4.60 intercept, b_1 the 0.20 slope and x is 10 thousand steps. 4.60 + 2.00 = 6.60 points. That's the average for people walking 10,000 steps; we're fairly sure it lies between 5.97 and 7.23. One new person walking 10,000 steps could plausibly rate anywhere from 3.43 to 9.77 (a 95% prediction interval), almost the whole scale. Participant 1 walked 4,200 steps, so their prediction is 5.44; they actually rated 6.0, a residual of +0.56.
-
4. Measure How Far People Typically Fall from the Line
The line is a prediction for the average person at each step count. The standard error of the estimate says how far real people typically land from it.
s_e = \sqrt{\dfrac{\sum (y_i - \hat{y}_i)^2}{n - 2}} =\sqrt{\dfrac{190.01}{80 - 2}}= 1.56 points
y_i - \hat{y}_i is each person's residual. Squared and added up over all 80 people they come to 190.01; divided by n - 2 = 78 that's 2.44, and its square root is 1.56 mood points. Predictions from steps miss by about a point and a half, barely less than mood's own 1.60-point standard deviation.
-
5. Figure Out How Precisely We've Pinned Down the Slope
A different sample of 80 would give a somewhat different slope. The slope's standard error measures that wobble; it shrinks with less scatter around the line, more people, and a wider spread of step counts.
SE_{b_1} = \dfrac{s_e}{s_x\sqrt{n - 1}} =\dfrac{1.56}{2.00\sqrt{79}}= 0.088 points per 1,000 steps
s_e is the 1.56-point scatter, s_x the 2.00-thousand-step spread and n - 1 is 79. √79 is 8.89, so 1.56 ÷ (2.00 × 8.89) = 0.088 points per 1,000 steps.
-
6. Compare the Slope with That Uncertainty
A slope only means something relative to its wobble. t counts how many standard errors the slope sits from zero, the slope we'd expect if the true slope were zero.
t = \dfrac{b_1}{SE_{b_1}} =\dfrac{0.200}{0.0878}= 2.28
df = 78
F = t^2 = 5.20
Our 0.20 slope divided by its 0.088 standard error is 2.28. The line estimated two things, a slope and an intercept, so df = n - 2 = 78. Squared, 2.28² = 5.20: the F that SPSS prints in its ANOVA table.
-
7. Put the Slope on a Common Scale and See How Much It Predicts
Points per 1,000 steps can't be compared directly with a study that measured sleep in hours. Rescaling the slope into standard deviations gives β, which here equals the correlation; squaring it gives the share of the differences in mood that steps predict.
\beta = b_1\,\dfrac{s_x}{s_y} = r =0.20 \times \dfrac{2.00}{1.60}= .25
R^2 = .06
s_x is the 2.00-thousand-step spread and s_y the 1.60-point spread. 0.20 × 2.00 ÷ 1.60 = .25: people one standard deviation apart in steps differ by a quarter of a standard deviation in mood. Because .25 is below 1, predictions sit closer to the average than the scores they're predicted from; Galton (1886) saw the same pull in parents' and children's heights and named it regression. Squared, .25 × .25 = .06, so steps predict 6% of the variation in mood.
-
8. Give a Range of Plausible Values for the True Slope
Instead of one best guess, give the range of population slopes that fit the data.
b_1 \pm t^{*}_{n-2} \times SE_{b_1} =0.20 \pm 1.991 \times 0.0878= [0.03, 0.37]
t^{*}_{78} = 1.991 is the cutoff that holds the middle 95% of a t distribution with 78 df. 1.991 × 0.088 = 0.17, so the range is 0.20 ± 0.17, from 0.03 to 0.37 points per 1,000 steps.
-
9. Ask How Surprising the Slope Would Be If the True Slope Were Zero
Finally, ask how often a slope at least this far from zero would turn up if the true slope were zero.
p = 2 \times P(T_{df} \le -|t|) =2 \times P(T_{78} \le -2.28)= .025
P(T_{78} \le -2.28) is the chance that a t with 78 df lands at or below −2.28. Doubling it counts both directions: .025, about 1 in 40.
Run It in Software
The code below reads the running example's data from data.csv; save it in the same folder you run the code from.
SPSS (version 30, as documented by IBM)
- Analyze → Regression → Linear.
- Move
evening_moodinto Dependent andthousand_stepsinto Independent(s). - Click Statistics, check Estimates, Confidence intervals (95%) and Model fit, and click Continue.
- Click Plots, put *ZRESID on Y and *ZPRED on X to check the spread of the residuals, and click Continue, then OK.
Or paste this syntax:
REGRESSION /MISSING LISTWISE /STATISTICS COEFF OUTS CI(95) R ANOVA /CRITERIA=PIN(.05) POUT(.10) /NOORIGIN /DEPENDENT evening_mood /METHOD=ENTER thousand_steps /SCATTERPLOT=(*ZRESID ,*ZPRED).
Reading the Output
SPSS prints three tables. Read the Coefficients table first: the thousand_steps row's B (the slope), Standardized Coefficients Beta, and the 95.0% Confidence Interval for B, then t and Sig. After that, R Square from the Model Summary.
Model Summary
| Model | R | R Square | Adjusted R Square | Std. Error of the Estimate |
|---|---|---|---|---|
| 1 | .250 | .063 | .050 | 1.561 |
ANOVA
| Model | Sum of Squares | df | Mean Square | F | Sig. | |
|---|---|---|---|---|---|---|
| 1 | Regression | 12.669 | 1 | 12.669 | 5.201 | .025 |
| Residual | 190.011 | 78 | 2.436 | |||
| Total | 202.680 | 79 |
Coefficients
| Model | B | Std. Error | Beta | t | Sig. | 95% CI Lower | Upper | |
|---|---|---|---|---|---|---|---|---|
| 1 | (Constant) | 4.599 | .639 | 7.203 | <.001 | 3.328 | 5.870 | |
| thousand_steps | .200 | .088 | .250 | 2.280 | .025 | .025 | .375 |
The ANOVA table's F is the slope's t squared (2.280² = 5.20), with the same p: with one predictor they're one test. The (Constant) row tests whether the intercept is zero, which isn't the question here.
Output laid out from this page's computed results, not captured from a live run; labels and layout may differ slightly in your version.
R
steps <- read.csv("data.csv")
model <- lm(evening_mood ~ thousand_steps, data = steps)
summary(model)
round(confint(model), 3)
# Standardized slope (beta)
coef(lm(scale(evening_mood) ~ scale(thousand_steps), data = steps))[2]
# 95% prediction interval for one person at 10,000 steps
predict(model, data.frame(thousand_steps = 10), interval = "prediction")
Call:
lm(formula = evening_mood ~ thousand_steps, data = steps)
Residuals:
Min 1Q Median 3Q Max
-3.2006 -0.9606 0.1201 1.0097 3.8196
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 4.59930 0.63852 7.203 3.23e-10 ***
thousand_steps 0.20017 0.08778 2.280 0.0253 *
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 1.561 on 78 degrees of freedom
Multiple R-squared: 0.06251, Adjusted R-squared: 0.05049
F-statistic: 5.201 on 1 and 78 DF, p-value: 0.02531
2.5 % 97.5 %
(Intercept) 3.328 5.870
thousand_steps 0.025 0.375
scale(thousand_steps)
0.2500131
fit lwr upr
1 6.601014 3.430657 9.77137
Read the thousand_steps row: Estimate is the slope, then the interval from confint, then t value and Pr(>|t|). Residual standard error is the standard error of the estimate. Adjusted R-squared (.05) shrinks R² slightly to allow for chance fit in a sample of 80; with one predictor the two are close, and either can be reported. predict gives the range for one new person (lwr to upr), much wider than the uncertainty about the average mood at 10,000 steps.
Output laid out from this page's computed results, not captured from a live run; labels and layout may differ slightly in your version.
Python
import pandas as pd
import statsmodels.formula.api as smf
steps = pd.read_csv("data.csv")
model = smf.ols("evening_mood ~ thousand_steps", data=steps).fit()
model.params["thousand_steps"] # 0.2002
model.conf_int().loc["thousand_steps"] # 0.0254, 0.3749
model.tvalues["thousand_steps"] # 2.2805
model.pvalues["thousand_steps"] # 0.02531
model.rsquared # 0.0625
model.params["Intercept"] # 4.5993
model.summary() # the full table
Impact
Effect Size
Regression gives the effect in two forms. The slope b is in the variables' own units, which is what a reader can picture. The standardized slope β removes the units, so studies on different scales can be compared; with one predictor it equals Pearson's r, and its square, R², is the share of the outcome's variation the line predicts.
People 1,000 daily steps apart differed by 0.20 points in evening mood. On the common scale that's β = .25, a small effect by convention, and steps predict 6% of the differences in mood; the other 94% the line can't predict.
No units doesn't mean automatically comparable. β depends on how spread out the sample is: a study of only very active adults, with a narrower range of steps, would find a smaller β even with the same b. Noise in the step counts pulls both b and β toward zero, which averaging two weeks of counts reduces. When studies use the same units, compare b.
Benchmarks are conventions. Whether a 0.20-point difference in mood between people 1,000 steps (about ten minutes of walking) apart matters is a question about people's lives, not statistics, and this design can't say whether walking more would raise anyone's mood.
Benchmarks Standardized Slope (\beta) at a Glance
Statistical Significance
p answers one narrow question: if the true slope were zero, so that there were no straight-line trend, how often would a sample slope land at least this far from zero, in either direction? Below the conventional α = .05, the slope is called statistically significant.
If the true slope were zero, a sample of 80 adults would show a slope at least this far from zero about 2.5% of the time, roughly 1 in 40. That's below .05, so it's significant.
p is not the probability that the true slope is zero, and it says nothing about how steep the slope is or how well it predicts: with 10,000 people, a slope of 0.02 points per 1,000 steps would be significant too. That's what b, its interval and R² are for.
Benchmarks p at a Glance
Strength of evidence against the null hypothesis.
Power and Sample Size
Power is the chance that a study detects an effect that's really there. A real slope doesn't guarantee a significant result: with too few people, it can easily come out non-significant. Power depends on four things: how strong the true relationship is, how many people you have, α, and whether the test is one- or two-sided.
With one predictor, the test of the slope is exactly the test of the correlation, so you plan it the same way: choose the standardized slope β (which equals r) you expect, and find the sample size for that correlation.
Suppose, before collecting any data, we expected steps and mood to correlate about .25, the β we found. With 124 adults, the study would come out significant about 80% of the time. Our 80 adults gave only about 61% power: a real slope this size would be missed nearly 4 times in 10. These figures use the Fisher z approximation, as the calculator below does; R's pwr.r.test, which adds a small correction to the same approximation, gives 123 adults and 62%.
Researchers usually aim for at least 80% power. Plan it before collecting data, using the smallest slope that would matter, converted to β. Small effects need big samples: 80% power for β = .10 takes about 783 people (782 in pwr.r.test). Try it:
Other Info
Follow-Up and Post Hoc Tests
None needed. This test answers a single question, so there are no planned contrasts or post hoc tests to run.
References
- Galton, F. (1886). Regression towards mediocrity in hereditary stature. The Journal of the Anthropological Institute of Great Britain and Ireland, 15, 246–263. https://doi.org/10.2307/2841583
- Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Lawrence Erlbaum Associates. https://doi.org/10.4324/9780203774441
-
Williams, M. N., Gómez Grajales, C. A., & Kurkiewicz, D. (2013). Assumptions of multiple regression: Correcting two misconceptions. Practical Assessment, Research & Evaluation, 18(11), 1–14.
https://doi.org/10.7275/55hn-wk47
“In reality, only the assumption of normally distributed errors is relevant to multiple regression” (p. 2)
“ordinary least squares estimates will be unbiased and consistent as long as the errors are independent, but will not be efficient” (p. 9)
-
Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159.
https://doi.org/10.1037/0033-2909.112.1.155
“small, medium, and large ESs are respectively .10, .30, and .50” (p. 156)
- Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.
-
Muff, S., Nilsen, E. B., O'Hara, R. B., & Nater, C. R. (2022). Rewriting results sections in the language of evidence. Trends in Ecology & Evolution, 37(3), 203–210.
https://doi.org/10.1016/j.tree.2021.10.009
“we suggest language of evidence that allows for a more nuanced approach to communicate scientific findings as a simple and intuitive alternative to statistical significance testing” (Abstract)
- Hayes, A. F., & Cai, L. (2007). Using heteroskedasticity-consistent standard error estimators in OLS regression: An introduction and software implementation. Behavior Research Methods, 39(4), 709–722. https://doi.org/10.3758/BF03192961
Spot an error? Something still confusing? Tell us at support@statsforpsychology.com.