Stats for

Agreement and Reliability

Cronbach's Alpha

Cronbach's alpha measures how well a scale's items hang together.

Cronbach's alpha shows how strongly a set of items meant to measure one thing hang together and, under certain assumptions, estimates how reliable their total score is.

Example

How reliable are total scores on a new six-item loneliness scale? Two hundred adults rated six statements about loneliness from 1 (never) to 5 (always), and each person's six answers are added into one loneliness score.

One Scale + 2+ Items + Item Scores → Cronbach's Alpha
Estimates
Reliability of a total score
Observations
Independent
Headline Statistic
\alpha
Key Estimate
Cronbach's \alpha
Groups
1

The Basics

When to Use It

Use it when one group of people answered several items meant to measure the same thing, at one sitting, and you'll add the items into a total score. Picture six loneliness questions: if someone gives a lonely answer to one, do they tend to give lonely answers to the others? Alpha measures how strongly the answers hang together as a group, allowing for how many questions there are. When the questions measure the same trait equally strongly and their errors are unrelated, it estimates how reliable the total score is (Cronbach, 1951). The catch: questions can agree for reasons other than the intended trait, so a high alpha doesn't show that the scale measures loneliness, or only one thing. For example:

  • How reliable are total scores on a new six-item loneliness scale?
  • Do the ten items of a math-anxiety questionnaire hang together well enough to sum in a classroom study?
  • Is a five-item job-satisfaction measure reliable enough to correlate with burnout in a sample of nurses?

Vocabulary You Need

Definition Reliability

The extent to which a measure's scores can be replicated: measured again, by another rater or with other items, would people get the same scores? Formally, it's the share of the differences between people's scores that reflects real differences in what's measured rather than measurement error, so it runs from 0 (all noise) to 1 (no noise). It belongs to scores from a particular group of people under particular conditions, not to the test itself: the same test can be more reliable in a group whose scores vary widely. There are several kinds, each asking about a different source of noise: test-retest (occasions), inter-rater (raters) and internal consistency (items). A measure has to be reliable to be valid, but reliable scores can still measure the wrong thing. Glossary →

Definition True Score (T)

In classical test theory, each observed score is a true score plus measurement error, X = T + E. The true score is what the person would average over endless repetitions of the same measurement, with the random errors cancelling out. It's a property of the person and the measure together, not a perfect measure of the underlying trait: a biased test has biased true scores. It can't be observed directly; reliability says how much of the spread in observed scores is spread in true scores. Glossary →

Definition Internal Consistency

How strongly the items of a scale go together: whether people who score high on one item tend to score high on the others. It is one way to judge reliability from a single sitting, because items that all reflect the same thing should agree with one another. Items that measure one trait generally show a pattern of positive correlations, though they needn't be strong when each item carries a lot of noise. Strong correlations don't prove unidimensionality either: two related clusters of items can agree well overall. Coefficients such as Cronbach's alpha and McDonald's omega are often called internal consistency coefficients, though strictly they estimate the reliability of the total score. Glossary →

Definition Item-Total Correlation (r_{it})

The correlation between one item and the scale total. The corrected version, which is what software usually reports, leaves the item out of the total (correlating it with the remaining items) so it isn't correlated with itself. A high value means the item rises and falls with the rest of the scale; a value near zero means it measures something else, is mostly noise, or barely varies (as when nearly everyone gives the same answer), and a negative value usually means a reverse-keyed item wasn't recoded. It is read alongside “alpha if item deleted,” which shows how the scale's alpha would change without that item. Glossary →

Assumptions and Inference

1. One Underlying Trait Your items all measure one and the same trait.
What It Means
A single common factor is enough to reproduce how the items go together (with several constructs, each construct's items measure one trait). Coefficients built on a one-factor model, such as omega and alpha, and item-level models such as IRT, describe how well the items measure one trait only if there really is one. A scale made of distinct parts, such as worry and bodily symptoms, breaks it; score and analyze each part separately, or use a model with several factors.
Applies To
Items
How to Check
Start from the scale's content and theory. Then fit a one-factor model (or, with several constructs, one factor per construct) and look at its fit (the chi-square test, RMSEA, SRMR) and at the residual correlations it leaves between pairs of items. For right/wrong or yes/no items, fit that model to tetrachoric correlations (in lavaan, declare the items ordered, which uses WLSMV) or as a one-factor IRT model with its own fit statistics (M2, RMSEA, SRMSR); fit indices from ordinary Pearson correlations are misleading for 0/1 items. An exploratory factor analysis or parallel analysis can suggest how many factors there are. A high omega or alpha is not evidence of unidimensionality: several correlated factors can produce one.
How Much It Matters
A lot, and alpha won't warn you. A scale made of two loosely related clusters can still reach a high alpha, especially with many items (Cortina, 1993), and the total score then mixes two things. Alpha can still be calculated for such a scale; it's the reading as one trait's reliability that fails.

What to Do

  • Alternative Test Confirmatory Factor Analysis (CFA). Fit a one-factor model first and look at its fit and loadings.
  • Design Change If the items split into clear clusters, score and report each subscale, with its own reliability.
  • Alternative Test McDonald's Omega. For a scale with a general factor plus subfactors, hierarchical omega estimates how much of the total reflects the general factor.
2. No Shared Noise between Items Your items don't share anything beyond the trait, such as near-identical wording.
What It Means
The part of each item that the trait doesn't explain is unrelated from item to item. Items that repeat the same wording, share a response format the others don't, or are all reverse-keyed can share a little variance of their own. The factor model then counts that shared noise as trait, so the loadings, and the coefficients built from them (omega, alpha, AVE, composite reliability), come out too high. If you model the shared noise as a residual correlation, count it as error: omega and composite reliability then need twice that covariance added to the noise in their denominator.
Applies To
Items
How to Check
Look at the item wording for near-duplicates and for clusters of reverse-keyed or differently formatted items. After fitting the factor model, check the residual correlations between items: pairs the factor leaves strongly correlated point to shared noise.
How Much It Matters
A lot, and in the risky direction. Items that share wording, a response style or a format the others don't (such as all the reverse-keyed items) share noise, which alpha counts as shared trait, so it can overestimate the reliability, and the lower-bound guarantee no longer holds (Dunn et al., 2014). In the loneliness study, the one-factor model leaves no residual correlation between items larger than .06.

What to Do

  • Alternative Test McDonald's Omega. Fit a factor model with a residual correlation for the offending items, and count that shared noise as error in omega.
  • Design Change Replace near-duplicate items with ones that approach the trait from a different angle.
3. Tau-Equivalence Every item reflects the construct about equally strongly, though their averages may differ.
What It Means
Each item's true score changes by the same amount when the construct changes, so every item has the same loading on the common factor. Items may still differ in their average level and in how much error they carry. Coefficient alpha assumes it: when it holds and the items' errors are unrelated, alpha equals the total score's reliability; when some items reflect the construct more strongly than others, alpha underestimates it.
Applies To
Items
How to Check
Fit a one-factor model and compare the unstandardized loadings (in the items' own units): roughly equal loadings are fine. Without a factor model, look at the inter-item covariances (correlations, if the items have similar spreads) for a wide spread. Loadings that clearly differ are a reason to report omega.
How Much It Matters
With uncorrelated errors, alpha becomes a lower bound: when some items track the construct more strongly than others, alpha underestimates the total score's reliability, a little when loadings are similar and more the further they are from equal (Dunn et al., 2014). In the loneliness study a one-factor model fits the six items well, χ²(9) = 7.06, p = .63, RMSEA = .00, SRMR = .024: a non-significant χ² and a small RMSEA and SRMR mean the model reproduces the item correlations closely. The loadings run from .59 to .68 points (standardized, .57 to .67); tau-equivalence is about the unstandardized ones, in the items' own units. Constraining the loadings to be equal doesn't noticeably worsen the fit, Δχ²(5) = 1.07, p = .96. That doesn't prove the loadings are equal, but the data give no sign against it, and omega (.80) matches alpha.

What to Do

  • Alternative Test McDonald's Omega. Omega uses each item's own loading, so it doesn't need them equal.
  • Proceed with Caveat If you report alpha anyway, say it's a lower bound when the items differ in how strongly they relate to the rest, provided their errors are uncorrelated (no shared wording, method or response style). With correlated errors, alpha can overestimate reliability.
4. Independent Observations (for the Confidence Interval) The people (or other units) in your study don't influence each other.
What It Means
Each person's data comes from a separate, unconnected source, so knowing one person's scores tells you nothing about anyone else's. In designs that measure people more than once, their own scores are meant to be related; it's different people who must be independent. Methods built for clustered data, such as multilevel models and GEE, also allow members of the same group to be related, as long as the analysis names the grouping; then it's the groups that must be independent.
Applies To
Observations
How to Check
By design, not by a test. Did each person contribute their own data, with no clusters such as roommates, couples, therapy groups or classrooms whose members influence each other? If there are clusters, does the analysis model every one of them?
How Much It Matters
Moderate. Alpha itself is still a fair description of the sample, but Feldt's interval treats every person as new information, so clusters (classrooms, couples, repeated answers from the same person) make it too narrow.

What to Do

  • Alternative Test Multilevel SEM. A multilevel model that separates reliability within and between clusters.
  • Design Change Keep one completion of the scale per person.

Limitations

This Is Not Your Test If…

What It Doesn't Tell You

  • Whether the items measure one thing. An alpha of .80 fits a unidimensional scale, but also a scale made of two related clusters (Sijtsma, 2009). Check dimensionality separately, usually with exploratory or confirmatory factor analysis alongside what the items' content says; even a well-fitting one-factor model supports unidimensionality rather than proving it.
  • Whether it measures loneliness. Reliability is about consistency, not meaning. Six items could agree nicely while tapping social anxiety or low mood instead; that's a question of validity.
  • Who else. The .80 describes scores from these 200 adults. In a group with less variation in loneliness, such as only very lonely people, the same items would usually give a lower alpha.
  • How stable scores are over time. Alpha comes from one sitting, so it says nothing about whether a person's score would be the same next month. That's test-retest reliability.

Common Mistakes

  • Don'tForget to recode reverse-keyed items before computing alpha.

    InsteadFlip them first (6 − x on a 1–5 scale). A negative item-total correlation is the tell-tale sign you missed one.

  • Don'tHold every scale to a single “α ≥ .70” cutoff.

    InsteadAsk what the scores are for. Nunnally's standards rise from .70 for early research to .90 or more for decisions about individuals.

  • Don'tReport alpha as a fixed property of the scale.

    InsteadReport the alpha from your own sample, with its interval. It changes with the people answering and the number of items (Streiner, 2003).

  • Don'tDrop every item with a lower item-total correlation or a higher “alpha if item deleted.”

    InsteadJudge the item's content too, and expect small gains in alpha to shrink in new data.

  • Don'tRead alpha as the share of items that measure the construct, or as proof the scale is unidimensional.

    InsteadRead it as an estimate of the total score's reliability, and check dimensionality separately, with a factor analysis.

Result Decoder

Interpretation Guidelines

Benchmarks Cronbach's \alpha
  • α < .70 Low

    Below even the modest level Nunnally (1978) suggested for early-stage research. Check for unrecoded items or items that don't belong before blaming the construct.

  • .70 ≤ α < .80 Modest

    Nunnally's (1978) level for early stages of research, such as trying out a new scale, where modest reliability saves time.

  • .80 ≤ α < .90 Adequate for Research

    Nunnally's (1978) level for basic research that compares group means or correlates scores.

  • α ≥ .90 High

    Nunnally's (1978) minimum when important decisions rest on one person's score, with .95 desirable. In research scales, very high values can signal items that repeat each other.

Nunnally (1978, pp. 245–246) tied these levels to what the scores are for, and wrote them for reliability in general, not as a pass mark for alpha: .70 for early-stage research, .80 for basic research, and .90 as the minimum (with .95 desirable) when important decisions rest on one person's score. The popular “α ≥ .70 is acceptable” rule drops that context. Alpha also rises with the number of items, so the same value means less on a long scale, and a very high alpha can mean redundant items rather than a better scale.

More about effect size ↓

How It's Reported

Equation Form

Example

α = .80, 95% CI [.76, .84], r̄ = .40, rit = .51 to .58

Table 1. Item Statistics for the Six-Item Loneliness Scale

Example
MSDr_{it}\alpha if Item Deleted
  1. I feel cut off from other people.
2.8201.05.55.77
  1. I wish I had more people to talk to.
2.9151.01.55.77
  1. I spend more time alone than I'd like.
3.0651.03.51.78
  1. I have people I can count on. (R)
2.8700.99.57.77
  1. I feel that no one really understands me.
2.6250.98.58.77
  1. I feel lonely, even when I'm with others.
2.7251.02.58.77

Note. N = 200. Items rated 1 (never) to 5 (always). (R) = reverse-keyed, recoded as 6 − x before analysis. rit is the corrected item-total correlation, each item with the total of the other five. Cronbach's α for all six items = .80, 95% CI [.76, .84] (Feldt).

Full Text (APA Style)

Example

Total scores on the six-item loneliness scale averaged 17.02 out of 30 (SD = 4.31). A one-factor model fit the six items well, χ²(9) = 7.06, p = .63, RMSEA = .00, SRMR = .024, and constraining the loadings to be equal did not worsen fit, Δχ²(5) = 1.07, p = .96. Reliability of the total score was adequate for research use, Cronbach's α = .80, 95% CI [.76, .84] (Feldt), with a mean inter-item correlation of .40. Corrected item-total correlations ranged from .51 to .58, and removing any one item would have lowered alpha. Item 4 was reverse-scored before analysis.

  • Report alpha from your own sample, not only the value from the scale's original paper. Reliability belongs to scores from particular people.
  • Give the interval and say which method produced it (Feldt's F-based interval here).
  • Say which items were reverse-scored, and how many items the scale has; alpha rises with the number of items.
  • Report the checks on alpha's assumptions: whether a one-factor model fits reasonably well and whether constraining the loadings to be equal noticeably worsens the fit. They're evidence about the assumptions, not proof: a non-significant difference test means the data didn't argue against equal loadings, and good overall fit doesn't guarantee uncorrelated errors. If equal loadings clearly fit worse, report McDonald's omega alongside or instead.

Reading Each Piece

The equation form above, piece by piece, in this order: how big, then how certain.

α = .80 The six items move together strongly enough to give an alpha of .80. If their errors are unrelated, that's a lower-bound estimate of the total score's reliability; if the items also measure the trait equally strongly, as our checks suggest, it estimates it directly: about 80% of the variation in totals reflects true-score differences between people rather than random error. Alpha alone can't show that those differences are loneliness. It meets Nunnally's standard for basic research, not the .90 he set for decisions about individuals. Benchmarks ↑
r̄ = .40 The average correlation between pairs of items is .40. Alpha is high not because any two items agree closely, but because six moderately related items add up to a steadier total.
rit = .51 to .58 Every item goes with the rest about equally well, and dropping any of them would lower alpha. No item stands out as measuring something else.
95% CI [.76, .84] The scale's alpha in the population these adults represent is plausibly anywhere from .76 to .84: comfortably above .70, but not clearly above .80.

Calculations

The Running Example

Example

The loneliness scale study: 200 adults, six loneliness items rated 1 to 5 (one of them reverse-keyed), added into a total score whose reliability we estimate.

There's no null hypothesis to test here. Cronbach's alpha is an estimate, like a mean, and its uncertainty comes from a confidence interval. Testing whether alpha is above 0 would tell us almost nothing, since any set of items meant to measure one thing should clear that bar.

What Your Data Should Look Like

One row per person and one numeric column per item, each item on the same response scale. Reverse-keyed items can be stored as answered and recoded in the analysis, as long as you know which ones they are. People with missing items are usually dropped, or the missing answers handled before alpha is computed. Software differs: SPSS drops anyone with a missing item, while R's psych::alpha and Python's pingouin use every available pair of items by default, so say which you used.

Example
participantlonely_1lonely_2lonely_3lonely_4lonely_5lonely_6
1413343
2353333
3334331
4122422
5233423
6444232
… 194 more rows
ItemMSDItem–Total r
Lonely 12.821.05.55
Lonely 22.921.01.55
Lonely 33.071.03.51
Lonely 4 (R)2.870.99.57
Lonely 52.630.98.58
Lonely 62.731.02.58
Total17.024.31

N = 200. The total adds the 6 items; each item's r is with the total of the other 5 (the corrected item–total correlation).

(R) Reverse-keyed: shown after recoding as 6 − the answer, so a high score means more of the trait on every item.

Simulated data.

Download all 200 rows (data.csv)

Napkin Math

The whole test, one idea at a time. Under each heading, a line says why the step matters, and the green note reads the equation in plain English with the loneliness scale study's numbers. Collapse all to read just the headings: they tell the whole story on their own.

  1. 1. Flip the Item That Is Worded the Other Way

    Every item has to point the same way before we add them up. On item 4, a high answer means less loneliness, so we turn its scale around first.

    x' = (\text{min} + \text{max}) - x =6 - 3.13

    \bar{x}_4 \text{ (as answered)} = 3.13

    \bar{x}_4' \text{ (recoded)} = 2.87

    On our 1–5 scale, min + max = 6, so each answer x on item 4 becomes 6 - x. The item's average goes from 3.13 as answered to 6 − 3.13 = 2.87 once it points toward loneliness like the rest. Before the flip it correlated −.57 with the other five items' total; after, +.57.

  2. 2. Add Up How Much Each Item Varies on Its Own

    Each item's variance mixes the construct with that item's own noise. Adding them gives how much the total would vary if the items had nothing in common.

    \sum_{i=1}^{k} s_i^2 =1.10 + 1.02 + 1.06 + 0.98 + 0.97 + 1.04

    \sum s_i^2 = 6.17

    s_i^2 is how much item i varies across our 200 adults, and k is our 6 items. The six variances are all close to 1 point², and together they come to 6.17.

  3. 3. Find How Much the Total Score Varies

    When items go together, people who are high on one are high on the others, so the totals spread out much more than the items' variances alone would suggest. That extra spread comes from the covariances between items.

    s_X^2 = \dfrac{\sum (X_j - \bar{X})^2}{n - 1}, \quad X_j = \sum_{i} x_{ij} =4.31^2

    = 18.58 points²

    \bar{X} = 17.02

    s_X = 4.31

    X_j is one adult's total, the sum of their six recoded answers (6 to 30). The totals average \bar{X} = 17.02 with an SD of 4.31, so s_X^2 = 4.31² = 18.58, three times the 6.17 the items would give if they were unrelated.

  4. 4. Compare the Two to Estimate Reliability

    When items measure the same trait, people who score high on one tend to score high on the others, so the total varies more than the items' separate variances add up to. Alpha compares the two, the sum of the item variances against the total's variance. That reads as reliability, the true-score share of the total, when the items' errors are unrelated and the items measure the trait equally strongly. Shared wording, response habits or a second trait can also make items move together, so shared variation isn't automatically reliable measurement of loneliness. Alpha scales that share up by k/(k − 1) so a perfectly consistent set of items can reach 1.

    \alpha = \dfrac{k}{k - 1}\left(1 - \dfrac{\sum s_i^2}{s_X^2}\right) =\dfrac{6}{5}\left(1 - \dfrac{6.17}{18.58}\right)

    = .80

    k = 6 items, \sum s_i^2 = 6.17 and s_X^2 = 18.58. 6.17 ÷ 18.58 = .332, so 1 − .332 = .668, and 6/5 × .668 = .80.

  5. 5. Check Alpha against the Average Correlation between Items

    Alpha is really about how strongly items go together, and how many there are. Working from the average correlation between pairs of items gives nearly the same answer and shows both ingredients.

    \bar{r} = \dfrac{\sum r_{ij}}{k(k - 1)/2} =\dfrac{6.04}{15}

    = .40

    \alpha_{\text{std}} = .80

    \bar{r} is the average of the 15 correlations r_{ij} between pairs of our items (they run from .35 to .48 and add to 6.04), so \bar{r} = .40. Standardized alpha, \alpha_{\text{std}} = k\bar{r} / (1 + (k - 1)\bar{r}), steps it up for six items: 6 × .40 = 2.40 on top and 1 + 5 × .40 = 3.00 below give .80, the same as alpha because the six items have nearly equal spreads.

  6. 6. See How Well Each Item Fits with the Rest

    A scale is only as good as its items. Correlating each item with the total of the other five, and recomputing alpha without it, shows whether any item is pulling in a different direction.

    r_{it} = r\left(x_i,\; X - x_i\right) =r(x_3,\; X - x_3)

    = .51

    \alpha \text{ without item 3} = .78

    r_{it} \text{ (highest, item 5)} = .58

    For each item x_i, X - x_i is the total of the other five. The weakest fit is item 3, “I spend more time alone than I'd like,” at .51, and dropping it would lower alpha from .80 to .78. Every item's r_{it} is between .51 and .58 (Table 1).

  7. 7. Give a Range of Plausible Values for the Scale's Alpha

    Another 200 adults would give a somewhat different alpha. Feldt's (1965) interval uses (1 − α)/(1 − α̂), the population's share of error over the sample's, to show how far the population value could plausibly be from ours. That ratio follows an F distribution exactly only under a specific model, with normally distributed scores and the items related in the same way (Feldt, Woodruff & Salih, 1987); with real items it's an approximation. For ordinal, skewed or few-category items, a bootstrap interval is the usual check. Ours are close to symmetric (each item's skew is between −.11 and .17), and resampling our 200 adults 5,000 times gives .75 to .84, nearly the same range.

    \left[\,1 - (1 - \hat{\alpha})\,F_{.975},\;\; 1 - (1 - \hat{\alpha})\,F_{.025}\,\right] =1 - .199 \times 1.231,\;\; 1 - .199 \times 0.800

    = [.76, .84]

    F_{.975} = 1.231

    F_{.025} = 0.800

    1 - \hat{\alpha} is our .199 (1 − .801). F_{.975} = 1.231 and F_{.025} = 0.800 cut off the top and bottom 2.5% of an F distribution with 199 and 995 df (n − 1 and (n − 1)(k − 1)). That gives 1 − .245 = .76 and 1 − .159 = .84.

Run It in Software

The code below reads the running example's data from data.csv; save it in the same folder you run the code from.

SPSS (version 30, as documented by IBM)
  1. Recode the reverse-keyed item first: Transform → Compute Variable, name the target lonely_4r, and enter 6 - lonely_4.
  2. Analyze → Scale → Reliability Analysis.
  3. Move lonely_1, lonely_2, lonely_3, lonely_4r, lonely_5 and lonely_6 into Items, and leave Model on Alpha.
  4. Under Statistics, tick Scale and Scale if item deleted (under Descriptives for), Correlations (under Inter-Item), and Intraclass correlation coefficient with Model Two-Way Mixed and Type Consistency (this gives the confidence interval for alpha).
  5. Click Continue, then OK.

Or paste this syntax:

COMPUTE lonely_4r = 6 - lonely_4.
EXECUTE.

RELIABILITY
  /VARIABLES=lonely_1 lonely_2 lonely_3 lonely_4r lonely_5 lonely_6
  /SCALE('Loneliness') ALL
  /MODEL=ALPHA
  /STATISTICS=SCALE CORR
  /SUMMARY=TOTAL
  /ICC=MODEL(MIXED) TYPE(CONSISTENCY) CIN=95 TESTVAL=0.

Reading the Output

Read Cronbach's Alpha first, then its 95% confidence interval from the Average Measures row of the intraclass correlation table, then the Item-Total Statistics to see how each item fits.

Reliability Statistics

Cronbach's Alpha Cronbach's Alpha Based on Standardized Items N of Items
.801 .802 6

Item-Total Statistics

Scale Mean if Item Deleted Scale Variance if Item Deleted Corrected Item-Total Correlation Squared Multiple Correlation Cronbach's Alpha if Item Deleted
lonely_1 14.20 13.266 .551 .315 .772
lonely_2 14.11 13.451 .554 .321 .771
lonely_3 13.96 13.671 .507 .262 .782
lonely_4r 14.15 13.465 .570 .332 .768
lonely_5 14.40 13.426 .580 .342 .765
lonely_6 14.30 13.224 .580 .357 .765

Scale Statistics

Mean Variance Std. Deviation N of Items
17.02 18.583 4.311 6

Intraclass Correlation Coefficient

Intraclass Correlation 95% CI Lower Bound Upper Bound F Test Value df1 df2 Sig
Single Measures .402 .340 .469 5.033 199 995 .000
Average Measures .801 .755 .841 5.033 199 995 .000

The Average Measures ICC for a two-way mixed, consistency model is Cronbach's alpha, and its interval is Feldt's. SPSS also prints the inter-item correlation matrix (not shown; the correlations run from .35 to .48). Ignore the Single Measures row here (it's the reliability of one item) and the F test (it only tests alpha against 0). If you forget to recode item 4, alpha drops to .48 and item 4's corrected item-total correlation is negative.

Output laid out from this page's computed results, not captured from a live run; labels and layout may differ slightly in your version.

R
library(psych)

loneliness <- read.csv("data.csv")
items <- loneliness[, paste0("lonely_", 1:6)]
items$lonely_4 <- 6 - items$lonely_4     # recode the reverse-keyed item

result <- alpha(items)
result$total          # raw_alpha, std.alpha, average_r
result$item.stats     # r.drop = corrected item-total correlations
result$alpha.drop     # raw_alpha if each item is dropped

print(alpha.ci(result$total$raw_alpha, n.obs = 200, n.var = 6))   # Feldt 95% interval
 raw_alpha std.alpha  G6(smc) average_r      S/N        ase     mean        sd
  0.801311 0.8017028 0.774753 0.4025651 4.042935 0.02179075 2.836667 0.7184573
  median_r
 0.3987189
           n     raw.r     std.r     r.cor    r.drop  mean        sd
lonely_1 200 0.7089159 0.7043730 0.6170720 0.5506584 2.820 1.0502931
lonely_2 200 0.7057859 0.7060598 0.6206729 0.5537918 2.915 1.0113800
lonely_3 200 0.6735156 0.6717072 0.5659552 0.5072941 3.065 1.0276455
lonely_4 200 0.7149450 0.7178891 0.6369729 0.5703890 2.870 0.9889337
lonely_5 200 0.7217005 0.7252540 0.6481473 0.5803711 2.625 0.9844903
lonely_6 200 0.7266541 0.7264169 0.6535283 0.5803291 2.725 1.0220558
         raw_alpha std.alpha   G6(smc) average_r      S/N   alpha se
lonely_1 0.7722159 0.7724410 0.7340994 0.4043693 3.394464 0.02548833
lonely_2 0.7713282 0.7719170 0.7331977 0.4036522 3.384368 0.02559701
lonely_3 0.7820670 0.7823661 0.7457149 0.4182579 3.594873 0.02439292
lonely_4 0.7676432 0.7682099 0.7298310 0.3986227 3.314248 0.02601605
lonely_5 0.7654358 0.7658727 0.7276901 0.3954914 3.271181 0.02626763
lonely_6 0.7651586 0.7655016 0.7247178 0.3949969 3.264421 0.02625729
                var.r     med.r
lonely_1 0.0017480027 0.4037964
lonely_2 0.0013101983 0.4037964
lonely_3 0.0013149155 0.4220417
lonely_4 0.0020511270 0.3822500
lonely_5 0.0017795588 0.3816946
lonely_6 0.0009067396 0.3931747
   95% confidence boundaries (Feldt)
 lower alpha upper
  0.76   0.8  0.84

raw_alpha is Cronbach's alpha (.801) and std.alpha the standardized version (.802) from the average inter-item correlation (average_r, .40). r.drop is the corrected item-total correlation, .51 to .58, and alpha.drop gives alpha with each item left out, .765 to .782. The last block is Feldt's 95% interval, .76 to .84 (alpha.ci() keeps the unrounded limits, .755 and .841, in lower.ci and upper.ci). The mean and sd in result$total are for the average of the items, not the sum, unless you set cumulative = TRUE.

Printing result also shows "95% confidence boundaries" by two methods: Feldt's, the same interval as here, and Duhachek and Iacobucci's, which comes from a standard error and can differ in small samples. Say which one you report. Feldt's assumes roughly normal item scores; alpha(items, n.iter = 1000) adds a bootstrap interval (result$boot.ci), which doesn't. Here it came out at about .75 to .84, close to Feldt's; the limits shift slightly from run to run.

Instead of recoding by hand, alpha(items, keys = c("lonely_4")) reverses the named items for you. Leave check.keys at its default (FALSE): set to TRUE, it flips any item with a negative loading on the first principal component, which can hide a real problem with an item.

Output from a real run: R 4.6.1 (psych 2.6.5), run 2026-10-03.

Python
import pandas as pd
import pingouin as pg

loneliness = pd.read_csv("data.csv")
items = loneliness[[f"lonely_{j}" for j in range(1, 7)]].copy()
items["lonely_4"] = 6 - items["lonely_4"]           # recode the reverse-keyed item

print(pg.cronbach_alpha(data=items))

total = items.sum(axis=1)
print({c: round(float(items[c].corr(total - items[c])), 3) for c in items})      # corrected item-total r
print({c: round(float(pg.cronbach_alpha(data=items.drop(columns=c))[0]), 3) for c in items})   # alpha if deleted
(0.801311007268951, array([0.755, 0.841]))
{'lonely_1': 0.551, 'lonely_2': 0.554, 'lonely_3': 0.507, 'lonely_4': 0.57, 'lonely_5': 0.58, 'lonely_6': 0.58}
{'lonely_1': 0.772, 'lonely_2': 0.771, 'lonely_3': 0.782, 'lonely_4': 0.768, 'lonely_5': 0.765, 'lonely_6': 0.765}

pg.cronbach_alpha returns alpha and a 95% interval. The interval is Feldt's F-based one, with the limits rounded to three decimals. The next two lines are the corrected item-total correlations and alpha if each item is deleted. With NumPy 2, the first line prints as (np.float64(0.801311007268951), …); the value is the same.

Output from a real run: Python 3.9.6 (pingouin 0.5.5, pandas 2.3.3, numpy 1.26.4), run 2026-10-03.

Impact

Effect Size

Alpha is its own size measure. What it calculates is how strongly the items move together, compared with how much their total varies. Only under two conditions, items that measure one trait equally strongly (tau-equivalence) and errors that are uncorrelated, does that estimate the share of the total score's variance that reflects true differences between people rather than measurement error. When those conditions fail, alpha can still be calculated, but it no longer equals the total score's reliability. It has no units, so it can be compared across scales scored in different units, but it still depends on the number of items (see below) and on how varied the sample is. A sample alpha can even fall below 0, usually a sign of an unrecoded item.

\alpha = \dfrac{k}{k - 1}\left(1 - \dfrac{\sum s_i^2}{s_X^2}\right) = \dfrac{6}{5}\left(1 - \dfrac{6.17}{18.58}\right) = .80

Our checks support the conditions (see Assumptions), so about four-fifths of the spread in our adults' totals is estimated to be consistent differences between people, and one-fifth noise. Whether those differences are loneliness is a question of validity, not reliability. That's enough to compare groups or correlate the scores with other measures. For one person, though, the standard error of measurement is 4.31 × √(1 − .80) ≈ 1.9 points. The conventional ± 1.96 × SEM band around a score of 17 runs from about 13 to 21. That's a rough picture of how much measurement error one score carries, assuming error is the same size at every score level, not a precise interval for that person's true score or a prediction of their next score. That's why Nunnally (1978) asked for .90 before making decisions about one person.

Two things change alpha besides the quality of the items. It rises with the number of items: six items with our average inter-item correlation of .40 give .80, but three such items would give .67 and twelve would give .89 (Cortina, 1993). And with uncorrelated item errors, it's a lower bound: equal to the reliability under tau-equivalence, and an underestimate when items differ in how strongly they reflect the construct (Sijtsma, 2009; McNeish, 2018). The lower bound isn't guaranteed beyond that. Items that share wording or a response style have correlated errors, which can push alpha above the true reliability (Dunn et al., 2014). With unequal loadings, McDonald's omega is usually the better estimate, though it rests on its own factor model, which has to fit. The benchmarks are conventions tied to a purpose; whether our loneliness scores are reliable enough depends on what they'll be used for.

Benchmarks Cronbach's \alpha at a Glance
Low
α < .70
Modest
.70 ≤ α < .80
Adequate for Research
.80 ≤ α < .90
High
α ≥ .90

Full benchmarks and caveats ↑

Uncertainty

Alpha is usually reported without a p-value. Testing whether it's above 0 is possible, but nearly any set of items meant to measure one thing clears that bar, so the answer is uninformative. The question that matters is how precisely alpha is estimated, and that's what the confidence interval shows.

Our interval, .76 to .84, excludes 0 by a wide margin, and also excludes .70. It doesn't exclude .80, so we can't say the population alpha is above .80.

If you do need a test, Feldt's F distribution gives one against any specific value you name in advance, such as .70 (Feldt, Woodruff & Salih, 1987). Like any p-value, it wouldn't say how reliable is reliable enough; that's a judgment about what the scores are for.

Precision and Sample Size

Power is the chance that a study detects an effect that's really there. Alpha is an estimate rather than a test, so for reliability the planning question is precision: how many people you need for the confidence interval to be usefully narrow. The width depends mostly on the number of people and on alpha itself; adding items helps only a little once there are more than a handful.

With an alpha around .80 and six items, Feldt's interval runs from about .70 to .88 with 50 people, .73 to .86 with 100, .75 to .84 with our 200, and .77 to .83 with 400. Doubling the items to twelve at the same alpha barely narrows any of these.

Plan the sample size for the interval you need, not for a significance test. When alpha is near .80, about 200 people pin it down to within roughly ±.04 (233 for a total width of .08; the interval is slightly lopsided, reaching a little further below alpha than above). The planner below uses Feldt's formula from the Napkin Math, taking the sample alpha to equal your guess. Try it:

Other Info

Follow-Up and Post Hoc Tests

None. This page estimates rather than tests, so there are no follow-up comparisons to run.

References

  • Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/BF02310555
    “is shown to be the mean of all split-half coefficients resulting from different splittings of a test” (Abstract, p. 297)
  • Feldt, L. S. (1965). The approximate sampling distribution of Kuder-Richardson reliability coefficient twenty. Psychometrika, 30(3), 357–370. https://doi.org/10.1007/BF02289499
  • Feldt, L. S., Woodruff, D. J., & Salih, F. A. (1987). Statistical inference for coefficient alpha. Applied Psychological Measurement, 11(1), 93–103. https://doi.org/10.1177/014662168701100107
    “Kristof (1963) and Feldt (1965) independently derived a transformation of the sample alpha coefficient, which is proven to be distributed as F” (p. 94)
  • Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78(1), 98–104. https://doi.org/10.1037/0021-9010.78.1.98
    “a demonstration of the effects of test length and dimensionality on alpha” (Abstract)
  • Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach's alpha. Psychometrika, 74(1), 107–120. https://doi.org/10.1007/s11336-008-9101-0
    “Alpha is not a measure of internal consistency. Neither is it a measure of the degree of unidimensionality” (Section 6, Conclusions)
    “Alpha is a lower bound to the reliability, in many cases, even a gross underestimate” (Section 1, Introduction)
  • McNeish, D. (2018). Thanks coefficient alpha, we'll take it from here. Psychological Methods, 23(3), 412–433. https://doi.org/10.1037/met0000144
    “In many circumstances, violating these assumptions yields estimates of reliability that are too small, making measures look less reliable than they actually are.” (Abstract)
  • Dunn, T. J., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105(3), 399–412. https://doi.org/10.1111/bjop.12046
    “The model that defines alpha is the essentially tau-equivalent model” (Accepted manuscript, p. 6)
    “Unidimensionality is a requisite for all the models considered here.” (A practical alternative to alpha)
    “Generally, the larger the violation of tau-equivalence the more coefficient alpha underestimates score reliability” (Accepted manuscript, p. 8)
  • Streiner, D. L. (2003). Starting at the beginning: An introduction to coefficient alpha and internal consistency. Journal of Personality Assessment, 80(1), 99–103. https://doi.org/10.1207/S15327752JPA8001_18
    “(a) that it is a fixed property of the scale, (b) that it measures only the internal consistency of the scale, (c) that higher values are always preferred over lower ones, and (d) that it is restricted to the range of 0 to 1” (Abstract)
  • Nunnally, J. C. (1978). Psychometric theory (2nd ed.). McGraw-Hill.
  • Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012
    “Reliability is defined as the extent to which measurements can be replicated.” (Introduction)
  • Aldridge, V. K., Dovey, T. M., & Wade, A. (2017). Assessing test-retest reliability of psychological measures: Persistent methodological problems. European Psychologist, 22(4), 207–218. https://doi.org/10.1027/1016-9040/a000298

Spot an error? Something still confusing? Tell us at support@statsforpsychology.com.