Categorical Data and Proportions
Pearson Chi-Square Test
A chi-square test checks whether two categorical variables are related.
A Pearson chi-square test asks whether the categories of two variables go together more than we'd expect by chance. It compares how many people actually fall into each combination with how many we'd expect if the variables were unrelated.
Does where students turn first for help with stress depend on their year in school? Two hundred undergraduates, 80 first-years and 120 upper-years, said where they would turn first when feeling stressed, choosing friends or family, campus counseling, or no one.
- Tests
- Association between two categorical variables
- Observations
- Independent
- Parametric
- No
- Headline Statistic
- \chi^2
- Effect Size
- Cramér's V
- Groups
- 1
The Basics
When to Use It
Use it when you've sorted each person into one category on each of two categorical variables and want to know whether the two are associated: whether the mix of one variable changes across the categories of the other. Each person counts once, in one cell. The same test applies when you sampled separate groups, such as a set number of first-years and upper-years, and compare their mix of categories (sometimes called a test of homogeneity). A related test, the goodness-of-fit test, works on one categorical variable: it asks whether that variable follows a set of proportions you specify in advance (say, equal thirds), rather than whether two variables go together. Its χ² is computed the same way, but the expected counts come from those set proportions, not from the table's totals, and df = categories − 1. For example:
- Does where students turn first for help with stress depend on their year in school?
- Are therapy clients' reasons for dropping out (cost, scheduling, felt better) different across individual, group and online formats?
- Does the attachment style assigned in the Strange Situation differ between children in home care and those in daycare?
Vocabulary You Need
Definition Contingency Table
A table that cross-classifies people on two categorical variables, one along the rows and one along the columns, so each cell counts the people with that combination. The row and column totals (the margins) show how each variable is spread on its own; the cells show how the two go together. Tables are described by their size, rows × columns, such as a 2 × 3 table. Glossary →
Definition Expected Frequency (E)
The count the null hypothesis predicts for a cell. For one group compared with a fixed proportion it's n × π₀. In a test of independence it's the row total times the column total divided by the total sample size, so each row is spread across the columns in the same proportions as everyone combined. Other tests have their own; McNemar's test, for instance, expects the people who switched to split evenly between the two directions. It's an average, so it needn't be a whole number. Chi-square tests are only accurate when expected frequencies aren't too small. Glossary →
Definition Chi-Square Statistic (\chi^2)
In a table test, for each cell, the squared gap between the observed and expected count, divided by the expected count, added up across all the cells. It is 0 when every count matches its expectation and grows as they drift apart. Other tests, such as likelihood-ratio tests that compare two models, also produce a χ², which grows with how much worse the more constrained model fits. Either way, it is compared with a chi-square distribution, the values it would take across many samples if the null hypothesis were true, to get a p-value; that distribution depends on the degrees of freedom, and its average equals them. It also grows with sample size, so it isn't a measure of how strong an effect is. Glossary →
Assumptions and Inference
1. Independent Observations (for the p-Value and Interval) The people (or other units) in your study don't influence each other.
- What It Means
- Each person's data comes from a separate, unconnected source, so knowing one person's scores tells you nothing about anyone else's. In designs that measure people more than once, their own scores are meant to be related; it's different people who must be independent. Methods built for clustered data, such as multilevel models and GEE, also allow members of the same group to be related, as long as the analysis names the grouping; then it's the groups that must be independent.
- Applies To
- Observations
- How to Check
- By design, not by a test. Did each person contribute their own data, with no clusters such as roommates, couples, therapy groups or classrooms whose members influence each other? If there are clusters, does the analysis model every one of them?
- How Much It Matters
- A lot. Each person must be counted once, in exactly one cell. Counting the same people twice (before and after, or several answers each) or sampling clusters such as friends or classmates makes χ² too big and p too small.
What to Do
- Design Change Count each person once. If people gave several answers, analyze one (say, their first choice) or ask a question with one answer.
- Alternative Test If you asked the same people a yes/no question twice, McNemar's test compares the two answers within each person (see Related Methods below).
- Alternative Test Multilevel Logistic Regression. If people come in clusters (classrooms, therapy groups, families), model the clustering instead of ignoring it.
2. Large Enough Expected Counts (for the p-Value and Interval) χ² can be computed from any table, but its usual p-value relies on a large-sample approximation that sparse tables can make unreliable.
- What It Means
- The p-value comes from a smooth curve (the chi-square curve, or the normal curve for a z-test), which only approximates the lumpy distribution of real counts. The approximation works when the expected count in each cell, not the observed count, is reasonably large, and breaks down when many cells expect only a handful of people.
- Applies To
- Observations
- How to Check
- Look at the expected counts (row total × column total ÷ N), which most software prints; SPSS Crosstabs reports the smallest one under the Chi-Square Tests table. A common rule: at least 80% of cells expect 5 or more, and none expects fewer than 1 (McHugh, 2013). The rule is cautious: for 2 × 2 tables, Campbell (2007) found the “N − 1” chi-square (χ² × (N − 1) ÷ N, which SPSS prints as Linear-by-Linear Association) accurate whenever every expected count is at least 1.
- How Much It Matters
- Moderate, and only when expected counts are small. Small expected counts don't make the arithmetic wrong; they make the chi-square curve a poor fit, so p can be off in either direction. A large sample doesn't settle it on its own: if some categories are rare, a big table can still have sparse cells. The familiar cutoffs (no more than 20% of cells expecting under 5, none under 1) are guidelines, not exact rules. In the help-seeking study the smallest expected count is 16, so it's fine.
What to Do
-
Alternative Test
Fisher's Exact Test. It computes p exactly, conditioning on the table's totals, instead of from the chi-square curve, so small counts aren't a problem. It's usually introduced for 2 × 2 tables; for larger ones like ours, the Fisher–Freeman–Halton extension does the same (in R,
fisher.test(tab), or with a Monte Carlo option for big tables). -
Robust Estimation
Get p by simulation, which reshuffles the table many times instead of relying on the curve (in R,
chisq.test(tab, simulate.p.value = TRUE)). - Design Change Merge categories that are rare and similar in meaning, such as two small help sources, before running the test. Decide this on meaning, not on which merge gives a significant result.
Limitations
This Is Not Your Test If…
- Your table is too sparse for the usual chi-square approximation (exact tests exist for 2 × 2 and larger tables) Fisher's Exact Test →
- You measured the same people twice (yes/no, before and after) McNemar's Test →
- You're comparing two proportions and want an interval z-Test for Proportions →
- Your categories are ordered and the order matters Goodman and Kruskal's Gamma →
- You want to predict a category from several variables Logistic Regression →
- Your outcome is a score and you're comparing two groups t-Tests →
- Your outcome is a score and you're comparing three or more groups ANOVA Family →
What It Doesn't Tell You
- Why. Nothing here shows that being a first-year causes students to avoid counseling; first-years may simply not know the service exists yet, or differ in other ways. A third variable, such as living on campus, can create, hide or even reverse an association in a table that leaves it out.
- Whether it matters. Significance doesn't make a 17-point gap in counseling use important; that's a judgment for the people who run student services.
- Who else. The result describes students like these 200 at one point in time, and only as well as “where would you turn first?” predicts what students actually do.
- That there's no association, if p had been above .05. A non-significant χ² would have meant the data were inconclusive, not that year and help-seeking are unrelated. With small samples, even sizable differences in percentages can miss significance.
Common Mistakes
-
Don'tRun a chi-square test on percentages, means or averaged ratings.
InsteadRun it on raw counts of people. Percentages hide the sample size the test depends on.
-
Don'tCount the same person in more than one cell.
InsteadGive each person exactly one row and one category per variable. For before-and-after answers, use McNemar's test.
-
Don'tReport a significant χ² and stop, as though it says which categories differ.
InsteadShow the row percentages and check adjusted residuals to see which cells drive the result.
-
Don'tCheck whether the observed counts are at least 5.
InsteadCheck the expected counts. A cell can observe 0 people and still be fine if it expects enough.
-
Don'tApply Yates' continuity correction to a table larger than 2 × 2.
InsteadReport the uncorrected Pearson χ². Yates' correction only exists for 2 × 2 tables, and even there it tends to be overly cautious. For a 2 × 2 table, Pearson's test, Fisher's exact test and other small-sample options behave differently; choose one in your analysis plan, based on the sample, and say which you used.
-
Don'tRead a bigger χ² as a stronger effect.
InsteadReport an effect size suited to the design, such as Cramér's V, an odds ratio or a model's coefficients. χ² grows with sample size even when the effect doesn't.
-
Don'tRead a p-value as the chance that the null hypothesis is true.
InsteadRead it as: if the null were true, how often data at least this extreme would turn up.
Result Decoder
Significance and Effect Size Guidelines
Benchmarks Significance (p Value)
-
p < .001
Very strong evidence against the null hypothesis.
-
.001 ≤ p < .01
Strong evidence against the null hypothesis.
-
.01 ≤ p < .05
Statistically significant at α = .05. Moderate evidence against the null hypothesis.
-
.05 ≤ p < .10
Not significant. Weak evidence; sometimes called a “trend.” Treat it as inconclusive.
-
p ≥ .10
Not significant. Little evidence against the null, which is not the same as evidence of no effect.
These evidence labels are one common convention for describing p in graded language rather than as significant or not; the boundaries are rough guides, not rules, and other authors draw them differently. Muff et al. (2022) argue for this kind of language of evidence over a single cutoff. α is chosen before the analysis, and p never says how big or important an effect is.
Benchmarks Effect Size (Cramér's V)
-
V < .10
Below Small
Smaller than the conventional small benchmark. Whether an effect this size matters depends on the setting.
-
.10 ≤ V < .30
Small
A relatively modest overall association, by convention.
-
.30 ≤ V < .50
Medium
A moderate overall association, by convention.
-
V ≥ .50
Large
A relatively substantial overall association, by convention.
These cutoffs are Cohen's (1992) conventional values for w. They apply to V as-is when the table's smaller side has two categories, as in our 2 × 3 table. For larger tables divide them by √(k − 1), where k is the smaller of the number of rows and columns. They are conventions translated into V units, not independently validated benchmarks for V, and not measures of practical importance.
How It's Reported
Equation Form
V = .21, 95% CI [.04, .34], χ²(2, N = 200) = 8.61, p = .014
Full Text (APA Style)
Upper-year students were more likely than first-years to turn first to campus counseling (27% vs. 10%), while first-years turned somewhat more often to friends or family (60% vs. 52%) or to no one (30% vs. 22%), gaps too small to single out. The association between year and first source of help was small, Cramér's V = .21, 95% CI [.04, .34]. A chi-square test of independence showed it was statistically significant, χ²(2, N = 200) = 8.61, p = .014. Adjusted residuals showed the difference lay in counseling (±2.89); no other cell exceeded ±1.96.
- Give the percentages for each group, so readers can see the pattern χ² summarizes. Percentage within the groups you're comparing (here, within each year).
- Put the degrees of freedom and the sample size in parentheses: χ²(2, N = 200).
- Give the interval for V and say how it was computed (here, by inverting the noncentral chi-square, as DescTools'
CramerV(tab, conf.level = 0.95)does). - Say which correction you used, if any. Yates' correction applies only to 2 × 2 tables; this table has none.
- Italicize Latin symbols (V, N, p) but not Greek ones (χ²), and report exact p to three decimals.
Reading Each Piece
The equation form above, piece by piece, in this order: how big, how certain, then how surprising.
| V = .21 | The association between year and first source of help is .21 on a 0-to-1 scale, small by the benchmarks above. Whether a 17-point gap in counseling use matters is a question about student services, not statistics. Benchmarks ↑ |
| 95% CI [.04, .34] | The population association is plausibly anywhere from barely there (.04) to past medium (.34). The interval is wide: 200 students pin V down only roughly. |
| χ²(2, N = 200) = 8.61 | Each cell's squared gap from its expected count, divided by that count and added over the six cells, comes to 8.61. It has no units: it isn't a count of people. If the two were unrelated, χ² would average 2 and pass 5.99 only 5% of the time. The 2 is the degrees of freedom, (r − 1)(c − 1) (Fisher, 1922), and N = 200 is the number of students. |
| p = .014 | If year and first source of help were unrelated, counts at least this far from expected would turn up about 1.4% of the time. That's below .05, so the association is statistically significant. Benchmarks ↑ |
Calculations
The Running Example
The help-seeking study: 200 undergraduates (80 first-years, 120 upper-years), each choosing one first source of help: friends or family, counseling, or no one.
- Null Hypothesis:
- Year and first source of help are independent
- Alternative:
- Year and first source of help are associated
Independent means the three help sources would be chosen in the same proportions by first-years and upper-years in the population. The alternative doesn't say which sources differ or in what direction.
What Your Data Should Look Like
One row per person, with one column for each categorical variable. Software builds the table of counts for you; never enter percentages or averages.
Below are our first rows and the 2 × 3 table that all 200 rows become. Each cell shows the observed count, the row's percentage and the count expected if year and help source were unrelated (E).
| participant | year | first_help |
|---|---|---|
| 1 | first-year | no_one |
| 2 | upper-year | friends_family |
| 3 | upper-year | friends_family |
| 4 | first-year | no_one |
| 5 | upper-year | friends_family |
| 6 | first-year | no_one |
| … 194 more rows | ||
| Friends or Family | Counseling | No One | Total | |
|---|---|---|---|---|
| First-Year | 48 (60%, E = 44) | 8 (10%, E = 16) | 24 (30%, E = 20) | 80 |
| Upper-Year | 62 (52%, E = 66) | 32 (27%, E = 24) | 26 (22%, E = 30) | 120 |
| Total | 110 | 40 | 50 | 200 |
Counts, with each row's percentages, and the count expected if the two were unrelated (E).
Simulated data.
Napkin Math
The whole test, one idea at a time. Under each heading, a line says why the step matters, and the green note reads the equation in plain English with the help-seeking study's numbers. Collapse all to read just the headings: they tell the whole story on their own.
-
1. Count How Many Students Fall into Each Combination
Everything starts with a table of counts, one cell for each combination of year and help source. The contingency table is the data this test works on.
n = \sum O_{ij} =48 + 8 + 24 + 62 + 32 + 26= 200 students
O_{ij} is the count in row i, column j. First-years: 48 friends or family, 8 counseling, 24 no one (80 in all). Upper-years: 62, 32 and 26 (120 in all). Together that's 200 students.
-
2. Compare How Often Each Year Chose Each Source
The years are different sizes, so compare shares, not counts. This is the observed result, the pattern the test will judge.
\dfrac{O_{ij}}{R_i} =\dfrac{8}{80} \;\text{vs.}\; \dfrac{32}{120}\text{first-years, counseling} = .10
\text{upper-years, counseling} = .27
R_i is a row's total. 8 of our 80 first-years chose counseling, .10, against 32 of 120 upper-years, .27. First-years leaned more toward friends or family (.60 vs. .52) and no one (.30 vs. .22).
-
3. Work Out What Each Cell Would Hold If Year and Help Source Were Unrelated
The test asks two questions: if first-years and upper-years chose help sources in the same proportions, what would the table look like? And how different is our actual table from that? For the first, both years would split across the sources in the same proportions as everyone combined. Those expected counts are the yardstick for the observed ones.
E_{ij} = \dfrac{R_i \times C_j}{n} =\dfrac{80 \times 40}{200}= 16.0 students
E_{\text{first, friends}} = 44.0
E_{\text{first, no one}} = 20.0
E_{\text{upper, friends}} = 66.0
E_{\text{upper, counseling}} = 24.0
E_{\text{upper, no one}} = 30.0
R_i is the row total, our 80 first-years, and C_j the column total, the 40 students who chose counseling. 80 × 40 ÷ 200 = 16: if the two were unrelated, 16 first-years would have chosen counseling, not 8. That's also the smallest expected count, well above 5.
-
4. Measure How Far Each Cell Strays from Its Expected Count
Square each gap so shortfalls and surpluses both count, then divide by what was expected, so a gap of 8 counts for more in a cell expecting 16 than in one expecting 66. That way each gap is judged against what was expected, instead of every 8-person gap being treated as equally surprising.
\dfrac{(O_{ij} - E_{ij})^2}{E_{ij}} =\dfrac{(8 - 16)^2}{16}\text{first, friends} = 0.364
\text{first, counseling} = 4.000
\text{first, no one} = 0.800
\text{upper, friends} = 0.242
\text{upper, counseling} = 2.667
\text{upper, no one} = 0.533
For first-years and counseling, O is 8 and E is 16. (8 − 16)² ÷ 16 = 64 ÷ 16 = 4.000. The two counseling cells are by far the largest; the friends-or-family cells, only 4 off in a big column, barely register.
-
5. Add Up the Mismatches across the Whole Table
One number for the whole table. Even if year and help source were unrelated, chance would leave some gaps, so χ² wouldn't be 0. How big it tends to get by chance alone is what the chi-square distribution shows.
\chi^2 = \sum \dfrac{(O_{ij} - E_{ij})^2}{E_{ij}} =0.364 + 4.000 + 0.800 + 0.242 + 2.667 + 0.533= 8.61
\sum means “add them all up” over the six cells. 0.364 + 4.000 + 0.800 + 0.242 + 2.667 + 0.533 = 8.61, and about three quarters of it comes from the two counseling cells.
-
6. Account for How Much Information the Table Had
Once the row and column totals are fixed, only some cells are free to vary; the rest are filled in by subtraction. df picks the chi-square distribution to judge the total against: the spread of χ² values that tables like this produce by chance when the two variables are unrelated. Its average is its df, so with 2 df, 8.61 sits far out in the right tail.
df = (r - 1)(c - 1) =(2 - 1)(3 - 1)= 2
r is our 2 years and c our 3 help sources. (2 − 1) × (3 − 1) = 2: pick the first-year counts for two sources and every other cell follows from the totals.
-
7. Measure How Strong the Association Is
χ² grows with sample size; V doesn't. Dividing by n and by the smaller side of the table, less one, puts the association on a 0-to-1 scale that tops out at 1 in any table.
V = \sqrt{\dfrac{\chi^2}{n\,(k - 1)}} =\sqrt{\dfrac{8.61}{200 \times 1}}= .21
95\%\ \text{CI} = [.04, .34]
n is our 200 students and k is the smaller side of the table, 2 rows, so k - 1 = 1. 8.61 ÷ 200 = .043, and its square root is .21. Inverting the noncentral chi-square, as the Cramér's V page shows step by step, gives a 95% interval of .04 to .34.
-
8. Ask How Surprising the Result Would Be If the Two Were Unrelated
Finally, ask how often a table would stray at least this far from its expected counts if year and help source were unrelated. That's the share of the chi-square curve at or beyond 8.61. With 2 df, the top 5% of the curve starts at 5.99, the usual cutoff.
p = P(\chi^2_{df} \ge \chi^2) =P(\chi^2_{2} \ge 8.61)= .014
\chi^2_{2} is a chi-square distribution with 2 df. The chance it lands at or above 8.61 is .014, about 1 in 74. (With 2 df there's a shortcut: p = e^{-\chi^2/2} = e^{-4.30}.)
-
9. Find the Cells Where the Association Lies
A significant χ² says the two variables are associated, not where. Turning each cell's gap into a z-score shows which cells hold more or fewer people than independence predicts.
z_{ij} = \dfrac{O_{ij} - E_{ij}}{\sqrt{E_{ij}\,(1 - R_i/n)(1 - C_j/n)}} =\dfrac{8 - 16}{\sqrt{16\,(1 - 80/200)(1 - 40/200)}}= −2.89
z_{\text{upper, counseling}} = 2.89
z_{\text{first, no one}} = 1.33
z_{\text{first, friends}} = 1.16
For first-years and counseling, the gap of −8 divided by its standard error, \sqrt{16 \times .60 \times .80} = 2.77, is −2.89. Beyond ±1.96, and even beyond the stricter ±2.64 for six cells: first-years chose counseling less often than independence predicts. Upper-years mirror it at +2.89: with only two rows, each column's two residuals are equal and opposite, so the two counseling cells are the same difference between the years, seen from both sides, not two separate findings.
Run It in Software
The code below reads the running example's data from data.csv; save it in the same folder you run the code from.
SPSS (version 30, as documented by IBM)
- Analyze → Descriptive Statistics → Crosstabs.
- Move
yearinto Row(s) andfirst_helpinto Column(s). - Click Statistics, check Chi-square and Phi and Cramer's V, and click Continue.
- Click Cells, check Observed and Expected counts, Row percentages and Adjusted standardized residuals, and click Continue, then OK.
Or paste this syntax:
CROSSTABS /TABLES=year BY first_help /FORMAT=AVALUE TABLES /STATISTICS=CHISQ PHI /CELLS=COUNT EXPECTED ROW ASRESID /COUNT ROUND CELL.
Reading the Output
Software sorts the categories alphabetically, so counseling comes first here, not friends or family.
Read the tables in this order: the row percentages in the Crosstabulation (the pattern), then Cramer's V (the size), then the Pearson Chi-Square row (the test), and finally the Adjusted Residuals (where it lies). Check the footnote under Chi-Square Tests for the smallest expected count.
year * first_help Crosstabulation (abridged)
| counseling | friends_family | no_one | Total | ||
|---|---|---|---|---|---|
| first-year | Count | 8 | 48 | 24 | 80 |
| Expected Count | 16.0 | 44.0 | 20.0 | 80.0 | |
| % within year | 10.0% | 60.0% | 30.0% | 100.0% | |
| Adjusted Residual | -2.9 | 1.2 | 1.3 | ||
| upper-year | Count | 32 | 62 | 26 | 120 |
| Expected Count | 24.0 | 66.0 | 30.0 | 120.0 | |
| % within year | 26.7% | 51.7% | 21.7% | 100.0% | |
| Adjusted Residual | 2.9 | -1.2 | -1.3 |
Chi-Square Tests
| Value | df | Asymptotic Significance (2-sided) | |
|---|---|---|---|
| Pearson Chi-Square | 8.606ᵃ | 2 | .014 |
| Likelihood Ratio | 9.232 | 2 | .010 |
| N of Valid Cases | 200 |
a. 0 cells (0.0%) have expected count less than 5. The minimum expected count is 16.00.
Symmetric Measures
| Value | Approximate Significance | ||
|---|---|---|---|
| Nominal by Nominal | Phi | .207 | .014 |
| Cramer's V | .207 | .014 | |
| N of Valid Cases | 200 |
Use Cramer's V. SPSS prints Phi for every table, but it only has a 0-to-1 meaning for 2 × 2 tables (here, with two rows, the two match). “Asymptotic Significance (2-sided)” is the ordinary p: χ² has only one tail, and “2-sided” just means it catches differences in any direction. For a 2 × 2 table SPSS adds a Continuity Correction row and Fisher's Exact Test. Report the Pearson Chi-Square row, not the Continuity Correction. Linear-by-Linear Association tests a straight-line trend across ordered categories, so for a table like ours it isn't a substitute for Pearson's χ². Only in a 2 × 2 table does it equal the “N − 1” chi-square, χ² × (N − 1) ÷ N, one of several small-sample options; if you use it, say so.
Output laid out from this page's computed results, not captured from a live run; labels and layout may differ slightly in your version.
R
helpseek <- read.csv("data.csv")
tab <- table(helpseek$year, helpseek$first_help)
test <- chisq.test(tab) # no Yates correction is applied to tables bigger than 2 × 2
test
test$expected
round(test$stdres, 2) # adjusted residuals
# Cramér's V with its 95% CI
library(DescTools)
CramerV(tab, conf.level = 0.95)
Pearson's Chi-squared test
data: tab
X-squared = 8.6061, df = 2, p-value = 0.01353
counseling friends_family no_one
first-year 16 44 20
upper-year 24 66 30
counseling friends_family no_one
first-year -2.89 1.16 1.33
upper-year 2.89 -1.16 -1.33
Cramer V lwr.ci upr.ci
0.20743747 0.04228339 0.33652444
R's $stdres are the adjusted residuals (SPSS's “Adjusted Residual”); $residuals are the smaller Pearson residuals. For a 2 × 2 table, chisq.test applies Yates' correction unless you add correct = FALSE. CramerV's default interval inverts the noncentral chi-square, as on this page; its search stops at a looser tolerance, so the lower limit differs from ours in the fifth decimal.
Output from a real run: R 4.6.1 (DescTools 0.99.60), run 2026-10-03.
Python
import pandas as pd
from scipy import stats
from scipy.stats.contingency import association
helpseek = pd.read_csv("data.csv")
table = pd.crosstab(helpseek["year"], helpseek["first_help"])
result = stats.chi2_contingency(table, correction=False)
print(f"chi2 = {result.statistic:.4f}, df = {result.dof}, p = {result.pvalue:.5f}")
print(result.expected_freq)
print(f"Cramer's V = {association(table, method='cramer'):.4f}")
chi2 = 8.6061, df = 2, p = 0.01353
[[16. 44. 20.]
[24. 66. 30.]]
Cramer's V = 0.2074
The expected counts are in alphabetical order too: counseling, friends or family, no one.
correction=False turns off Yates' correction. scipy only applies it to tables with 1 degree of freedom (2 × 2), so it makes no difference here, but saying so keeps the code honest when the table shrinks.
Output from a real run: Python 3.9.6 (scipy 1.12.0, pandas 2.3.3), run 2026-10-03.
Impact
Effect Size
Cramér's (1946) V rescales χ² so it is standardized for sample size and always runs from 0 (the two variables are unrelated) to 1 (knowing the variable with more categories tells you the other exactly), whatever the size of the table. Its conventional benchmarks still depend on the table's shape (below), so compare Vs from tables of the same shape, or convert them to w.
Year in school and first source of help are associated at V = .21, small by convention. The table has two rows, so k − 1 = 1 and Cohen's cutoffs of .10, .30 and .50 apply as they are. The 95% interval, .04 to .34, runs from barely there to past medium.
The benchmarks depend on the table. Cohen (1992) gave .10, .30 and .50 for w, the effect size behind power calculations, and V = w ÷ √(k − 1). In a 3 × 3 table (k − 1 = 2), the same conventions become .07, .21 and .35, so a V of .21 there would already be medium. In a 2 × 2 table, V is the size of the phi coefficient, φ.
The interval comes from the noncentral chi-square distribution: find the population associations that would put our χ² at the 97.5th and 2.5th percentiles of their sampling curves, and convert each to V. The Cramér's V page works through it. The interval and the test don't share an exact cutoff: the test puts all 5% in the upper tail, while the interval leaves 2.5% at each end, so its lower limit clears zero only when p is below about .025. That page explains why. V also runs a little high in small samples and big tables, because χ² averages about its df even with no association; Bergsma's (2013) bias correction removes most of that, and is worth reporting when the two differ. Here it gives .18 against .21: still small.
For a 2 × 2 table, an odds ratio with its interval is often easier to read than φ.
V has no sign and doesn't say which categories go together. For that, go back to the percentages: 10% of first-years turned first to counseling, against 27% of upper-years, a gap of 17 percentage points. Benchmarks are conventions. Whether that gap is worth an outreach campaign for first-years is a question about student services, not statistics.
Benchmarks Cramér's V at a Glance
Statistical Significance
p answers one narrow question: if the two variables were unrelated in the population, how often would a table's counts stray at least this far from the expected counts? χ² only grows as counts drift from expected, so p is the upper tail, but it catches a difference in any cell and any direction. Below the conventional α = .05, the association is called statistically significant.
If year and first source of help were truly unrelated, 200 students would produce counts at least this far from expected about 1.4% of the time, roughly 1 in 74. That's below .05, so it's significant.
p is not the probability that the variables are unrelated, and it says nothing about how strong the association is: with 2,000 students, a V of .07 would be significant too. It also doesn't say which cells differ. That's what V, the percentages and the adjusted residuals are for.
Benchmarks p at a Glance
Strength of evidence against the null hypothesis.
Power and Sample Size
Power is the chance that a study detects an effect that's really there. A real association doesn't guarantee a significant result: with too few people, it can easily come out non-significant. For a chi-square test, power depends on how strong the true association is, how many people you have, α, and the degrees of freedom: a bigger table spreads the same sample over more cells and needs more people.
Power for chi-square tests is planned with Cohen's w, which here is the same as Cramér's V because the table has two rows. Cohen's (1992) conventions are .10, .30 and .50 for small, medium and large.
Suppose, before collecting any data, we expected a medium association, w = .30, in a 2 × 3 table (df = 2). About 108 students would give 80% power. If instead we'd decided that the smallest association worth finding was w = .20, 200 students would give about 72% power: a real association that size would be missed more than a quarter of the time.
Researchers usually aim for at least 80% power. Plan it before collecting data, using the smallest association that would matter. Small associations need big samples: 80% power for w = .10 with 2 df takes about 964 people (Cohen, 1992, Table 2). The calculator below uses our table's df = 2; for a different table, G*Power's “Goodness-of-fit tests: Contingency tables” option takes w, α and df. Try it:
Other Info
Follow-Up and Post Hoc Tests
Post Hoc Tests
- Adjusted Residuals: Use when your question calls for knowing which cells drive an association, typically after a significant χ² on a table bigger than 2 × 2: they find the cells with more or fewer people than independence predicts. They show where the table departs from independence, not that one group differs significantly from every other. Compare each with ±1.96, or with a Bonferroni-adjusted cutoff when there are many cells: for our 6 cells, ±2.64. It's step 9 of the Napkin Math: only the counseling cells clear it (±2.89). With two rows, the two years' residuals in each column are mirror images, so those two cells are one difference seen from both sides, not two findings.
- Bonferroni Correction: Use when you follow up by splitting the table into smaller 2 × 2 tables (say, counseling versus friends or family, by year), dividing α by the number of tables you test.
References
- Pearson, K. (1900). On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 50(302), 157–175. https://doi.org/10.1080/14786440009463897
- Fisher, R. A. (1922). On the interpretation of χ² from contingency tables, and the calculation of P. Journal of the Royal Statistical Society, 85(1), 87–94. https://doi.org/10.1111/j.2397-2335.1922.tb00768.x
-
Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159.
https://doi.org/10.1037/0033-2909.112.1.155
“for contingency tables, df = (a − 1)(b − 1), where a and b are the number of levels in the two variables.” (p. 157)
-
McHugh, M. L. (2013). The chi-square test of independence. Biochemia Medica, 23(2), 143–149.
https://doi.org/10.11613/BM.2013.018
“Each subject may contribute data to one and only one cell in the χ2.” (Assumptions of the Chi-square)
“The value of the cell expecteds should be 5 or more in at least 80% of the cells, and no cell should have an expected of less than one” (Assumptions of the Chi-square)
-
Sharpe, D. (2015). Your chi-square test is statistically significant: Now what? Practical Assessment, Research & Evaluation, 20(8), 1–10.
https://doi.org/10.7275/tbfa-x148
“MacDonald and Gardner (2000) suggest a Bonferroni adjustment to the z critical of 1.96 (from which the +/- 2 criteria is derived) if the number of cells in the contingency table is large.” (p. 3)
- Cramér, H. (1946). Mathematical methods of statistics. Princeton University Press.
- Bergsma, W. (2013). A bias-correction for Cramér's V and Tschuprow's T. Journal of the Korean Statistical Society, 42(3), 323–328. https://doi.org/10.1016/j.jkss.2012.10.002
-
Muff, S., Nilsen, E. B., O'Hara, R. B., & Nater, C. R. (2022). Rewriting results sections in the language of evidence. Trends in Ecology & Evolution, 37(3), 203–210.
https://doi.org/10.1016/j.tree.2021.10.009
“we suggest language of evidence that allows for a more nuanced approach to communicate scientific findings as a simple and intuitive alternative to statistical significance testing” (Abstract)
- Campbell, I. (2007). Chi-squared and Fisher–Irwin tests of two-by-two tables with small sample recommendations. Statistics in Medicine, 26(19), 3661–3675. https://doi.org/10.1002/sim.2832
Spot an error? Something still confusing? Tell us at support@statsforpsychology.com.