Stats for

Categorical Data and Proportions

McNemar's Test

McNemar's test checks whether a yes/no proportion changed in the same people.

McNemar's test takes a yes/no outcome measured twice in the same people and tests whether the proportion answering yes changed between the two measurements.

Example

Does a lecture on how memory works change how many students believe memory works like a video camera? One hundred intro-psych students answered yes or no to “Human memory works like a video camera” before and after the lecture, so each student has two answers.

Same People, Two Conditions + Binary Outcome → McNemar's Test
Tests
Difference between two paired population proportions
Observations
Related
Parametric
No
Headline Statistic
\chi^2
Effect Size
Paired Odds Ratio
Groups
2

The Basics

When to Use It

Use it when the same people answer one yes/no question twice, before and after something or under two conditions, and you want to know whether the share answering yes changed. Pairs of closely matched people work the same way. If the answer has three or more options (such as yes, unsure, no), look for the McNemar–Bowker or Stuart–Maxwell test instead; this site doesn't cover them yet. For example:

  • Does a lecture on how memory works change how many students believe memory works like a video camera?
  • Do fewer clients meet the PHQ-9 cutoff for depression after eight weeks of therapy than before it?
  • Do more participants recognize a face from a lineup when it's shown from the front than when it's shown in profile?

Vocabulary You Need

Definition Contingency Table

A table that cross-classifies people on two categorical variables, one along the rows and one along the columns, so each cell counts the people with that combination. The row and column totals (the margins) show how each variable is spread on its own; the cells show how the two go together. Tables are described by their size, rows × columns, such as a 2 × 3 table. Glossary →

Definition Discordant Pair

In paired yes/no data, a person (or matched pair) whose two answers differ, yes then no or no then yes. People whose answers match, the concordant pairs, say nothing about whether the proportion changed, so tests of change, like McNemar's test, use only the discordant pairs. Glossary →

Definition Odds Ratio (\text{OR})

One group's odds of an outcome divided by another group's. 1 means the odds are the same, 2 means the first group's odds are twice as high, and 0.5 means half as high; 0.5 and 2 are equally strong, in opposite directions. It's the usual effect size for a 2 × 2 table of two separate groups and what logistic regression reports. When the same people answer twice, as in McNemar's test, the paired odds ratio is different: it compares the people who switched one way with those who switched the other. It compares odds, not chances: when an outcome is common, the odds ratio is further from 1 than the ratio of the two proportions. A single zero cell, with the other counts positive, makes it 0 or infinite; more than one zero can leave it undefined. Glossary →

Assumptions and Inference

1. Independent Pairs (for the p-Value and Interval) Different people (or matched pairs) are independent of each other. The two answers within a pair are meant to be related.
What It Means
Each person's data comes from a separate, unconnected source, so knowing one person's scores tells you nothing about anyone else's. In designs that measure people more than once, their own scores are meant to be related; it's different people who must be independent. Methods built for clustered data, such as multilevel models and GEE, also allow members of the same group to be related, as long as the analysis names the grouping; then it's the groups that must be independent.
Applies To
Observations
How to Check
By design, not by a test. Did each person contribute their own data, with no clusters such as roommates, couples, therapy groups or classrooms whose members influence each other? If there are clusters, does the analysis model every one of them?
How Much It Matters
A lot. If people's answers influence each other, as when friends sit together and talk through the question, the switches aren't separate pieces of evidence and p comes out too small.

What to Do

  • Alternative Test Generalized Estimating Equations (GEE). A GEE logistic model can account for clusters such as classrooms while still comparing the two occasions.
  • Design Change Have people answer on their own, and collect one pair of answers per person.
2. Enough People Who Changed (for the Chi-Square p-Value) The chi-square p-value is a large-sample approximation built on the people who switched. The exact and mid-p versions need no minimum; if nobody switches, there's no directional difference to test.
What It Means
McNemar's test uses only the people whose yes/no answer changed. Its chi-square p-value is an approximation that works best when there are more than a handful of them.
Applies To
Outcome
How to Check
Count the two discordant cells, b and c, in the before-by-after table. If there are only a few changers in all, report the mid-p version alongside the chi-square test.
How Much It Matters
Less than older textbooks suggest. The uncorrected test's p stays close to accurate even with fairly few people who changed (Fagerland et al., 2013). With only a handful of changers, any version of the test has little power, and the chi-square approximation is at its roughest.

What to Do

  • Correction Use the mid-p version of the exact test, which Fagerland et al. (2013) recommend for any sample size.
  • Design Change Recruit more people. Only those who change carry information, so plan for how many you expect to change.

Limitations

This Is Not Your Test If…

What It Doesn't Tell You

  • Why. With no control group, the drop could come from the lecture, or from students guessing what the instructor wanted to hear the second time.
  • Whether it matters. Significance doesn't say whether 12 fewer believers in 100 is worth the time, even if the lecture caused the drop.
  • Whether individual people changed their answers. The test asks whether more people switched one way than the other. Many people could change their answers while the overall proportion stays the same, if the two directions balance: 40 switching each way out of 100 would give no net change at all. Report the switching counts to show how much individual change there was.
  • Who else. The result describes intro-psych students like these 100, and only as well as one yes/no item captures what they believe.
  • That there's no change, if p had been above .05. A non-significant result would have meant the data were inconclusive, not that minds didn't change.

Common Mistakes

  • Don'tRun an ordinary chi-square test on the before-by-after table.

    InsteadUse McNemar's test. The chi-square test asks whether before and after answers are related, which they almost always are, not whether the proportion changed.

  • Don'tTreat the before and after answers as two separate groups.

    InsteadKeep each person's two answers together. Stacking them counts the same people twice and ignores the pairing.

  • Don'tReport “McNemar's test” without saying which version.

    InsteadSay whether you used the uncorrected, continuity-corrected, exact or mid-p version; their p-values differ.

  • Don'tReport only the paired odds ratio.

    InsteadReport the change in the proportion too. The odds ratio ignores everyone whose answer didn't change.

  • Don'tRead a bigger χ² as a stronger effect.

    InsteadReport the change in the proportion with its interval, and the paired odds ratio b ÷ c. χ² grows with the number of switchers even when the split between them stays the same.

  • Don'tRead an odds ratio of 3 as “three times as likely.”

    InsteadSay how many people switched one way for each one who switched the other. The paired odds ratio isn't the odds of yes before compared with after, so give the before and after percentages separately.

  • Don'tRead a p-value as the chance that the null hypothesis is true.

    InsteadRead it as: if the null were true, how often data at least this extreme would turn up.

Result Decoder

Significance and Effect Size Guidelines

Benchmarks Significance (p Value)
  • p < .001

    Very strong evidence against the null hypothesis.

  • .001 ≤ p < .01

    Strong evidence against the null hypothesis.

  • .01 ≤ p < .05

    Statistically significant at α = .05. Moderate evidence against the null hypothesis.

  • .05 ≤ p < .10

    Not significant. Weak evidence; sometimes called a “trend.” Treat it as inconclusive.

  • p ≥ .10

    Not significant. Little evidence against the null, which is not the same as evidence of no effect.

These evidence labels are one common convention for describing p in graded language rather than as significant or not; the boundaries are rough guides, not rules, and other authors draw them differently. Muff et al. (2022) argue for this kind of language of evidence over a single cutoff. α is chosen before the analysis, and p never says how big or important an effect is.

More about statistical significance ↓

Benchmarks Effect Size (Paired Odds Ratio)
  • OR = 1 Balanced Switching

    Equal numbers switched in each direction.

  • OR > 1 More One Way

    More people switched in the direction in the numerator (here, yes to no).

  • 0 < OR < 1 More the Other Way

    Fewer switched in that direction. Flip the comparison and it becomes 1 ÷ OR: 2.50 and 0.40 are the same result.

  • Further from 1, multiplicatively More Unbalanced Switching

    Values like 4 and 0.25 are equally unbalanced, in opposite directions. It doesn't necessarily mean a bigger change in the overall proportion.

There are no generally accepted small, medium and large benchmarks for the paired odds ratio. It describes the balance between the two directions of change, not how much the overall proportion changed. Report the two switching counts and the percentage-point change alongside it.

More about effect size ↓

How It's Reported

Equation Form

Example

p̂after − p̂before = −.12, 95% CI [−.22, −.02], OR = 2.50, χ²(1, N = 100) = 5.14, p = .023

Full Text (APA Style)

Example

Before the lecture, 60% of students agreed that memory works like a video camera; afterward, 48% did, a change of −.12 in the proportion agreeing, 95% CI [−.22, −.02] (Wald interval). Of the 28 students whose answer changed, 20 dropped the belief and 8 took it up, a paired odds ratio of 2.50. McNemar's test (without continuity correction) showed the change was statistically significant, χ²(1, N = 100) = 5.14, p = .023.

  • Give both proportions (or percentages) and the change, so readers see the size before the test.
  • Say how many people changed in each direction. Those two counts are the whole test.
  • Name the version of McNemar's test. Here the continuity-corrected version gives p = .038 and the exact version p = .036.
  • Name the interval method. The Wald interval here is the one you can do by hand; for published work, Fagerland et al. (2014) recommend the asymptotic score interval, or one of the two simple closed-form intervals they describe, for the difference between paired proportions, and a transformed Wilson score interval for the odds ratio. The choice matters most when few people switched.
  • Italicize Latin symbols (p̂, OR, N, p) but not Greek ones (χ²), and report exact p to three decimals.

Reading Each Piece

The equation form above, piece by piece, in this order: how big, how certain, then how surprising.

p̂after − p̂before = −.12 The share of students agreeing that memory works like a video camera fell by 12 percentage points, from 60% to 48%. With no comparison group, that drop could come from the lecture or from answering the same question twice and guessing what the instructor wanted to hear.
OR = 2.50 Among students who changed their answer, two and a half dropped the belief for every one who took it up. There are no accepted benchmarks for a ratio of switchers, so we don't put a label on it; the 12-point drop and its interval say more, and whether that matters is a teaching question, not a statistical one. Benchmarks ↑
95% CI [−.22, −.02] The true drop is plausibly anywhere from about 2 to 22 percentage points. The whole range is below zero, but only just.
χ²(1, N = 100) = 5.14 How far the 20-versus-8 split is from an even 14 and 14; the bigger, the further. The 1 is the degrees of freedom, and N = 100 is the number of students.
p = .023 If the share agreeing were the same before and after, switches at least this lopsided, in either direction, would turn up about 2% of the time. That's below .05, so the result is statistically significant. Benchmarks ↑

Calculations

The Running Example

Example

The memory-myth study: 100 intro-psych students, each answering the same yes/no question about memory before and after a lecture on how memory works.

Null Hypothesis:
\pi_{\text{before}} = \pi_{\text{after}}
Alternative (Two-Sided):
\pi_{\text{before}} \neq \pi_{\text{after}}

\pi (“pi”) is the population proportion, the share of all students like these who would agree, before and after the lecture. The sample's .60 and .48 estimate them. Saying the two are equal is the same as saying that, among students who would change their answer, changing either way is equally likely.

What Your Data Should Look Like

One row per person and two yes/no columns, one for each measurement, coded the same way (here 1 = yes, 0 = no). Software counts the rows into a 2 × 2 table of first answer by second answer; anyone missing one of the two answers drops out. If people with particular answers are more likely to miss the second measurement, analyzing only the complete pairs can give a misleading picture of change.

Example
participantagree_beforeagree_after
111
200
301
410
511
610
… 94 more rows
Agree after: YesAgree after: NoTotal
Agree before: Yes402060
Agree before: No83240
Total4852100

Simulated data.

Download all 100 rows (data.csv)

Napkin Math

The whole test, one idea at a time. Under each heading, a line says why the step matters, and the green note reads the equation in plain English with the memory-myth study's numbers. Collapse all to read just the headings: they tell the whole story on their own.

  1. 1. Count How Each Student Answered Both Times

    Each student lands in one of four cells, depending on their answer before and after. The four counts are all the test needs.

    \begin{array}{l|cc} & \text{After: yes} & \text{After: no} \\ \hline \text{Before: yes} & a & b \\ \text{Before: no} & c & d \end{array} =\begin{array}{l|cc} & \text{yes} & \text{no} \\ \hline \text{yes} & 40 & 20 \\ \text{no} & 8 & 32 \end{array}

    a = 40 students

    b = 20 students

    c = 8 students

    d = 32 students

    a is the 40 students who agreed both times and d the 32 who disagreed both times. b is the 20 who agreed before but not after, and c the 8 who went the other way. 40 + 20 + 8 + 32 = 100.

  2. 2. Set Aside the Students Whose Answer Didn't Change

    Someone who said yes both times, or no both times, tells us nothing about which way minds changed. Only the students who switched carry information about that, so the test uses only them.

    b + c =20 + 8

    = 28 students

    b is the 20 who dropped the belief and c the 8 who took it up. 20 + 8 = 28 students changed their answer. The other 72 didn't; they still count in the before and after percentages and the estimated change, but not in the test statistic, which depends only on the 28 who switched.

  3. 3. Find How Much the Share Agreeing Changed

    Everything starts with the change in the proportion who agree. The students who didn't switch count the same on both sides, so the change comes down to the switchers.

    \hat{p}_2 - \hat{p}_1 = \dfrac{c - b}{n} =\dfrac{8 - 20}{100}

    = −.12

    \hat{p}_1 = .60

    \hat{p}_2 = .48

    \hat{p}_1 is the proportion before (the 60 of 100 who agreed) and \hat{p}_2 the proportion after (48 of 100); the report calls them \hat{p}_{\text{before}} and \hat{p}_{\text{after}}. The change is c - b = 8 − 20 = −12 students out of n = 100, the number of students (N in the report), or −.12, the same as .48 − .60.

  4. 4. Work Out What We'd Expect If the Share Agreeing Didn't Change

    If the share agreeing were the same before and after, the students who switched would be just as likely to switch one way as the other, so they'd split evenly.

    E = \dfrac{b + c}{2} =\dfrac{20 + 8}{2}

    = 14 students

    b + c is our 28 switchers. Split evenly, that's 28 ÷ 2 = 14 expected in each direction.

  5. 5. Compare the Switches We Saw with That Even Split

    The further the two switch counts are from each other, the less an even split fits. Chi-square measures that gap, scaled by how many people switched.

    \chi^2 = \dfrac{(b - c)^2}{b + c} =\dfrac{(20 - 8)^2}{20 + 8} = \dfrac{144}{28}

    = 5.14

    b - c = 20 − 8 = 12, and 12² = 144. Divided by the 28 switchers, that's 5.14. It's the same as adding (O - E)^2 / E for both cells, (20 − 14)² ÷ 14 + (8 − 14)² ÷ 14.

  6. 6. Account for How Much Information We Had

    Once we know how many people switched, only one of the two switch counts is free to vary. df tells us which chi-square distribution to judge the result against.

    df = 1

    = 1

    With 28 switchers, once b = 20 is known, c has to be 8. One count is free, so df = 1.

  7. 7. Measure How Lopsided the Switching Was

    Chi-square grows with sample size; the odds ratio doesn't. Dividing one switch count by the other says how many students changed one way for every one who changed the other way. That ratio is an odds, a switcher's odds of having dropped the belief rather than taken it up.

    OR = \dfrac{b}{c} =\dfrac{20}{8}

    = 2.50

    b is the 20 who dropped the belief and c the 8 who took it up. 20 ÷ 8 = 2.50 students dropped it for every one who took it up.

  8. 8. Give a Range of Plausible Values for the True Change

    Instead of a single best guess, give the range of true changes in the proportion that fit the data. Its width comes from the standard error. For a given number of students, it grows the more people switch and the more evenly they split; more students overall shrink it.

    (\hat{p}_2 - \hat{p}_1) \pm z^{*} \times \dfrac{\sqrt{(b + c) - (b - c)^2 / n}}{n} =-.12 \pm 1.96 \times \dfrac{\sqrt{28 - 144/100}}{100}

    = [−.22, −.02]

    z^{*} = 1.96 holds the middle 95% of a normal distribution. The standard error is √(28 − 1.44) ÷ 100 = .052, and 1.96 × .052 = .10, so the range is −.12 ± .10, from −.22 to −.02. This is the Wald interval, the one you can do by hand; with few switchers, other methods, such as a score-based interval, perform better. It's a separate calculation from the uncorrected test in step 9 (both are easy by hand), so near the cutoff the two can disagree.

  9. 9. Ask How Surprising the Result Would Be If There Were No Real Change

    Finally, ask how often the switchers would split at least this unevenly, in either direction, if switching either way were equally likely.

    p = P(\chi^2_{df} \ge \chi^2) =P(\chi^2_{1} \ge 5.14)

    = .023

    P(\chi^2_1 \ge 5.14) is the chance that a chi-square with 1 df lands at or above 5.14. That's .023, about 1 in 43. Squaring b - c makes a 20-to-8 split and an 8-to-20 split give the same χ², so this one upper tail already counts both directions.

Run It in Software

The code below reads the running example's data from data.csv; save it in the same folder you run the code from.

SPSS (version 30, as documented by IBM)
  1. Analyze → Descriptive Statistics → Crosstabs.
  2. Move agree_before into Row(s) and agree_after into Column(s).
  3. Click Statistics, check McNemar, and click Continue.
  4. Click Cells, check Total under Percentages (to get the before and after proportions), click Continue, then OK.

Or paste this syntax:

CROSSTABS
  /TABLES=agree_before BY agree_after
  /STATISTICS=MCNEMAR
  /CELLS=COUNT TOTAL
  /COUNT ROUND CELL.

Reading the Output

Read the Crosstabulation first: the row and column totals give the proportions before (60%) and after (48%), and the two off-diagonal cells are the switchers, 20 and 8. Work out the change (−.12) and the odds ratio (20 ÷ 8 = 2.50) from them; SPSS doesn't print either. Then read the McNemar Test row.

agree_before * agree_after Crosstabulation

agree_after 0 agree_after 1 Total
agree_before 0 Count 32 8 40
% of Total 32.0% 8.0% 40.0%
agree_before 1 Count 20 40 60
% of Total 20.0% 40.0% 60.0%
Total Count 52 48 100
% of Total 52.0% 48.0% 100.0%

Chi-Square Tests

Value Exact Sig. (2-sided)
McNemar Test .036ᵃ
N of Valid Cases 100

a. Binomial distribution used.

SPSS gives the exact binomial version of the test, not the chi-square this page computes, so its p (.036) is a little larger than ours (.023) and there's no χ² to report. Say which version you used.

Output laid out from this page's computed results, not captured from a live run; labels and layout may differ slightly in your version.

R
memory <- read.csv("data.csv")
tab <- table(before = memory$agree_before, after = memory$agree_after)
tab

mcnemar.test(tab, correct = FALSE)

# Paired odds ratio, b / c
b <- tab["1", "0"]
c <- tab["0", "1"]
b / c

# Mid-p version: the exact binomial p minus the chance of the observed split
binom.test(b, b + c)$p.value - dbinom(min(b, c), b + c, 0.5)
      after
before  0  1
     0 32  8
     1 20 40

	McNemar's Chi-squared test

data:  tab
McNemar's chi-squared = 5.1429, df = 1, p-value = 0.02334

[1] 2.5
[1] 0.02411954

The last line is the mid-p version, p = .024. correct = FALSE turns off the continuity correction, which R applies by default. With the default, R reports χ² = 4.32, p = .038.

Output from a real run: R 4.6.1, run 2026-10-10.

Python
import pandas as pd
from statsmodels.stats.contingency_tables import mcnemar

memory = pd.read_csv("data.csv")
table = pd.crosstab(memory["agree_before"], memory["agree_after"])
print(table)

result = mcnemar(table, exact=False, correction=False)
print(result.statistic, result.pvalue)

print(table.loc[1, 0] / table.loc[0, 1])   # paired odds ratio
agree_after    0   1
agree_before        
0             32   8
1             20  40
5.142857142857143 0.02334220201289086
2.5

statsmodels' mcnemar runs the exact binomial version unless you pass exact=False, and applies the continuity correction unless you pass correction=False. The defaults give p = .036.

Output from a real run: Python 3.9.6 (statsmodels 0.14.6, pandas 2.3.3), run 2026-10-03.

Impact

Effect Size

There are two ways to describe change in a before-and-after yes/no table, and it's worth reporting both:

  • Net change. The overall share saying yes fell from 60% to 48%, a drop of 12 percentage points. This counts everyone.
  • Direction of switching. Of the 28 students who changed, 20 switched from yes to no and 8 from no to yes. The paired odds ratio, 20 ÷ 8, sums this up.

They complement each other rather than standing in for each other: the first says how the group's proportion changed, the second how individual switches produced that change.

OR = \dfrac{b}{c} = \dfrac{20}{8} = 2.50

Of the 28 students who changed their answer, 20 dropped the belief and 8 took it up: two and a half one way for every one the other. Across all 100 students, that's a 12-point drop, from 60% to 48%.

Why call it an odds ratio? Among the students who switched, the odds of switching away from the belief rather than toward it are 20 to 8. It isn't the class's odds of agreeing before compared with after: those are 60 to 40 and 48 to 52, an odds ratio of (.60 ÷ .40) ÷ (.48 ÷ .52) = 1.63. And it isn't the usual odds ratio for a 2 × 2 table, ad ÷ bc = (40 × 32) ÷ (20 × 8) = 8.0, which measures how closely students' two answers match, not whether the share agreeing changed.

Say which direction is on top. Here it's yes-to-no over no-to-yes; turned around, it's 8 ÷ 20 = 0.40, the same result. An odds ratio of 1 means switching went equally often both ways. Because it ignores everyone who didn't switch, the same odds ratio can come with a big or a tiny change in the overall proportion: 20 against 8 switchers out of 100 students is a 12-point change, but out of 1,000 students it would be barely more than 1 point. That's why the change in the proportion belongs next to it.

The odds ratio's own 95% interval (a transformed Wilson score interval, which Fagerland et al., 2014, recommend) runs from about 1.12 to 5.56. It's wide because it rests on only 28 students.

There are no accepted small, medium and large benchmarks for the paired odds ratio; the ones sometimes borrowed (Chen et al., 2010) were worked out for two separate groups, a different quantity. Lean on the change in the proportion and its interval, and give the two switching counts. Whether a 12-point drop, if the lecture caused it, is worth an hour of class is a teaching judgment, not a statistical one.

Benchmarks Paired Odds Ratio at a Glance
Balanced Switching
OR = 1
More One Way
OR > 1
More the Other Way
0 < OR < 1
More Unbalanced Switching
Further from 1, multiplicatively

Full benchmarks and caveats ↑

Statistical Significance

p answers one narrow question: if switching either way were equally likely, how often would the people who switched split at least this unevenly, in either direction? Below the conventional α = .05, the result is called statistically significant.

If the share agreeing were the same before and after, 28 switchers would split at least as unevenly as 20 to 8 about 2.3% of the time, roughly 1 in 43. That's below .05, so it's significant.

Four ways to test the same switching pattern. All four versions test whether switching is equally likely in either direction. They differ in how they work out the probability:

  • Uncorrected: a chi-square approximation with no adjustment.
  • Continuity-corrected: adjusts the approximation for counts, usually making it more cautious.
  • Exact: works out the probabilities directly from the binomial distribution.
  • Mid-p: the exact calculation, adjusted to be less cautious.

This page uses the plain chi-square version, without a continuity correction; Fagerland et al. (2013) found it keeps close to the stated error rate even in fairly small samples, though it often exceeds that rate slightly. They recommend it if those small, frequent excesses are acceptable, and the mid-p version, which never exceeded the rate in their study, for any sample size. They advise against the continuity-corrected version, R's default, and the exact version, statsmodels' default: both are more cautious than they need to be.

For our 20 and 8 switchers: uncorrected p = .023, mid-p = .024, exact p = .036, and continuity-corrected p = .038. All are below .05 here, but they won't always agree, so decide which you'll use before you look.

The exact version treats the 28 switchers like 28 coin flips and asks how often a fair coin gives 8 or fewer heads, or 8 or fewer tails: p = .036. The mid-p version counts the observed 20-to-8 split only half, giving .024; in R it's binom.test(b, b + c)$p.value - dbinom(min(b, c), b + c, 0.5). The continuity correction subtracts 1 from |b - c| before squaring: (12 − 1)² ÷ 28 = 4.32, p = .038.

p is not the probability that the null hypothesis is true, it says nothing about how large or important the change is, and it can't say the lecture caused it. That's what the effect size, the interval and the design are for.

Benchmarks p at a Glance
Very Strong
p < .001
Strong
.001 ≤ p < .01
Significant
.01 ≤ p < .05
Weak
.05 ≤ p < .10
Little
p ≥ .10

Strength of evidence against the null hypothesis.

Full benchmarks and caveats ↑

Power and Sample Size

Power is the chance that a study detects an effect that's really there. A real effect doesn't guarantee a significant result: with too few people, it can easily come out non-significant. For McNemar's test, power depends on how many people change their answer and how lopsided those changes are, as well as on α. People whose answer stays the same add nothing, so a study where few people switch needs a lot of people.

Suppose, before collecting any data, we decided a 12-point drop was the smallest change worth detecting, and expected it to come from 20% of students dropping the belief and 8% taking it up. Computed exactly for the uncorrected test (summing over every table 100 students could produce), 100 students would come out significant only about 65% of the time. It would take about 147 students to reach 80%. The usual normal-approximation formula for paired proportions gives slightly lower figures, 62% and about 150, so say which method you used.

Researchers usually aim for at least 80% power. Plan it before collecting data, from how many people you expect to switch each way, using the smallest change that would matter. This site doesn't have a calculator for McNemar's test yet, so use dedicated power software for the calculation.

Other Info

Follow-Up and Post Hoc Tests

None needed. This test answers a single question, so there are no planned contrasts or post hoc tests to run.

References

  • McNemar, Q. (1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2), 153–157. https://doi.org/10.1007/BF02295996
  • Fagerland, M. W., Lydersen, S., & Laake, P. (2013). The McNemar test for binary matched-pairs data: Mid-p and asymptotic are better than exact conditional. BMC Medical Research Methodology, 13, 91. https://doi.org/10.1186/1471-2288-13-91
    “the asymptotic McNemar test (without CC) performs surprisingly well, even for quite small sample sizes” (Discussion)
  • Fagerland, M. W., Lydersen, S., & Laake, P. (2014). Recommended tests and confidence intervals for paired binomial proportions. Statistics in Medicine, 33(16), 2850–2875. https://doi.org/10.1002/sim.6148
    “For the difference between proportions, we recommend two simple confidence intervals with closed-form expressions and the asymptotic score interval.” (Abstract)
  • Muff, S., Nilsen, E. B., O'Hara, R. B., & Nater, C. R. (2022). Rewriting results sections in the language of evidence. Trends in Ecology & Evolution, 37(3), 203–210. https://doi.org/10.1016/j.tree.2021.10.009
    “we suggest language of evidence that allows for a more nuanced approach to communicate scientific findings as a simple and intuitive alternative to statistical significance testing” (Abstract)

Spot an error? Something still confusing? Tell us at support@statsforpsychology.com.