z-Test for Proportions: Interpretation and APA Reporting

The Basics

When to Use It

Use it when you have two separate groups of different people, one yes/no outcome, and you want to know whether the share answering yes differs between the groups. For example:

Does a peer-story email get more first-year students to book a counseling appointment?
Are clients who get text reminders less likely to miss their next therapy session than clients who don't?
Do children in a growth-mindset condition choose the harder puzzle more often than children in a control condition?

Vocabulary You Need

Definition Proportion (

\hat{p}

)

The number of cases in a category divided by the total, such as the share of people who answered yes. It runs from 0 (none) to 1 (all), and multiplying by 100 turns it into a percentage. A sample proportion, written p̂ (“p-hat”), estimates the population proportion. If a yes/no outcome is coded 1 and 0, its mean is the proportion of 1s. Glossary →

Definition Pooled Proportion (

\bar{p}

)

The total number of cases in a category divided by the total number of cases, across all the groups together. It's the best estimate of the one shared proportion if the groups don't really differ, which is why tests of equal proportions use it to work out how much chance variation to expect. Glossary →

Definition Standard Error (

SE

)

How much a statistic, such as a mean, a proportion or a difference between two of them, would vary from one sample to the next. SD asks how much individuals vary; SE asks how sure we are about the estimate. It shrinks as the sample grows. Glossary →

Definition z Statistic (

z

)

A difference divided by its standard error, like t, but compared with the standard normal distribution instead of a t distribution. Tests use z when the standard error comes from a known formula rather than from the sample's spread, as with proportions, or when samples are large. About 95% of z values fall between −1.96 and 1.96 when the null hypothesis is true. Glossary →

Assumptions and Inference

1. Independent Observations (for the p-Value and Interval) The people (or other units) in your study don't influence each other.

What It Means: Each person's data comes from a separate, unconnected source, so knowing one person's scores tells you nothing about anyone else's. In designs that measure people more than once, their own scores are meant to be related; it's different people who must be independent. Methods built for clustered data, such as multilevel models and GEE, also allow members of the same group to be related, as long as the analysis names the grouping; then it's the groups that must be independent.
Applies To: Observations
How to Check: By design, not by a test. Did each person contribute their own data, with no clusters such as roommates, couples, therapy groups or classrooms whose members influence each other? If there are clusters, does the analysis model every one of them?
How Much It Matters: A lot. Students who share a dorm, a class or a friend group can influence each other's choices, and anyone who got both messages breaks the two-group design. Either makes the standard error too small, so p comes out too small.

What to Do

Alternative Test Multilevel Logistic Regression. A multilevel logistic regression, which accounts for the clustering.
Design Change Make sure each person contributes one yes/no answer, to one group only. If the same people answered in both conditions, that's a paired design: see McNemar's test.

2. Large Enough Expected Counts (for the p-Value and Interval) The difference in proportions can be computed from any two groups, but the usual z-test p-value relies on a large-sample approximation that small expected counts can make unreliable.

What It Means: The p-value comes from a smooth curve (the chi-square curve, or the normal curve for a z-test), which only approximates the lumpy distribution of real counts. The approximation works when the expected count in each cell, not the observed count, is reasonably large, and breaks down when many cells expect only a handful of people.
Applies To: Outcome
How to Check: Look at the expected counts (row total × column total ÷ N), which most software prints; SPSS Crosstabs reports the smallest one under the Chi-Square Tests table. A common rule: at least 80% of cells expect 5 or more, and none expects fewer than 1 (McHugh, 2013). The rule is cautious: for 2 × 2 tables, Campbell (2007) found the “N − 1” chi-square (χ² × (N − 1) ÷ N, which SPSS prints as Linear-by-Linear Association) accurate whenever every expected count is at least 1.
How Much It Matters: Little here: the smallest of the four expected counts is about 31. It matters when a group is small or the outcome is rare. Then the normal curve fits the counts poorly and p can be off. The usual cutoff of 5 is a rough convention: for a 2 × 2 table, Campbell (2007) found the test holds up with expected counts as low as 1 after a small correction, and recommended Fisher's exact test below that; that's one well-supported recommendation, not a universal rule.

What to Do

Alternative Test Fisher's Exact Test. It computes p from the exact distribution of the counts, so it needs no approximation.
Correction Multiply z² by (N − 1)/N and compare it with a chi-square distribution with 1 df. Campbell (2007) recommends this whenever every expected count is at least 1.
Design Change Plan groups big enough to expect at least 5 of each outcome, especially when the outcome is rare.

Limitations

This Is Not Your Test If…

Your outcome is a score, not yes/no Independent-Samples t-Test →
You measured the same people twice, or matched pairs McNemar's Test →
Some expected counts are below 1 Fisher's Exact Test →
You have three or more groups or outcome categories Pearson Chi-Square Test →
You need to adjust for other variables Logistic Regression →
Your people come in clusters, like classrooms or clinics Multilevel Logistic Regression →

What It Doesn't Tell You

Why. Random assignment lets us credit the email rather than differences between students, but not which part of it worked: the stories, the tone or simply a longer message.
Whether it matters. Significance doesn't tell the counseling center whether 12 more bookings per 100 students justify the new email. And a booking is a meaningful step, but it isn't attendance, engagement in counseling or better mental health; the test says nothing about whether those students kept their appointments or felt better.
Who else. The result describes first-year students at this one campus, in one term; students at another school, or a later email in the same term, might respond differently.
That there's no difference, if p had been above .05. A non-significant result would have meant the data were inconclusive, not that the two emails work equally well.

Common Mistakes

Don'tSwitch to a one-sided p because the result went the way you hoped.

InsteadUse a one-sided test only if you specified the direction before collecting data and have a defensible reason for treating an effect in the opposite direction differently. Otherwise use a two-sided test, as here: an email that cut bookings would matter too.
Don'tCall a change from 20% to 32% a “12% increase.”

InsteadSay 12 percentage points. As a relative change it's a 60% increase, and readers need to know which you mean.
Don'tReport only percentages.

InsteadGive each group's count and size (40 of 125), so readers can rebuild the table.
Don'tUse the z-test when some expected counts are tiny.

InsteadCheck the smallest of the four expected counts first (row total × column total ÷ N). When they're small, consider Fisher's exact test or another small-sample method such as the “N − 1” chi-square. Campbell (2007) recommends N − 1 when every expected count is at least 1 and Fisher's test otherwise; that's a recommendation, not a universal cutoff, so choose the method in your analysis plan and report it.
Don'tRun separate z-tests for every pair when you have three or more groups.

InsteadRun a chi-square test on the whole table first, then compare specific pairs with a correction.
Don'tRead a p-value as the chance that the null hypothesis is true.

InsteadRead it as: if the null were true, how often data at least this extreme would turn up.

Result Decoder

Significance and Effect Size Guidelines

Benchmarks Significance (p Value)

p < .001
Very strong evidence against the null hypothesis.
.001 ≤ p < .01
Strong evidence against the null hypothesis.
.01 ≤ p < .05
Statistically significant at α = .05. Moderate evidence against the null hypothesis.
.05 ≤ p < .10
Not significant. Weak evidence; sometimes called a “trend.” Treat it as inconclusive.
p ≥ .10
Not significant. Little evidence against the null, which is not the same as evidence of no effect.

These evidence labels are one common convention for describing p in graded language rather than as significant or not; the boundaries are rough guides, not rules, and other authors draw them differently. Muff et al. (2022) argue for this kind of language of evidence over a single cutoff. α is chosen before the analysis, and p never says how big or important an effect is.

More about statistical significance ↓

Benchmarks Effect Size (Cohen's h)

|h| < 0.2 Below Small
Smaller than the conventional small benchmark. Whether an effect this size matters depends on the setting.
0.2 ≤ |h| < 0.5 Small
A relatively modest standardized difference, by convention.
0.5 ≤ |h| < 0.8 Medium
A moderate standardized difference, by convention.
|h| ≥ 0.8 Large
A relatively substantial standardized difference, by convention.

The sign shows direction; the benchmarks apply to its size. They are conventions (Cohen, 1988, 1992), not measures of practical or clinical importance. As illustrations only, 50% against 75% is h ≈ 0.52 and 50% against 88% is h ≈ 0.86; whether a difference is noticeable depends on the baseline rate, the sample and the setting.

More about effect size ↓

How It's Reported

Equation Form

Example

RD = .12, 95% CI [.01, .23], h = 0.28, z = 2.14, p = .033

Full Text (APA Style)

Example

More students booked a counseling appointment after the peer-story email (40 of 125, 32%) than after the standard email (24 of 120, 20%), a difference of 12 percentage points, RD = .12, 95% CI [.01, .23] (Newcombe's method), h = 0.28. A two-proportion z-test showed the difference was statistically significant, z = 2.14, p = .033.

Give each group's count and size as well as the percentage, so readers can rebuild the 2 × 2 table.
Say "percentage points" for the difference, and say which group was subtracted from which.
Name the interval method. Newcombe's (or Agresti and Caffo's) is recommended over the simple Wald interval, which has poor coverage with small groups or rare outcomes (Newcombe, 1998). Here the two barely differ.
If you report χ² instead of z, say whether you used the continuity correction; R's prop.test uses it by default.

Reading Each Piece

The equation form above, piece by piece, in this order: how big, how certain, then how surprising.

RD = .12	Twelve more students in every 100 booked an appointment after the peer-story email than after the standard one (32% vs 20%).
95% CI [.01, .23] (Newcombe)	The true difference is plausibly anywhere from about 1 to 23 percentage points in favor of the peer-story email. The whole range is above zero, but its low end is barely there.
h = 0.28	A small effect by the benchmarks above. Whether 12 more bookings per 100 students is worth the new email is a judgment about the counseling center, not statistics. Benchmarks ↑
z = 2.14	The difference is 2.14 standard errors above zero. z has no degrees of freedom; it's judged against the normal curve.
p = .033	If the two emails worked equally well, a gap at least this big, in either direction, would turn up about 3% of the time. That's below .05, so the result is statistically significant. Benchmarks ↑

Calculations

The Running Example

Example

The outreach study: 245 first-year students randomly sent a peer-story email (125) or the standard email (120), compared on whether they booked a counseling appointment within four weeks.

Null Hypothesis:: $\pi_1 = \pi_2$
Alternative (Two-Sided):: $\pi_1 \neq \pi_2$

$\pi_1$ and $\pi_2$ (“pi one” and “pi two”; here π is a population proportion, not 3.14) are the shares of all first-year students like ours who would book an appointment after the peer-story email ( $\pi_1$ ) or the standard email ( $\pi_2$ ). The samples' .32 and .20 estimate them.

What Your Data Should Look Like

One row per person, with a column saying which group they're in and a column coding the outcome 1 (yes) or 0 (no). Each person appears once, in one group only. Some software wants the counts instead: a 2 × 2 contingency table of group by outcome, or just each group's number of yeses and size.

Example

student	message	sought_help
1	peer	0
2	peer	0
3	standard	0
4	peer	0
5	peer	0
6	peer	1
… 239 more rows

	n	Yes	%
Peer	125	40	32.0%
Standard	120	24	20.0%

Simulated data.

Download all 245 rows (data.csv)

Napkin Math

The whole test, one idea at a time. Under each heading, a line says why the step matters, and the green note reads the equation in plain English with the outreach study's numbers. Collapse all to read just the headings: they tell the whole story on their own.

1. Find the Share of Each Group Who Booked

A yes/no outcome boils down to one number per group, the proportion who said yes.

$\hat{p} = \dfrac{x}{n}$ = $\dfrac{40}{125}, \quad \dfrac{24}{120}$

$\hat{p}_1$ = .32
$\hat{p}_2$ = .20

$x$ is the number who booked and $n$ is the group's size. 40 of the 125 peer-story students booked, so $\hat{p}_1$ = .32 (32%); 24 of the 120 standard-email students did, so $\hat{p}_2$ = .20 (20%).
2. Find the Difference between the Two Groups

Everything starts with the gap between the two proportions. A positive gap means the peer-story group booked more often.

$RD = \hat{p}_1 - \hat{p}_2$ = $.32 - .20$

= .12

$\hat{p}_1$ is the peer-story group's .32 and $\hat{p}_2$ is the standard group's .20. .32 − .20 = .12, or 12 percentage points.
3. Find the Overall Share, Pretending the Emails Made No Difference

The null hypothesis says both groups share one true proportion. Our best guess at it combines everyone (Fleiss et al., 2003).

$\bar{p} = \dfrac{x_1 + x_2}{n_1 + n_2}$ = $\dfrac{40 + 24}{125 + 120}$

= .26

$x_1 + x_2$ is everyone who booked, 40 + 24 = 64, and $n_1 + n_2$ is all 245 students. 64 ÷ 245 = .26, so about 26% booked overall.
4. Figure Out How Much the Difference Would Wobble by Chance

Even if the emails worked equally well, two groups would rarely book at exactly the same rate. The standard error says how big that chance gap typically is, if everyone shared the overall rate.

$SE_0 = \sqrt{\bar{p}(1 - \bar{p})\left(\dfrac{1}{n_1} + \dfrac{1}{n_2}\right)}$ = $\sqrt{.261 \times .739 \times \left(\dfrac{1}{125} + \dfrac{1}{120}\right)}$

= .056

$\bar{p}(1 - \bar{p})$ is how much a single yes/no answer varies. It's largest when the rate is near 50% and zero when everyone answers the same way. Dividing by each group's size shrinks it, because a proportion averages many answers, and averages vary less than single answers. Here $\bar{p}$ is the overall .261, so .261 × .739 = .193. $1/n_1 + 1/n_2$ = .0163. Multiply to get .00315, and its square root is .056, about 5.6 percentage points.
5. Compare the Difference with That Wobble

A gap only means something relative to that chance wobble. z says how many standard errors the gap is.

$z = \dfrac{\hat{p}_1 - \hat{p}_2}{SE_0}$ = $\dfrac{.12}{.056}$

= 2.14

Our .12 gap divided by the .056 $SE_0$ puts the peer-story group 2.14 standard errors above the standard group. With enough people in each group (the smallest expected count here is about 31), chance gaps like this follow a roughly normal curve, so z can be judged against it.
6. Measure How Big the Difference Actually Is

z grows with sample size; h doesn't. Converting each proportion to the arcsine scale gives a size that's about equally easy to detect whatever the baseline rate, which is why power tables use it.

$h = 2\arcsin\sqrt{\hat{p}_1} - 2\arcsin\sqrt{\hat{p}_2}$ = $2\arcsin\sqrt{.32} - 2\arcsin\sqrt{.20}$

= 0.28

$2\arcsin\sqrt{\hat{p}_1}$ turns the peer group's .32 into 1.203, and $2\arcsin\sqrt{\hat{p}_2}$ turns the standard group's .20 into 0.927 (in radians). 1.203 − 0.927 = 0.28.
7. Give a Range of Plausible Values for the True Difference

Instead of a single best guess, give the range of true differences that fit the data. The interval doesn't assume the emails work equally, so its standard error uses each group's own proportion.

$(\hat{p}_1 - \hat{p}_2) \pm z^{*} \sqrt{\dfrac{\hat{p}_1(1 - \hat{p}_1)}{n_1} + \dfrac{\hat{p}_2(1 - \hat{p}_2)}{n_2}}$ = $.12 \pm 1.96 \sqrt{\dfrac{.32 \times .68}{125} + \dfrac{.20 \times .80}{120}}$

= [.01, .23]

$SE$ = .0554
$\text{Newcombe}$ = [.010, .226]

$z^{*}$ = 1.96 holds the middle 95% of the normal curve. The square root is .0554, the gap's standard error without pooling. 1.96 × .0554 = .109, so the range is .12 ± .109, from .01 to .23. This is the Wald interval, the one you can do by hand. The interval to report is Newcombe's, which combines a score interval for each group's proportion and holds its coverage much better than Wald's with small groups or rare outcomes (Newcombe, 1998). It runs from .010 to .226, the same at two decimals. The test uses the pooled SE and the interval doesn't, so when p is near .05 the two can occasionally disagree about whether zero is plausible. Report both.
8. Ask How Surprising the Result Would Be If There Were No Real Difference

Finally, ask how often a gap at least this big, in either direction, would turn up if the two emails really worked equally well.

$p = 2 \times P(Z \ge |z|)$ = $2 \times P(Z \ge 2.14)$

= .033

$P(Z \ge 2.14)$ is the chance that a standard normal value lands at or above 2.14, about .016 (from a z table, or `1 - pnorm(2.14)` in R). Doubling it counts both directions, .033, about 1 in 31. A one-sided test, planned in advance for the peer-story email doing better, would use only the upper tail, .016.

Run It in Software

The code below reads the running example's data from data.csv; save it in the same folder you run the code from.

SPSS (version 30, as documented by IBM)

Analyze → Compare Means and Proportions → Independent-Samples Proportions.
Move sought_help into Test Variable(s) and message into Grouping Variable.
Under Define Groups, choose Value(s) and enter 'peer' and 'standard'. Under Define Success, choose Value(s) and enter 1.
Click Confidence Intervals and check Wald and Newcombe. Leave Tests at Wald H0, the default, and click OK.

Or paste this syntax:

PROPORTIONS
  /INDEPENDENTSAMPLES sought_help BY message SELECT=LEVEL('peer', 'standard')
    TESTTYPES=WALDH0 CITYPES=WALD NEWCOMBE
  /SUCCESSVALUE=LEVEL(1)
  /CRITERIA CILEVEL=95
  /MISSING SCOPE=ANALYSIS USERMISSING=EXCLUDE.

Reading the Output

SPSS prints three tables. Read the Difference in Proportions, then the Newcombe 95% confidence interval (the one this page reports; the Wald row is the Napkin Math version), then the Wald H0 Z and its Two-Sided p. Wald H0 is this page's test: it uses the pooled proportion for the standard error. The plain Wald test, if you ask for it, uses each group's own proportion and gives a slightly different z (2.164 here).

Independent-Samples Proportions Group Statistics

message	Successes	Trials	Proportion
peer	40	125	.320
standard	24	120	.200

Independent-Samples Proportions Confidence Intervals (sought_help)

Interval Type	Difference in Proportions	Asymptotic Standard Error	95% CI Lower	Upper
Wald	.120	.055	.011	.229
Newcombe	.120	.055	.010	.226

Independent-Samples Proportions Tests (sought_help)

Test Type	Difference in Proportions	Asymptotic Standard Error	Z	Two-Sided p
Wald H0	.120	.056	2.137	.033

SPSS doesn't print Cohen's h. Compute it as 2 × ARSIN(SQRT(.32)) − 2 × ARSIN(SQRT(.20)) = 0.275. Newcombe's interval, SPSS's default alongside Agresti-Caffo, is the better choice, especially with small groups or proportions near 0 or 1 (Newcombe, 1998); here it barely differs from Wald's. If you leave CITYPES out, SPSS prints Agresti-Caffo and Newcombe. The syntax was written from IBM's PROPORTIONS reference, not pasted from SPSS; the dialog's Paste button gives the exact form for your version.

Output laid out from this page's computed results, not captured from a live run; labels and layout may differ slightly in your version.

R

outreach <- read.csv("data.csv")
tab <- table(outreach$message, outreach$sought_help)
prop.test(tab[, c("1", "0")], correct = FALSE)  # yes counts first

# Cohen's h
p <- prop.table(tab, 1)[, "1"]
2 * asin(sqrt(p["peer"])) - 2 * asin(sqrt(p["standard"]))

# Newcombe's interval (the one this page reports)
library(DescTools)
BinomDiffCI(40, 125, 24, 120, method = "score")

	2-sample test for equality of proportions without continuity correction

data:  tab[, c("1", "0")]
X-squared = 4.5684, df = 1, p-value = 0.03257
alternative hypothesis: two.sided
95 percent confidence interval:
 0.01133006 0.22866994
sample estimates:
prop 1 prop 2
  0.32   0.20

     peer
0.2752332
      est      lwr.ci   upr.ci
[1,] 0.12 0.009828434 0.225951

R reports χ² rather than z: 4.5684 is 2.137². prop.test's interval is the Wald interval, peer − standard (the rows in alphabetical order); DescTools' method = "score" is Newcombe's. correct = FALSE matters: R's default adds Yates's continuity correction, which gives X-squared = 3.9677 and p = .046.

Output from a real run: R 4.6.1 (DescTools 0.99.60), run 2026-10-03.

Python

import pandas as pd
from statsmodels.stats.proportion import (
    proportions_ztest, confint_proportions_2indep, proportion_effectsize)

outreach = pd.read_csv("data.csv")
counts = outreach.groupby("message")["sought_help"].sum()    # peer 40, standard 24
totals = outreach.groupby("message")["sought_help"].count()  # peer 125, standard 120

z, p = proportions_ztest(counts, totals)
z, p  # (2.1374, 0.03257)

confint_proportions_2indep(40, 125, 24, 120)  # Newcombe: (0.0098, 0.2260)
proportion_effectsize(40 / 125, 24 / 120)     # Cohen's h: 0.2752

proportions_ztest uses the pooled proportion for the standard error, as this page does. confint_proportions_2indep defaults to Newcombe's interval, the one this page reports; method="wald" gives the Napkin Math version, (0.0113, 0.2287).

Output from a real run: Python 3.9.6 (statsmodels 0.14.6, pandas 2.3.3), run 2026-10-03.

Impact

Effect Size

The simplest size is the risk difference itself: 12 percentage points, in the outcome's own units. It's what a counseling center can plan around. But the same 12 points is easier to detect when the proportions are near 0 or 1 than near 50%, because proportions vary less there. Cohen's h converts each proportion with an arcsine transformation that stretches the scale near 0 and 1, then takes the difference, so equal values of h are about equally easy to detect whatever the baseline (Cohen, 1988). That's why power tables use it. Equal h doesn't mean equally important: a 10-point rise is h = 0.48 from 1% to 11% but h = 0.20 from 40% to 50%, and which matters more depends on the setting.

h = 2\arcsin\sqrt{\hat{p}_1} - 2\arcsin\sqrt{\hat{p}_2} = 2\arcsin\sqrt{.32} - 2\arcsin\sqrt{.20} = 0.28

Going from 20% to 32% booked is 12 percentage points, and h = 0.28, a small effect by convention.

You'll also see the ratio of the two proportions (.32 / .20 = 1.6, so 60% more students booked), the odds ratio (1.88 here, the number logistic regression reports) and the phi coefficient (φ = .14, the correlation between group and outcome). They describe the same table in different ways; say which one you report.

Benchmarks are conventions. Whether 12 more students per 100 reaching the counseling center matters is a question about student mental health and the center's capacity, not statistics.

Benchmarks Cohen's h at a Glance

Below Small

|h| < 0.2

Small

0.2 ≤ |h| < 0.5

Medium

0.5 ≤ |h| < 0.8

Large

|h| ≥ 0.8

Full benchmarks and caveats ↑

Statistical Significance

p answers one narrow question: if the two populations had the same proportion, how often would two samples this size land at least this far apart, in either direction? Below the conventional α = .05, the result is called statistically significant.

If the two emails worked equally well, a gap of 12 percentage points or more would turn up about 3.3% of the time, roughly 1 in 31. That's below .05, so it's significant.

The same data can come out of software as a chi-square statistic instead of z. For a 2 × 2 table, χ² without the continuity correction is exactly z² (here 2.137² = 4.57), with the same p. With Yates's continuity correction, R's default, χ² is 3.97 and p = .046. Yates's correction shrinks each gap between observed and expected counts by 0.5, to allow for counts being whole numbers, and it usually makes p too large. Campbell (2007), comparing the versions, found that the “N − 1” chi-square, a slight rescaling of the uncorrected one, worked best and Yates's didn't, so this page reports the uncorrected test. Name the version you used.

p is not the probability that the null hypothesis is true, and it says nothing about how large or important the difference is. That's what the effect size and interval are for.

Benchmarks p at a Glance

Very Strong

p < .001

Strong

.001 ≤ p < .01

Significant

.01 ≤ p < .05

Weak

.05 ≤ p < .10

Little

p ≥ .10

Strength of evidence against the null hypothesis.

Full benchmarks and caveats ↑

Power and Sample Size

Power is the chance that a study detects an effect that's really there. A real effect doesn't guarantee a significant result: with too few people, it can easily come out non-significant. Power depends on four things: how big the true difference is (and where the proportions sit, which Cohen's h takes care of), how many people you have in each group, α, and whether the test is one- or two-sided. Yes/no outcomes carry less information per person than scores, so proportion studies usually need bigger samples.

Suppose, before collecting any data, we'd hoped the peer-story email would lift bookings from 20% to 30% (h = 0.23). We'd have needed about 292 students per group for 80% power. The 125 and 120 we actually had give only about 44% power for that difference, so a real lift of that size would be missed more often than not.

Researchers usually aim for at least 80% power. Plan it before collecting data, using the smallest effect that would matter. For a small effect (h = 0.20), that's about 392 people per group at α = .05 (Cohen, 1992); the exact figure is 392.4, so 393 per group guarantees at least 80%. Try it:

Other Info

Follow-Up and Post Hoc Tests

None needed. This test answers a single question, so there are no planned contrasts or post hoc tests to run.

Related Methods

General CasePearson Chi-Square Test: Put the counts in a 2 × 2 table and run a chi-square test without the continuity correction: χ² equals z² exactly, with the same p. The chi-square test extends the same idea to more groups or more outcome categories.
General CaseLogistic Regression: Code the group as 0 and 1 and predict the yes/no outcome. The slope is the log of the odds ratio. Its score test is this z-test squared; the Wald and likelihood-ratio tests software usually prints give a slightly different p.
Related DesignMcNemar's Test: Compares two proportions too, but when the same people give a yes/no answer twice, or people come in matched pairs.
Related DesignIndependent-Samples t-Test: Answers the same two-group question when the outcome is a score rather than yes/no.
Related DesignBinomial Test: Answers the one-group version of the question exactly: is a single group's share different from a fixed value, such as chance? It uses binomial probabilities instead of the normal curve, so it suits small samples.
Related DesignFisher's Exact Test: Answers the same question about two proportions with exact probabilities instead of a normal approximation, which suits small samples.
AlternativeMultilevel Logistic Regression: An alternative if “Independent Observations” doesn't hold. A multilevel logistic regression, which accounts for the clustering.

References

Fleiss, J. L., Levin, B., & Paik, M. C. (2003). Statistical methods for rates and proportions (3rd ed.). Wiley. https://doi.org/10.1002/0471445428
Newcombe, R. G. (1998). Interval estimation for the difference between independent proportions: Comparison of eleven methods. Statistics in Medicine, 17(8), 873–890. https://doi.org/10.1002/(SICI)1097-0258(19980430)17:8<873::AID-SIM779>3.0.CO;2-I
“Computationally simpler methods are prone to a variety of aberrations and poor coverage properties.” (Abstract)
Campbell, I. (2007). Chi-squared and Fisher–Irwin tests of two-by-two tables with small sample recommendations. Statistics in Medicine, 26(19), 3661–3675. https://doi.org/10.1002/sim.2832
“The optimum test policy was found to be analysis by the 'N-1' chi-squared test when the minimum expected number is at least 1, and otherwise, by the Fisher-Irwin test by Irwin's rule” (Abstract)
Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159. https://doi.org/10.1037/0033-2909.112.1.155
“To detect a small difference between two independent population proportions (h = .20 in Table 1) at α = .05 requires N = 392 cases in each group.” (p. 158)
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.
Muff, S., Nilsen, E. B., O'Hara, R. B., & Nater, C. R. (2022). Rewriting results sections in the language of evidence. Trends in Ecology & Evolution, 37(3), 203–210. https://doi.org/10.1016/j.tree.2021.10.009
“we suggest language of evidence that allows for a more nuanced approach to communicate scientific findings as a simple and intuitive alternative to statistical significance testing” (Abstract)
McHugh, M. L. (2013). The chi-square test of independence. Biochemia Medica, 23(2), 143–149. https://doi.org/10.11613/BM.2013.018
“The value of the cell expecteds should be 5 or more in at least 80% of the cells, and no cell should have an expected of less than one” (Assumptions of the Chi-square)