Homeโ€บGlossaryโ€บT-Test

T-Test

General

Student's T-Test

A statistical test that compares the means of one or two groups to determine whether an observed difference is likely real or just due to random sampling variation.

Definition

A t-test is a statistical hypothesis test that compares the means of one or two groups of numeric data to determine whether an observed difference is likely to be real, or simply the product of random sampling variation. It's the standard tool for questions like "did the new checkout page increase average order value?" or "is there a real difference in test scores between two teaching methods?"

The test produces a t-statistic and a corresponding p-value โ€” the probability of observing a difference this large (or larger) purely by chance if there were truly no difference between the groups. The T-Test Calculator computes both directly from your group data.

Formula

For an independent two-sample t-test with roughly equal variances:

t = (xฬ„โ‚ โˆ’ xฬ„โ‚‚) รท โˆš(sยฒโ‚š ร— (1/nโ‚ + 1/nโ‚‚))

where xฬ„โ‚ and xฬ„โ‚‚ are the two group means, sยฒโ‚š is the pooled variance, and nโ‚, nโ‚‚ are the sample sizes.

Worked Example

Comparing two website checkout designs: Design A has a mean order value of $58 (n=40, s=12), Design B has a mean of $63 (n=42, s=13). Running these through the t-test formula produces a t-statistic around 1.8, which โ€” checked against a t-distribution table with the appropriate degrees of freedom โ€” gives a p-value around 0.08, above the standard 0.05 threshold. This particular sample doesn't provide strong enough evidence to conclude Design B truly outperforms Design A, even though its mean is numerically higher.

Key Things to Know

  • Significance is about chance, not importance: a statistically significant result means the difference is unlikely due to chance โ€” it doesn't automatically mean the difference is large or practically valuable.
  • Sample size affects power: small samples can fail to detect a real difference (a false negative), while very large samples can detect trivially small, practically meaningless differences as "significant."
  • Choose the right variant: use a paired test for before/after data on the same subjects, and an independent two-sample test for separate groups.
  • Pair with a confidence interval: a t-test tells you whether a difference is significant, while a confidence interval tells you the likely range of that difference's true size.
  • Related to chi-square, but different data types: t-tests compare means of continuous measurements, while a chi-square test compares frequencies of categories.

Related Calculators

T-Test Calculator โ†’Run a two-sample t-test in seconds. Enter each group's mean, standard deviation, and sample size to get the t-statistic, degrees of freedom, and a significance verdict.
Chi-Square Test Calculator โ†’Run a chi-square goodness-of-fit test in seconds. Enter up to six rows of observed vs expected frequencies to get the chi-square statistic, degrees of freedom, and verdict.
Confidence Interval Calculator โ†’Calculate a confidence interval from a sample mean, standard deviation, and sample size in seconds. Get the lower bound, upper bound, and margin of error instantly.

Related Terms

P-ValueProbability ValueConfidence IntervalConfidence Interval Estimate

Frequently Asked Questions

A one-sample t-test compares a single sample's mean to a known or hypothesized value. An independent two-sample t-test compares means from two separate, unrelated groups. A paired t-test compares two measurements taken on the same subjects, such as before-and-after results. Use the [T-Test Calculator](/t-test-calculator/) for the two-sample case, the most common scenario in experiments and A/B tests.
The t-statistic measures how many standard errors apart two group means are, combining the size of the observed difference with the variability and sample size of the data. A larger absolute t-statistic means the observed difference is large relative to the noise in the data, making it less likely to have occurred by chance alone.
Compare the resulting p-value to your chosen significance threshold (commonly 0.05) โ€” if the p-value is below that threshold, the result is considered statistically significant, meaning the observed difference is unlikely to be due to chance alone. A p-value of 0.03, for example, would be significant at the standard 0.05 threshold but not at a stricter 0.01 threshold.
Not necessarily โ€” statistical significance only indicates the difference is unlikely to be due to random chance, not that the difference is large or meaningful in a practical sense. A very large sample size can produce a statistically significant result for a tiny, practically irrelevant difference, which is why effect size should always be considered alongside significance.
The standard t-test assumes the underlying data is approximately normally distributed (or the sample size is large enough for the Central Limit Theorem to apply), that observations are independent, and โ€” for the two-sample version โ€” that both groups have similar variance (though variants exist for unequal variances). Violating these assumptions substantially can make the test's p-value unreliable.