The Sign Test: Formula, Worked Example and Critical Values

In 1710, a royal physician counted London christening records for 82 straight years, found boys outnumbered girls every single year, and published what is now recognised as the first test of statistical significance. That test was the sign test.

The sign test is usually the first inferential test A level psychology students meet, and it is deliberately the simplest one on the specification. It asks a single question: when scores change, do they mostly change in one direction? If nearly everyone improves after an intervention, that pattern is unlikely to be chance. If improvements and declines are roughly balanced, it probably is.

What makes it approachable is also what makes it limited. The sign test ignores how much each score changed. A participant who improved by one point and a participant who improved by forty points both count as a single plus. That deliberate simplicity is why it can be calculated by hand in a few minutes, and why it needs a fairly convincing pattern before it will declare anything statistically significant.

This guide covers when the sign test applies, the formula and what S actually represents, two fully worked examples, how to read the critical values table, how to write up the result in the wording examiners expect, and the mistakes that cost marks most often. There is also a calculator below so you can check your own answers as you work.

What Is the Sign Test?

The sign test is a non-parametric test of difference for related data, which uses the direction of change in each pair of scores rather than the size of that change. It tells you whether the number of changes in one direction is too lopsided to be explained by chance alone.

The logic is the same as flipping a coin. If a revision workshop made no real difference to student confidence, then any change in a student’s score after the workshop would be as likely to go down as up – roughly half plus signs, half minus signs, like heads and tails. The sign test calculates how surprising your actual split of pluses and minuses would be if that coin were fair. A split of nine up and one down is surprising. A split of six up and four down is not.

It is called non-parametric because it makes no assumptions about the shape of the underlying distribution. Parametric tests such as the t-test assume the data are roughly normally distributed and measured on an interval scale. The sign test assumes neither, which is why it survives data that would make a t-test inappropriate.

When to Use the Sign Test: Three Conditions

Use the sign test when all three of these conditions are met. If any one fails, a different test is needed, and choosing the wrong test loses marks even when the arithmetic is perfect.

That third condition is the one students most often misread. “Nominal” does not mean your data must be nominal – it means nominal is the minimum. Interval data works perfectly well, because interval scores can always be reduced to a direction. You are simply choosing to discard the extra detail. If you want to keep that detail, the Wilcoxon signed-rank test uses the ranked size of each difference instead, and is more likely to detect a real effect.

A test of relationship is a different job entirely. If you want to know whether two variables move together rather than whether one condition differs from another, you need a correlation coefficient such as Spearman’s rho or Pearson’s r.

The Sign Test Formula and What S Means

There is no algebraic formula for the sign test in the way there is for a t-test. The calculated value, S, is simply a count:

• S = the number of times the less frequent sign occurs
• N = the number of participants left after all ties are removed

If eleven participants improved and three got worse, the less frequent sign is the minus, so S = 3 and N = 14. If three improved and eleven got worse, S is still 3. The test does not care which direction dominates, only how lopsided the split is.

This is why the sign test needs so little from your data. There is no mean to calculate, no standard deviation, no assumption of normality. You are counting arrows, not measuring distances.

Worked Example: Does a Revision Workshop Raise Confidence?

A psychology teacher runs a revision workshop and wants to know whether it raises exam confidence. Twelve students rate their confidence out of 10 immediately before the workshop and again immediately after. The hypothesis is directional: confidence will be higher after the workshop, so this is a one-tailed test.

Step 1: Record the direction of each change

Step 2: Count the signs and remove the ties

Nine students improved, one got worse, and two showed no change. Students 3 and 10 are removed completely, because a tie provides no evidence in either direction. That leaves ten usable participants.

• Positive signs: 9
• Negative signs: 1
• Ties removed: 2
• N = 10
• S = 1 (the less frequent sign)

Step 3: Find the critical value

For a one-tailed test at p < 0.05 with N = 10, the critical value of S is 1. Look this up in the table further down the page, or use the calculator.

Step 4: Compare and decide

Here is the rule that trips students up: for the sign test, the calculated value must be equal to or less than the critical value for the result to be significant. Most tests you meet later require the calculated value to be equal to or greater than the critical value. The sign test runs the other way round, because a small S means a lopsided split, and a lopsided split is what you are hoping to find.

S = 1 and the critical value = 1. Since 1 is equal to 1, the result is significant at p < 0.05. The null hypothesis is rejected: the workshop did raise confidence by more than chance would explain.

Sign Test Calculator and Critical Values

Enter your own counts below to check an answer. The calculator works out S, removes nothing you have not already excluded, and compares your result against the exact critical value for that sample size. The full table of critical values sits underneath it.

How to Read the Critical Values Table

Find the row matching your N after ties have been removed, then read across to the column for your hypothesis. That number is the largest value of S that still counts as significant at p < 0.05.

Three features of the table are worth noticing, because each one carries a mark in an exam answer.

• The critical values are small. At N = 10 you need S of 1 or 0, meaning at most one participant moved against the trend. The sign test demands a strikingly clean pattern before it will call anything significant.
• N = 5 has a dash in the two-tailed column. With five participants, even a perfect five-nil split only has a two-tailed probability of 0.0625, which is above 0.05. No result is significant at that sample size, however convincing it looks.
• Critical values grow slowly. Doubling N from 10 to 20 only moves the one-tailed critical value from 1 to 5. Larger samples buy you tolerance for participants who move the wrong way.

That first point explains why researchers rarely choose the sign test when they have better data available. It is asking for near-unanimity. The same logic appears whenever a calculated value is judged against a critical value, whether you are working with a z-score or an inferential test – what changes is the direction of the comparison and how much evidence the test demands.

One-Tailed or Two-Tailed: Why the Column Matters

Use the one-tailed column when your hypothesis predicted the direction of the difference, and the two-tailed column when it only predicted that a difference would exist. This choice is made before you collect data, not after you see which column gives the answer you want.

The consequences are real. Consider a study where ten participants take part, two produce ties, seven improve and one declines. That gives N = 8 and S = 1.

• One-tailed: the critical value at N = 8 is 1. S = 1 is equal to the critical value, so the result is significant at p < 0.05.
• Two-tailed: the critical value at N = 8 is 0. S = 1 is greater than 0, so the result is not significant.

Identical data, opposite conclusions. A directional hypothesis concentrates all 5% of your risk at one end of the distribution rather than splitting it across both, which makes significance easier to reach – but you may only claim that advantage if you committed to the direction in advance, and you must be able to justify it from previous research.

What to Do With Ties

Remove ties entirely and reduce N accordingly. A participant whose score was identical in both conditions provides no evidence for or against the hypothesis, so the sign test discards them rather than counting them as half a point in each direction.

This has a consequence students often miss: ties make significance harder to achieve, not easier. In the worked example above, twelve students took part but N was 10. Had four students scored identically instead of two, N would have dropped to 8, and the critical value with it. Losing participants to ties shrinks the sample the test actually runs on.

Ties are especially common with short rating scales. Asking for confidence out of 5 rather than out of 10 roughly doubles the chance that a participant lands on the same number twice. If you are designing a study that will be analysed with a sign test, a finer scale protects your sample size.

How to Report the Result

A complete sign test result states five things: the calculated value, the critical value, N, the significance level, and whether the test was one- or two-tailed. Examiners look for all five, and dropping any of them costs marks even when the conclusion is right.

A model sentence for the worked example:

The calculated value of S (1) is equal to the critical value (1) for N = 10 at p < 0.05 for a one-tailed test. The null hypothesis is therefore rejected, and the alternative hypothesis accepted: confidence ratings were significantly higher after the workshop than before it.

Two details of wording matter. First, results are significant at a level, not simply "significant". Second, you reject or retain the null hypothesis – you never "prove" the alternative. A significant result at p < 0.05 still carries a one-in-twenty risk that the pattern arose by chance, which is precisely what a Type I error is.

Second Worked Example: A Two-Tailed Study

This example runs a complete analysis on a non-directional hypothesis, so you can see how the write-up changes when you cannot predict which way the difference will go.

A researcher wants to know whether background music changes how many words participants recall from a list. Previous findings are mixed – some studies report improvement, others interference – so the hypothesis is non-directional: there will be a difference in recall between the silent and music conditions. Fourteen participants complete both conditions in a counterbalanced order.

Counting up: ten participants recalled fewer words with music, two recalled more, and two showed no change. The ties are removed, leaving N = 12. The less frequent sign is the plus, so S = 2.

For a two-tailed test at N = 12, the critical value is 2. S = 2 is equal to the critical value, so the result is significant at p < 0.05. Note how close this is: one more participant improving with music would have pushed S to 3 and the result would have failed.

The write-up: the calculated value of S (2) is equal to the critical value (2) for N = 12 at p < 0.05 for a two-tailed test, so the null hypothesis is rejected. Recall differed significantly between the two conditions, with most participants recalling fewer words when music was playing.

Because the hypothesis was non-directional, the conclusion states that a difference exists and then describes its direction separately as an observation. You cannot claim to have predicted it.

Where the Sign Test Fits in the Research Process

The decision to use a sign test is made at the design stage, not once the data are in. By the time you are counting signs, the choices that determine whether the test is appropriate have already been made.

1. Write the hypothesis. A directional hypothesis names the direction of the difference; a non-directional one only claims a difference exists. This fixes which column of the table you will use.
2. Choose the design. The sign test needs repeated measures or matched pairs. If participants are allocated to separate groups, the test is unavailable before a single score is collected.
3. Decide the measure. Any scale works, but a coarse one produces ties, and ties cost you sample size.
4. Set the significance level. Psychology conventionally uses p < 0.05. A stricter level such as 0.01 is used where a false positive would be costly.
5. Collect, count, compare. Only at this point does the arithmetic begin.

Ethical approval sits alongside these steps rather than after them, and the way a study is designed shapes what can responsibly be claimed from it – a question that runs well beyond statistics into research ethics and social sensitivity.

Sign Test, Wilcoxon or Chi-Squared: Which One?

All three are non-parametric, and students routinely confuse them. The deciding questions are whether your design is related or unrelated, and whether you are testing a difference or an association.

The sign test and Wilcoxon apply to exactly the same situations. The difference is how much of your data each one uses. If your scores are at least ordinal, Wilcoxon is the stronger choice, because a participant who improved by eight points contributes more evidence than one who improved by a single point. The sign test treats them identically. Choosing between them is really a question about what your data can legitimately support rather than which is easier to calculate.

The Weakness of the Sign Test: Statistical Power

The sign test has low statistical power, which means it frequently fails to detect real effects. Power is the probability that a test will find an effect that genuinely exists, and by discarding the size of every difference the sign test throws away most of the information that would let it do so.

The practical consequence is a raised risk of a Type II error – retaining the null hypothesis when you should have rejected it. A real, modest improvement across a group can easily fail the sign test simply because two or three participants drifted the wrong way. With interval data and a normal distribution, a related t-test would detect that same effect comfortably.

This is the trade-off behind every non-parametric test. You gain freedom from assumptions about distribution and measurement, and you pay for it in sensitivity. Where the assumptions genuinely hold, a parametric test extracts more from the same participants – which is also why the standard error and other measures of precision play no part in a sign test at all. There is no estimate of a population value here, only a count of directions.

Low power matters most with small samples, which is exactly where psychology students meet it. A study on eight participants has very little chance of producing a significant sign test whatever the true effect, in the same way that case study research trades statistical generalisability for depth. Knowing this in advance is worth a mark in an evaluation question.

Common Mistakes That Cost Marks

• Using the wrong N. N is the number of participants left after ties are removed, not the number who started the study.
• Comparing in the wrong direction. For the sign test, significance requires calculated ≤ critical. Most other tests require calculated ≥ critical, and the habit carries over.
• Choosing the tail after seeing the data. The hypothesis fixes the column before analysis begins.
• Counting the more frequent sign. S is always the less frequent one, so S can never be more than half of N.
• Applying it to an unrelated design. Two separate groups of people cannot be paired, so there are no differences to take signs from.
• Reporting a bare verdict. "Significant" on its own earns little. State S, the critical value, N, the significance level and the tail.

Where the Sign Test Came From

The sign test is the oldest significance test in use, predating the modern framework of hypothesis testing by more than two centuries. In 1710 John Arbuthnot, physician to Queen Anne, examined London christening records covering 82 consecutive years and found that male births had outnumbered female births in every single one of them (Arbuthnot, 1710).

His reasoning was a sign test in everything but name. If the sexes were equally likely, he argued, each year was effectively a coin flip, and 82 heads in a row has a probability of one in 2 to the power of 82 – a number so small he treated it as proof of divine providence. Modern statisticians draw a different conclusion from the same arithmetic, but the original paper in the Philosophical Transactions is recognisably the test students still perform today.

The test was formalised in its modern shape by Dixon and Mood (1946), who set out the binomial basis and published the tables of critical values that textbooks have reproduced ever since. Those tables are simply cumulative binomial probabilities for a fair coin, which is why the calculator above can generate them from scratch rather than looking them up.

For A level students, the sign test appears in the research methods content of the AQA A-level Psychology specification (7182), where it is the only inferential test you may be asked to calculate in full. The others must be selected and interpreted, but not computed by hand. The APA Dictionary of Psychology gives the broader definition used across the discipline, where the test is often framed as a comparison of medians.

Conclusion

The sign test earns its place on the specification by being the clearest possible demonstration of what inferential statistics actually do. Strip away the arithmetic and every significance test is asking the same question: is this pattern too lopsided to be chance? The sign test asks it with nothing more than a column of plus and minus symbols.

Remember the three things that separate it from every other test you will meet. N is what remains after ties are removed. S is the count of the less frequent sign. And significance requires the calculated value to be equal to or less than the critical value. Get those right and the calculation itself is straightforward.

Its weakness is the mirror image of its simplicity. By ignoring how much scores changed, the sign test needs a nearly unanimous pattern before it will report anything, which makes it a blunt instrument where better data are available. Knowing when to reach for something sharper is as much a part of the skill as performing the test.

Frequently Asked Questions

What is the sign test in psychology?

The sign test is a non-parametric test of difference used with related data, such as repeated measures or matched pairs. It records whether each participant’s score went up or down between two conditions and ignores the size of the change. The calculated value, S, is the number of times the less frequent sign appears. If S is equal to or less than the critical value for that sample size, the difference is significant at p < 0.05.

When do you use the sign test?

Use the sign test when three conditions are met: you are testing for a difference rather than a relationship, the design is related, and the data can be sorted into two directions. It requires only nominal data as a minimum, so it works where a t-test would be inappropriate. If the data are ordinal or better, the Wilcoxon signed-rank test uses more of the information and is more likely to detect a real effect.

How do you calculate the sign test?

Work out the direction of change for each participant and mark it plus or minus. Remove every tie and count the participants that remain to get N. Count the plus signs and the minus signs, and take the smaller of the two counts as S. Look up the critical value for your N in the one-tailed or two-tailed column, depending on your hypothesis. If S is equal to or less than that value, the result is significant.

What is the calculated value in the sign test?

The calculated value is S, the number of times the less frequent sign occurs. If nine participants improved and one declined, the less frequent sign is the minus, so S = 1. Because S counts the smaller group, it can never be more than half of N. A small S means a lopsided pattern, which is what makes a result significant – the opposite of most tests, where a large calculated value indicates significance.

Is the sign test nominal data?

The sign test requires data at the nominal level as a minimum, not exclusively nominal data. Each pair of scores must be classifiable into one of two categories, increased or decreased, which is a nominal judgement. Interval and ordinal data both qualify, because they can always be reduced to a direction. Doing so discards detail, which is why a test that uses the size of each difference is preferred when the data allow it.

What is the critical value for the sign test?

Critical values depend on N and on whether the test is one- or two-tailed. At p < 0.05 the one-tailed critical value is 0 for N = 7, 1 for N = 8, 2 for N = 11 and 5 for N = 19. The two-tailed value is the same or one lower at every sample size, because the 5% risk is split across both ends: 0 for N = 8, 1 for N = 11 and 4 for N = 19. At N = 5 no two-tailed result can reach significance at all. The values are cumulative binomial probabilities for a fair coin.

Do you include ties in the sign test?

No. Ties are removed completely and N is reduced accordingly. A participant whose score was identical in both conditions offers no evidence either way, so the test discards them rather than splitting them between the two signs. This makes significance harder to reach, because a smaller N comes with a smaller critical value. Short rating scales produce more ties, so a finer measure protects your sample size.

What is the difference between the sign test and the Wilcoxon test?

Both apply to related designs testing for a difference, but they use different amounts of information. The sign test records only the direction of each change, while Wilcoxon ranks the differences by size and uses those ranks. A participant who improved by ten points counts the same as one who improved by one point in a sign test, but contributes far more to a Wilcoxon. Wilcoxon has greater statistical power as a result.

References

• American Psychological Association. (n.d.). Sign test. In APA dictionary of psychology.
• Arbuthnot, J. (1710). An argument for divine providence, taken from the constant regularity observed in the births of both sexes. Philosophical Transactions of the Royal Society of London, 27(328), 186–190.
• Assessment and Qualifications Alliance. (2015). AS and A-level psychology specification (7181, 7182). AQA.
• Conover, W. J. (1999). Practical nonparametric statistics (3rd ed.). Wiley.
• Coolican, H. (2019). Research methods and statistics in psychology (7th ed.). Routledge.
• Dixon, W. J., & Mood, A. M. (1946). The statistical sign test. Journal of the American Statistical Association, 41(236), 557–566.
• Field, A. (2018). Discovering statistics using IBM SPSS statistics (5th ed.). Sage.
• Howitt, D., & Cramer, D. (2017). Introduction to statistics in psychology (7th ed.). Pearson.
• Siegel, S., & Castellan, N. J. (1988). Nonparametric statistics for the behavioral sciences (2nd ed.). McGraw-Hill.

Further Reading and Research

Recommended Articles

• Dixon, W. J., & Mood, A. M. (1946). The statistical sign test. Journal of the American Statistical Association, 41(236), 557–566.
• Arbuthnot, J. (1710). An argument for divine providence, taken from the constant regularity observed in the births of both sexes. Philosophical Transactions of the Royal Society of London, 27(328), 186–190.
• Coolican, H. (2019). Non-parametric tests. In Research methods and statistics in psychology (7th ed., pp. 486–512). Routledge.

Suggested Books

• Coolican, H. (2019). Research methods and statistics in psychology (7th ed.). Routledge.
  • The standard undergraduate reference in the UK, with a chapter on non-parametric tests that covers the sign test, Wilcoxon and Mann-Whitney side by side with worked examples.
• Siegel, S., & Castellan, N. J. (1988). Nonparametric statistics for the behavioral sciences (2nd ed.). McGraw-Hill.
  • The classic treatment of non-parametric methods, setting out the binomial reasoning behind the sign test and the conditions under which each alternative test is preferable.
• Howitt, D., & Cramer, D. (2017). Introduction to statistics in psychology (7th ed.). Pearson.
  • Written for students meeting inferential statistics for the first time, with step-by-step calculations and a clear explanation of why non-parametric tests sacrifice power.

Recommended Websites

• AQA A-level Psychology specification
  • The specification itself, including the research methods content and the mathematical requirements that set out exactly which tests must be calculated by hand.
• APA Dictionary of Psychology
  • Short, authoritative definitions of the sign test and every related term, written for a discipline-wide audience rather than one exam board.
• Philosophical Transactions of the Royal Society archive
  • Free access to Arbuthnot’s 1710 paper and the rest of the earliest scientific literature, useful for seeing statistical reasoning before the modern vocabulary existed.

Kathy Brodie

Kathy Brodie is an Early Years Professional, Trainer and Author of multiple books on Early Years Education and Child Development. She is the founder of Early Years TV and the Early Years Summit.

Kathy's Author Profile

To cite this article please use:

Early Years TV The Sign Test: Formula, Worked Example and Critical Values. Available at: https://www.earlyyears.tv/sign-test-formula-worked-example-critical-values/ (Accessed: 20 September 2026).