Are IQ Tests Biased? What the Word Means in Psychometrics

Ask whether IQ tests are biased and you will get confident answers in both directions, often citing the same research. That is not because anyone is lying. It is because “bias” means something specific and narrow in psychometrics and something much broader in ordinary speech, and the studies address the narrow version.

Untangling the two gives you a genuinely useful answer to both, so that is what this page does.

The two things called bias

In everyday use, calling a test biased usually means something like it is unfair, it favours some people over others for reasons that have nothing to do with ability. That is a reasonable thing to mean and a hard thing to measure.

Psychometrics uses the word for two specific properties that can be measured:

• Predictive bias. Does the test systematically over- or under-predict some outcome, such as school grades or job performance, for one group compared with another? If two people from different groups get the same score, does the same outcome tend to follow?
• Measurement bias. Does a particular question behave differently for people of equal underlying ability who belong to different groups? This is assessed as differential item functioning.

Both are testable claims with established methods behind them. Neither is the same as the everyday question, and a test can pass both while still producing outcomes people reasonably call unfair. That is not a gotcha, it is just what the terms mean.

What the research finds on predictive bias

This is the question that has been studied most heavily, largely because it is the one that matters legally for hiring and admissions.

The general finding, across a large literature, is that well-constructed ability tests predict outcomes about equally well across English-speaking groups within a country. A given score tends to be followed by similar grades or job performance regardless of which group the test-taker belongs to. Arthur Jensen’s extensive work on this in the 1970s and 1980s reached that conclusion, and the broad position has held up in subsequent work.

Where predictive bias has been detected, it has usually taken a form that surprises people: over-prediction for minority group members rather than under-prediction. In technical terms, studies tend to find intercept differences rather than slope differences, meaning the test predicts slightly better outcomes than actually occur for some groups. That is the opposite direction from what the everyday accusation implies.

The standard consensus statement on this and related questions is Intelligence: Knowns and Unknowns, produced by an American Psychological Association task force chaired by Ulric Neisser and published in 1996. It was commissioned specifically to establish what the field could and could not agree on, and it remains the most cited attempt to draw that line.

How individual questions are screened

Differential item functioning, usually shortened to DIF, is the tool for catching problems at the level of single questions rather than whole tests.

The logic is straightforward. Take people who score the same overall, so their underlying ability is comparable. If members of one group are systematically more likely to get a particular item right than equally able members of another group, something about that item is doing work other than measuring ability.

Modern test development screens for this routinely. Items flagged by DIF analysis are reviewed and often removed before a test is published, which is one of the more meaningful differences between a clinical instrument and a quiz someone assembled over a weekend.

Worth being precise about what a DIF flag means, though: detecting it identifies an item for review rather than proving bias. There can be legitimate reasons why an item behaves differently across groups, and judgement is required. It is a screening signal, not a verdict.

The limit nobody should skip

Here is where a lot of confident writing on this subject stops, and it should not.

Showing that a test predicts an outcome equally well across groups does not establish that the test is fair in the ordinary sense. It establishes that the test relates to that outcome consistently. If the outcome itself is shaped by unequal schooling, unequal opportunity or unequal treatment, then a test that predicts it accurately is faithfully tracking a system that is already uneven.

This is not a critic’s objection smuggled in at the end. Jensen, who argued as strongly as anyone that ability tests were not predictively biased, stated the point directly: a culturally biased test can still show good predictive validity for a culturally biased criterion. The statistical finding and the fairness question are different, and the first does not settle the second.

So the accurate summary is narrower than either camp tends to offer. Well-built tests are not, in general, predictively biased in the technical sense. Whether the whole arrangement is fair is a question about criteria, access and consequences that psychometrics is not equipped to answer.

Cultural loading is real and different

Cultural loading is not the same as statistical bias, and conflating them causes half the confusion in this debate.

An item is culturally loaded when answering it draws on knowledge that is unevenly distributed for reasons unrelated to reasoning ability. Vocabulary items are the obvious case: knowing a word requires having encountered it. General knowledge items are another. So, more subtly, is familiarity with the conventions of multiple-choice testing, working at speed under observation, and the tacit understanding that you should guess rather than leave blanks.

This is why the field distinguishes culture-loaded from culture-reduced tests. Matrix reasoning tasks, which ask you to complete abstract visual patterns, are culture-reduced because they require no specific vocabulary or knowledge. They are not culture-free, and tests marketed as culture-fair are more accurately described as culture-reduced. Familiarity with abstract puzzle formats is itself something schooling provides unevenly.

The unfairness that actually bites

In practice, the things most likely to make a particular score misleading for a particular person are not subtle statistical properties. They are mundane and well understood:

• Testing in a second language. The largest single issue. Verbal subtests measure language proficiency alongside reasoning when the test-taker is not a native speaker, and the resulting composite understates ability. Qualified assessors are expected to account for this, often by relying on non-verbal measures or interpreting verbal indices with explicit caution.
• Unfamiliarity with testing. Someone who has sat standardised tests throughout their schooling has an advantage over someone who has not, independent of ability.
• Testing conditions. Illness, exhaustion, stress, pain and medication all depress performance. So does an examiner the test-taker finds intimidating or unfamiliar.
• Uneven cognitive profiles. A composite score averages several quite different abilities, which can misrepresent someone with a specific learning difficulty. This is covered in our guide to an IQ of 127.

None of these makes the test biased in the technical sense. All of them can make a specific score wrong about a specific person, which is what most people are actually asking about.

Where online tests stand

Worth being straight about this, including about our own.

Clinical instruments like the WAIS-5 and WISC-V go through item-level DIF screening, large representative norming samples and published technical manuals that let anyone check the work. Most online tests do none of that. They are typically normed on whoever happened to take them, which is a self-selected group rather than a representative sample, and they publish nothing about how items were chosen.

That is a construction problem rather than a bias problem specifically, but it has the same practical consequence: you cannot tell what the number means. A well-built online test can give a useful indicative estimate. It is not equivalent to a supervised clinical assessment, and anyone claiming otherwise is overselling. More on that distinction in our guide to online test accuracy.

Frequently asked questions

The bottom line

In the technical sense psychometrics uses, well-constructed IQ tests are generally not biased: they predict outcomes about equally well across English-speaking groups within a country, and individual items are screened for differential functioning before publication. That finding is solid and it is narrower than it sounds.

What it does not establish is that the whole arrangement is fair, because a test can predict an unequal criterion faithfully. And it says nothing about the things most likely to make your own score misleading: testing in a second language, unfamiliarity with the format, poor conditions on the day, or a cognitive profile too uneven for a single composite to describe. Those are the questions worth asking about any particular result.