Where does bias enter an AI system?
At every stage, and rarely by intent. Training data records past decisions, so a model trained on past hires learns to prefer people like them. Labels carry the judgment of whoever produced them, so ground truth needs the same scrutiny as the model. Proxies rebuild a protected attribute the model never saw: a postcode, a school or a career gap can stand in for ethnicity, gender or age. Deployment adds the rest, when a system meets people it was never tested on. Removing the protected column does not remove the bias.
What are the main types of AI bias?
NIST groups AI bias into three categories, systemic, statistical and human, and states that “it is not possible to achieve zero risk of bias in an AI system” (NIST SP 1270, March 2022, read 5 November 2026). In a hiring tool they look like this:
| Type | Source | Hiring example |
|---|---|---|
| Historical | Past decisions in the data | Favors profiles like previous hires |
| Sampling | Groups underrepresented | Less accurate for rarer backgrounds |
| Label | Biased ratings used as truth | Interview scores carry interviewer bias |
| Proxy | Fields correlated with protected traits | Career gaps penalized, hitting parents |
| Deployment | Use outside the tested context | Built for one market, used in another |
Not every case needs machine learning. The EEOC alleged that iTutorGroup’s software automatically rejected women aged 55 or older and men aged 60 or older; the company paid 365,000 US dollars to settle (EEOC, 11 September 2023, read 5 November 2026). The law judges the outcome, not the technique.
What does the EU AI Act require on bias?
AI used to recruit, screen applications, evaluate candidates or decide promotion and termination is high risk under Annex III point 4, which is why AI in HR draws scrutiny. Article 10(2)(f) and (g) require providers of high-risk systems to examine their data for possible biases and take measures to detect, prevent and mitigate them (Article 10, read 5 November 2026). Article 4a, added by the digital omnibus, lets them exceptionally process special categories of personal data where strictly necessary for that work (Article 4a, read 5 November 2026). A company using a bought screening tool still answers for how it is used, so ask the vendor for its bias testing evidence before signing under the EU AI Act.
How do you test an AI system for bias?
Compare outcomes across groups on data that looks like yours. Build an eval dataset with group labels held aside, then measure selection rates and error rates per group, including false positives. Run counterfactual checks: change only a name, gender or age and see whether the result moves. Ask which features drive decisions, where AI explainability earns its keep. Then keep testing after launch, because model drift and a changing applicant pool can reopen a gap, and an independent AI audit gives the result weight.