Data
AI bias is when a model gives systematically unfair or skewed results for some groups, usually because its training data reflected real-world prejudice or gaps.
Models learn from data made by people, and people are not neutral. If past hiring records favored men, a model trained to predict 'good hires' learns that pattern. If a face dataset contains mostly light-skinned faces, the model works worse on darker skin. The model is not malicious. It faithfully copies what it was shown.
Bias can also come from the design: what gets measured, what counts as success, and who was in the room. Because AI systems make millions of decisions, a small skew becomes a large-scale harm. Testing performance separately for different groups is the first step to catching it.
Confusingly, 'bias' also has neutral technical meanings. It can be a small extra number added inside each neuron. In statistics it can mean a model that is too simple. When people discuss AI fairness, they mean the unfair kind described here.
A resume-screening tool, trained on ten years of a company's mostly male hires, quietly downgraded applications that mentioned a women's college.