an AI resume screener had been trained on CVs of employees already at the firm, giving people extra marks if they listed “baseball” or “basketball” – hobbies that were linked to more successful staff, often men. Those who mentioned “softball” – typically women – were downgraded.
Marginalised groups often “fall through the cracks, because they have different hobbies, they went to different schools”
Oh there is so much racist data that the AI is being trained on.
Your example is a simple one. But there are discriminations based on names for instance, so Johns are hired more than Quachin is, and that is by people, before it gets to the AI.