In the supposedly first report of its kind, researchers have shown that ChatGPT generates racist and other harmful dialogue in response to user questions. By asking the model to respond to questions as a good or a bad person, a man or woman, or as individuals of varying racial or ethnic backgrounds, it produced “biased and hurtful commentary” showing varying degrees of toxicity.

The report (12 pages, not very technical) can be downloaded here: https://dx.doi.org/10.48550/arxiv.2304.05335