This is a good question.

Open Empathic’s answer is that because AI is becoming more embedded in our lives, an “understanding” of emotions on AI’s part will help people in variety of ways, both within and outside of industries like healthcare, education, and, of course, general commercial endeavors. As far as they’re concerned, AI is a tool that will help encourage “ethical” human decision-making.

On the other hand…we have a ton of different ethical theories and industries ignore them wholesale to make profits. To me, this looks like your standard grade techno bro hubris. They intend to use “disruptive” technology to “revolutionize” whatever. The exploitative profit-making social hierarchy isn’t being challenged. The Hollywood’s writer strikes have just begun, for example. Once Open Empathic starts making breakthroughs in artificial emotional intelligence, the strikes will return and be even more prolonged, if not broken altogether.

I’d answer your question with people who care about other people should be deeply concerned.

Even without a focus on empathy, ChatGPT’s responses in a healthcare setting were rated as more empathic. At best, empathic AI is used to teach people how to be more empathetic to other humans, eventually needing it less and less over time. Far more likely is that human communication becomes mediated through empathic AI (and some company makes a lot of money off the platform of mediation) and the quality of face-to-face human interaction deteriorates.