there is… a lot going on here–and it’s part of a broader trend which is probably not for the better and speaks to some deeper-seated issues we currently have in society. a choice moment from the article here on another influencer doing a similar thing earlier this year, and how that went:
Siragusa isn’t the first influencer to create a voice-prompted AI chatbot using her likeness. The first would be Caryn Marjorie, a 23-year-old Snapchat creator, who has more than 1.8 million followers. CarynAI is trained on a combination of OpenAI’s ChatGPT-4 and some 2,000 hours of her now-deleted YouTube content, according to Fortune. On May 11, when CarynAI launched, Marjorie tweeted that the app would “cure loneliness.” She’d also told Fortune that the AI chatbot was not meant to make sexual advances. But on the day of its release, users commented on the AI’s tendency to bring up sexually explicit content. “The AI was not programmed to do this and has seemed to go rogue,” Marjorie told Insider, adding that her team was “working around the clock to prevent this from happening again.”
I don’t really like the logic of “chatbots like this will help cure loneliness.” It might help someone feel less lonely at first. But then it’ll be a crutch and, if anything, hurt people’s ability to socialize with other real people. Like it’s a quick dopamine hit that will slowly dig you deeper into the hole you feel you’re in.
Adding on to your comment, I read a really interesting study recently that suggests that interacting with AI engages the social parts of our brains but does not provide the same stimuli/feedback as interacting with a real person leading to increased loneliness and thus increased alcohol abuse and insomnia.
I was wondering. Seemingly, it could teach interaction skills as long as the bot wasn’t abnormally accomindating. Maybe it works for some and not others. So, it could be a therapy option. Just needs to be monitored for whether the person uses it to hurt themselves.
Fascinating, do you happen to have a link? No worries if it’s buried or you don’t remember, but I’d love to read it
Sure thing, here ya go!
Apologies for the extremely late reply but thank you so much for this fascinating article!
It is like any of these people have not seen any sci-fi released over the last 50 years.
The AI was not programmed to do this and has seemed to go rogue
The AI does exactly what it was programmed to do. It’s a machine. They fucked up making it, that’s the reality here.
The cynic in me says they didn’t actually fuck up. The AI being horny brought more attention to it and playing it off like “oh no! it went rogue!” gives them even more headlines. All a strategic play.
Eh, all chat bots trained on internet data become horny in the same way that virtually all AI is racist. It’s merely a reflection of the data it’s trained on, but a lot of people think they can magically control it by putting ‘safeguards’ after it’s been trained. The constant game of cat and mouse with gpt3/4 and prompts to getting it to ignore safeguards are proof this doesn’t work.
You can fix this through additional weighted layers and more complex upstream training tweaks, but then you have to pay to retrain which is by far the most expensive part about open AIs model.
You can also curate the training data so that it’s not problematic, but then you are biasing the model in other ways.
Impossible at scale. Data is always biased, it’s a matter of tradeoff
Yeah, I laughed at that part. The bot does what it interprets from the training data. Her entire schtick is leaning into sexual overtones to keep the guys subscribed. The fact that the bot started replicating that is entirely expected.
The trick there is that there is a fine line she walks between keeping it overt enough to keep the guys interested, but not so forward as to cross the terms of service boundaries on Twitch/YouTube. I think the problem is really that the bot isn’t adhering to the “tone it down so we don’t get banned from the service” unwritten rule. I’m not sure if I’d call that “rogue” though.
But, this is a great use case for a LLM, and if she can capitalise on it then good on her.
When I watched it, I really didn’t understand just how prescient Her was going to be. Great movie, I should rewatch it.
I don’t agree with Amouranth’s statement of “I don’t think it’s harmful for people to form a social aspect with it. As long as we’re aware that it’s not a real person.” You can tell people it’s not real all you want, but when you model it off your voice/likeness, you’re inherently attaching yourself to it. I’m sure the majority of people will understand it’s an AI modeled off her voice, but there will always be some that get more attached to it than they should. Just saying “I don’t think it’s harmful because people should know it’s not real” doesn’t magically solve the issue and cause everyone to behave.
Replika is another smutty type AI, but it’s clearly designated as such, and the avatar is pretty generic looking, which is more of what I’d want to see, at least for now. It being text-only also helps keep that separation. Who knows, it could end up fine, but this is closer to diving in the deep end before people have waded in to see what the water is like.
Exactly. Celebrities already have a problem with people forming unhealthy parasocial relationships with them when the interactions are unidirectional, having something answering back is going to make it much, much, worse. Hopefully no one is going to be seriously hurt in the aftermath of this money-grubbing stunt.
The slippery slope of “I know it’s not real but we’re so close and it’s based off of you so therefore you should love me too” does not feel like that much of a stretch
Yeah, and people got extremely attached to their Replikas. To the point that the subreddit stickied a suicide hotline number when the app removed sexual roleplay.
Imagine paying to talk to their chatbot, thinking it’s them.
Definitely a situation where the demand is more of a damning indictment than the fact that supply exists.
the conversation got even smuttier very quickly
lol I love the future.
It’s like Her but with exceptionally more horni.
She’s doing the same thing as always, taking advantage of vulnerable people while playing herself off as a victim. This is hardly news or something positive about AI chat bots…
Yeah, right, I don’t buy it.
Enough people will.
deleted by creator
This strikes me as very exploitative. Capitalizing on loneliness to enrich yourself gives me bad vibes (especially when the users of this thing may actually be worse off mentally in the end, as @cnnrduncan mentioned).
I have no doubts that for some users, it’ll turn into a cycle. They’ll feel lonelier each time they use it, which pushes them to use it more, and so on. I had the same feelings about Replika years before ChatGPT became a thing.
