This is what AI should actually be used for. Strengthen the algorithms that identify it to reduce the load humans need to review, and it should hopefully be more manageable
AI flagged my VR controller as a gun on Facebook and my account received a 30 day ban
I hope they win their lawsuit. I listened to a Radio lab episode a few years ago about FB moderators. The shit they have to see day in and day out sound absolutely horrible. Pics and videos of extreme violence and child pornography sounds like it’d give any normal person some major trauma.
Exposed to a firehose of the worst humanity has to offer. I can’t even imagine
Go on the Reddit front page or Twitter home page with a heart rate and blood pressure monitor and scroll for 10 minutes.
You will find that in almost every instance, both measures go up. The whole point of social media is to agitate because agitation correlates with engagement which correlates with mucho ad dinero.
One time on Reddit, a mod of /r/askhistorians described some of the content of this sort that he had seen, and he wasn’t as dispassionate about it as this article is. Just his verbal description is both disturbing and difficult to forget, so I can believe that these employees are traumatized.
With that said, what about other careers that expose people to disturbing things? I used to know a pathologist who once told me that she had a bad day because two infants died during childbirth at the hospital where she worked. She had to autopsy them. I didn’t know her well at the time so of course I assumed that she was upset for the same reason that such direct exposure to the death of babies would upset most people, but I was wrong. She was upset because she had to work late.
Why can pathologists do their job without being traumatized? Maybe the difference is that pathology isn’t something that a guy off the street just gets hired to do one day. The people who end up being pathologists usually have other options, and they choose pathology because it doesn’t particularly bother them for whatever reason. Meanwhile these moderators are immediately dumped into the deep end, so to speak, and they may not be financially secure enough to leave the job even after they experience what it is.
Can content moderation be done without traumatizing people? It isn’t a high-skilled, well-paid job so I don’t think filtering candidates the way that pathologist are filtered is practical. Not having content moderators also isn’t practical.
(I’m using pathology as an example because that’s what I know a little about, but I think my statements are probably valid for other careers, like homicide detective, which also involve regular exposure to disturbing things.)
It’s definitely to do with work conditions. I’ve been a paramedic for fifteen years, and suffice it to say, I’ve seen (and smelled and heard) some shit. I’ve always felt that I had a harder time processing the stuff from when I worked in a busy metro system and we had to go from coding a kid who drowned just fifteen feet from a party full of adults to holding grandma’s hand and making her warm and comfy on our five minute jaunt to dialysis to “hey, there’s a car on fire and bystanders report hearing screaming from the vehicle”. I would regularly get three hours of sleep over a 72 hour period and have almost no time to process the horrible shit we saw while still having to be a functioning, caring professional for every patient along the way. The also horrible shit we saw in the slower rural area I worked in has haunted me a lot less. There’s probably more to the whole picture than that, but I’m confident that work conditions are a huge factor.
In my other comment, I mentioned a Radio Lab episode that discussed it. The main problem is like you say, they take people off the street and offer little to no training. Additionally, they don’t offer any mental health resources for their employees. Their pay is also pretty awful. The turnover is extremely high for these and many other reasons.
That’s racist
Look… this is going to sound exceedingly stupid… but shouldn’t they find a way to use convicted sex offenders to filter out CSAM? They are not going to be traumatised by it and it saves some other poor soul from taking the brunt. How do you motivate them to actually do it? Well first one has to flag, and a second one earns a bonus every time the first one flags wrong. Motivation aligned!
</joking… but seriously though…>
