Am I missing something? The article seems to suggest it works via hidden text characters. Has OpenAI never heard of pasting text into a utf8 notepad before?
Am I missing something? The article seems to suggest it works via hidden text characters. Has OpenAI never heard of pasting text into a utf8 notepad before?
Remember they’re doing this so that they can detect it themselves. I’m far from an expert, so maybe I’m misunderstanding something but the way I understand it, they’d be defeating their own tool if they go down this route. If they cycle the logit biases, how can they themselves detect if a random piece of text is generated? Which set of biases do they test?
At the end of the day, you’re talking about raw text. There’s no option to sign it, or embed metadata or anything like that. You can’t even guarantee that you’re seeing the complete sample, or even a single sample! If there is a fingerprint, it’ll be detectable to anyone, and it’ll be easily removed.
They can cycle a some biases (dozens?) and test them all. Detokenization is super cheap to run, its not AI or anything.
I’m trying to think of a good analogy for how this would work, and I kinda came up with one. This would be kinda like an image encoder that biases itself towards coding RGB values (0-255) as even numbers. Subtly, say 30% odd 70% even.
That’s totally imperceptile to humans. And even a “small” sample of the image would carry this bias if pasted into a larger image verbatim, since the sample size is so large (just as the sample size for a bunch of tokens in text is pretty big.
And I’m not saying its fullproof… but if thats indeed what they’re doing, I think its a decent way to detect “lazy” OpenAI abusers who aren’t working so hard to scramble and defeat it.