A screwdriver beat a hammer in a screw driving competition. well done.
Maybe it sounds impressive when hammers are super hyped and everyone and their mother is driving their screws with them, but it honestly really isn’t.
Gotta remember that AI isn’t occasionally hallucinating and often recalling information, it’s never recalling information, it’s always hallucinating, just that we say we like some of the hallucinations, so it does those more often.
Well, not sure what you mean by hallucinating, but there are some LLMs (chatgpt3.5-turbo-instruct) that could play chess at a good amateur level (1750 elo), and by studying them, some people found parameters in the model that seemed to represent the current state of the chess board (as in, they corresponded to the game, and changing them artificially made the LLM play as if the game as always been that way).
it’s never recalling information, it’s always hallucinating
With the exception of information you have put within the context window
https://generativeai.pub/what-is-the-context-window-ais-memory-limit-ce8a966f954a
This is incredible. They’ve finally made an AI as good at chess as me.
K go sit in that corner and play with yourself lol
What if I told you everyone involved knew this was bullshit but they wanted people to talk about them
TIL that I play chess like an AI model.
What’s ChatGPT’s rating? It’s impressive it can play chess at all, considering that’s not its core skill.
I wouldn’t be surprised if it’s literally zero. I’ve tried with a few LLMs, and they’re all very confident that they know how to play chess, but they just start hallucinating illegal moves immediately.
Immediately? When was the last time you tried? The newer models can hold a game well for 10-20 moves.
A few weeks ago, Gemini got confused when it tried to go first as black multiple times, so that’s the most immediate one I can remember. Last week, chatGPT offered to set up chess puzzles for me, but it made mistakes 3 out of 3 times.
Maybe I’ll try again. Is there a certain one you’ve seen good performance out of?
Quick! Stop it before it takes over the world!
To be fair the Atari was built to play games. /s
But for real wasn’t AI supposed to be bare minimum good at this game. Is this not how were to train them in order to know if they are intelligent or not?
