To belabor the chess analogy: I would say a chessbot didn’t work if it randomly caused pieces to appear. Or if it made exceedingly lousy moves. You’d apparently say it was working because it technically changed the board.
Literally nobody is saying the token predictor isn’t predicting token. It’s just predicting wrong token, which normal people call “not working,” while tech evangelists prefer to call it “hallucination” or “misalignment” depending on the narrative they’re aiming for.
The goal of the token predictor is to produce coherent language - not factual information. If you can understand what it’s saying, it’s working - even if the content of what it says is factually inaccurate.
To belabor the chess analogy: I would say a chessbot didn’t work if it randomly caused pieces to appear. Or if it made exceedingly lousy moves. You’d apparently say it was working because it technically changed the board.
Literally nobody is saying the token predictor isn’t predicting token. It’s just predicting wrong token, which normal people call “not working,” while tech evangelists prefer to call it “hallucination” or “misalignment” depending on the narrative they’re aiming for.
The goal of the token predictor is to produce coherent language - not factual information. If you can understand what it’s saying, it’s working - even if the content of what it says is factually inaccurate.