

That would be my bet, LLMs really gravitate towards playing along and continuing whatever’s already written. And Gemini especially has a 1M long context so it could be going back for a book’s worth of text and reinforcing it up the wazoo.
That said, there is something really unhinged about Google’s Gemma series even in short conversations and I see the big version is no better. Something’s not quite right with their RLHF dataset.



Send Vance to every country to campaign for the far right, we welcome him with open arms ;)
We’ll have to hide all couches but it’ll be worth it in the long run.