AI has a secondary intent problem. I was negotiating a rent renewal (I live in N
By Allie K. Miller · May 15, 2026 · Curated by George's Blog
AI has a secondary intent problem.
I was negotiating a rent renewal (I live in NYC and they raised rent 10% because I guess they were bored). The property manager said she would "do everything she could" to get me a better rate. Claude celebrated and assumed a meaningful discount was coming. I had to tell it she was saying that to save face and was probably not going to move much (reader: I was right).
Primary intent = the words.
Secondary intent = the subtext, what the words actually mean in context.
I honestly think it's one of the reasons that AI is a mediocre TV and movie writer.
We don't want to train AI that all humans are manipulative. At worst, we would have models that assume the worst and distrust everything; at best, they would become less helpful.
But we do need it to understand that polite phrasing, face-saving language, and soft no's are real categories of human speech and might be deceptive. And missing language, emotion, and context cues has real-world consequences.
Big impact zone: literally everywhere AI interacts with people. Customer support, sales, BD, any system helping leaders make decisions or calls about human behavior.