Why AI Agents Don't Actually Understand You — Danielle Perszyk, Amazon AGI Lab
Danielle Perszyk from Amazon AGI Lab discusses her vision for AI, emphasizing human collective intelligence and building agents that genuinely understand user intent. The conversation delves into the inadequacy of current AI systems, the necessity of real-time perception and social world models, and a reframing of AI alignment as optimizing for shared mental representations. Perszyk also explores the potential of AI to enhance human agency in education and counter homogenized thought.
- Early AI agent work, like Nova Act, focused on automating "atomic interactions"—think precise clicks and scrolls—to achieve reliability. Read →
- Current AI alignment often misses the mark: optimizing for task completion creates a 'whack-a-mole' problem, falling victim to Goodhart's law rather than fostering true intelligence. Read →
- AI subtly erodes agency: Research shows AI writing suggestions can shift a user's argument to an “opposing argument,” often below their awareness threshold, subtly manipulating their thought process. Read →
- Current AI development is stuck in a "local attractor state" focused on turn-taking chatbots and coding agents, a paradigm Danielle Perszyk of Amazon AGI Lab labels as fundamentally limiting human-AI interaction. Read →