There’s a version of AI literacy that’s entirely theoretical. You follow the newsletters. You watch the demos. You understand conceptually what agents do, what context windows are, how agentic workflows compose.
And then you sit down with an actual agent and discover that none of it prepared you for the moment when the output is wrong and you don’t know why, or when the agent does something you didn’t ask for and you’re not sure if it’s right, or when context starts to drift halfway through a long session and you have to figure out how to recover it.
Reading and watching videos about how AI works can only get you a map for orientation. Building products gives you a mental model of how AI augments your capacity, and leads you to solve problems that affect people’s lives.
Directing, then evaluating
Only by working with an agent do you understand the difference between directing and doing. When I started, I was trying to review every piece of code the agent produced before accepting it. By the end I was directing: deciding whether the AI was going in the right direction, whether it met my intent, whether it held up — not just against the automated tests, but in how it felt when I ran it.
The second shift is where judgment goes. In the old workflow, most of it went into specifying exactly what I wanted. With an agent, specification is just the starting point. Judgment moves into evaluation: is this what I meant, does this actually serve the user, should I accept this or push back?
None of that shows up in a tutorial. It’s what you understand after you’ve built something real and watched it fail in ways you didn’t expect.
Also in August: Build to Learn AI · The Sandbox Is How You Learn · The Skill Under the Skill
Daniel Arevalo is a Product Architect and strategic product design leader. He helps product teams turn AI capability into trusted, adoptable workflows, with a background in industrial AI, IT/OT cybersecurity, and high-stakes enterprise software for critical infrastructure.