IMPACT: An AI Product Design Framework for Deciding Where AI Belongs (and Where It Doesn't)

Every PM framework was built for software that doesn't surprise you. AI broke them.

CIRCLES, RICE, the design-question scripts you memorized for your PM interview. Every one of them assumes the same thing: you know what the product will do before you build it. You write the spec. Engineering builds it. The product does what the spec said, every time.

AI doesn’t work that way. It’s probabilistic. The same input can yield different outputs. It hallucinates. It drifts. And the most important decision in the whole design isn’t a feature at all. It’s whether AI belongs in the product, and where it has to stop.

No traditional framework asks that. So I built one that does.

I needed something I could run before I started building, something that would stop me from asking “what should I build?” and start me asking “what problem am I actually solving?” I couldn’t find it. So I wrote it, and now it’s the thing I teach my UW students and walk through in talks before anyone touches a tool. Because the fastest way to ship a bad AI product, and the fastest way to fumble an AI design question in an interview, is to assume the answer is always more AI.

It’s called IMPACT. Six layers. I’ll walk all six through one real example: a podcast automation I built and demoed live. It scrapes my AI news sources every morning, has Claude pick and summarize the top stories, turns that into a script, and sends it to ElevenLabs to be turned into audio. Six nodes. I built it in about two hours, my first time ever in n8n. Watch where AI actually shows up.

I — Intent: Name the Burden Before You Reach for AI

This is the layer everyone rushes. Don’t.

When I ask builders for the intent of their AI feature, I usually get something like “make people more productive” or “reduce manual work.” I’ve heard that one enough times to finish the sentence for them. It’s a wish, not a burden.

A real intent is specific enough to act on. It names the actual pain a real person carries. So before you propose anything, answer three questions about the problem itself: What’s the ambiguity AI is handling that rules can’t? What cognitive load is the person drowning in? And what’s the cost of being wrong?

For the podcast, my intent was concrete: the cognitive overload of scanning five-plus AI news sources every single morning. Same tabs, same synthesis, every day. The user wants a spoken digest without having to read it themselves. That’s specific, it’s repetitive, and it’s fully delegatable. Which is exactly why it’s a good automation candidate.

Notice what just happened. I haven’t named a feature yet, and I already know AI has a real job here, because the work is genuine synthesis a tired person repeats daily. If the intent had come back as “make my mornings better,” I’d have nothing to build toward. Vague intent is where most weak answers die in the first thirty seconds.

M — Mental Model: Decide What AI Touches and What Stays Human

This is the layer that makes or breaks AI products, and it’s the one no other framework has.

Once you know AI belongs, you don’t hand it everything. You map the journey and decide, step by step, where AI acts and where a human stays in the loop. Skip this, and six months later, you’ve got an AI doing things it was never supposed to do. Resolving billing disputes on its own. Handing out airfare discounts nobody approved. We’ve all seen those chatbots.

For the podcast, the split is driven by the cost of being wrong, and here that cost is low. A bad summary is mildly annoying, but not catastrophic. So AI can do most of the work, and I made the human review optional, a quick approval before the audio gets generated. That’s the right call when a mistake just means a so-so podcast.

Flip the stakes and the map changes completely. Put AI inside a medical or healthcare flow and you’re not shipping unless it’s 99.99 percent accurate, with a STILL human doing the final check before anything reaches a patient. Same framework, opposite answer. The judgment is in reading the stakes and drawing the line, and that judgment is the thing an interviewer is actually screening for. Anyone can add AI. Knowing where to stop is the senior move.

P — Plumbing: Start Deterministic, Add AI Only Where It Earns It

Now, and only now, the architecture. And this is where the podcast makes the whole point for me.

Six nodes. Pull the RSS feeds, merge them, run a code node to shape the data, send it to Claude to summarize and prioritize the top stories, run another code node to shape the script, then hand it to ElevenLabs for audio and drop it in Drive.

Look at what’s actually happening, tier by tier. The RSS reads, the merge, and the formatting nodes are plain rules, the same deterministic code you’ve always written. The two Claude calls, summarize and write the script, are GenAI. And the ElevenLabs voice is its own kind of AI, a machine-learning model that turns text into speech. So one tiny pipeline already spans three rungs of the capability ladder: rules for the plumbing, GenAI for the language, ML for the voice. And most of it sits on the cheapest rung.

That’s the discipline. You start with the lowest capability that works and climb only when it can’t do the job. Rules first. Then machine learning. Then generative AI. Then agents, then multiple agents, if and only if the problem demands it. The goal is using the right capability at the right moment, not building the fanciest category because it’s exciting. Almost every time, the simpler thing would have worked fine, and the complex thing just ships later, breaks in production, and gets killed six months in.

A — Accuracy & Safety: Design the Failure Before It Ships

Every AI system sometimes produces incorrect outputs. The question isn’t whether it will happen. It’s what the consequences are when it does, and whether you designed for them.

Here’s mine, from the real build. One morning, the script opened with “This is the news from March 24, 2024.” Excuse me. We were in March 2026, and nowhere near the 24th. The model just made up a date and said it with total confidence.

That’s exactly why the human approval node exists in my flow. Before any audio gets generated, the script pauses and shows me what it wrote so I can catch the thing that’s confidently wrong. I also keep the attributions and transcript links verifiable, so I can check that a story is real and not hallucinated. None of that is bolted on at the end. It’s a product decision I made up front, because I already knew the model would be confident even when it shouldn’t be.

C — Cost & Constraints: Don’t Build a $100 Solution for a $5 Problem

Proof of concepts are cheap now. Lovable, Bolt, a dozen low-code tools will get you a working demo fast. The cost question bites later, when you turn that POC into something real and start paying for infrastructure at scale.

For the podcast, I’d run tests for a while and my Claude API bill was about $2.21. Skip one coffee and you’ve covered it. But that’s the POC. The number to watch is what it costs when something works and you scale it up, because that’s where teams burn their runway solving cheap problems with expensive tools. Match the spend to the stakes. A $5 problem doesn’t get a $100 solution.

T — Tracking: The Layer Software Never Had

The AI product lifecycle has one thing the software lifecycle never did: monitoring and maintaining.

You don’t launch and walk away. Once the model is live, you watch what the data tells you. Is it drifting? Does it need retraining, and how often? For the podcast, tracking means the stories stay fresh and I’m not getting duplicates day to day. For a bigger product, it means watching for drift and rebuilding the model as the data moves under you.

And track the promise, not the vanity metric. Engagement is a trap. Time spent could mean the thing is working or could mean it’s broken in a way that makes people keep poking at it. Measure whether you actually solved the burden you named back in Intent.

The One Thing to Take Into Your Next Build, or Your Next Interview

If you remember nothing else, remember the move that makes IMPACT different from every framework before it.

The first real decision in any AI product once you have figured out what to build is whether AI belongs at all, and where it has to stop. My podcast proves it. Six steps across three tiers of the capability ladder, and the real cognitive work lives in only two of them. Most of the rest is the cheapest rung, the deterministic plumbing you’ve always known how to build.

Traditional frameworks skip that question because the software they were built for doesn’t surprise you. AI does. So when you sit down to build, or an interviewer asks you to design an AI product and you open by mapping which tasks go to the model and which stay human, you’ve already shown the thing that matters. Not enthusiasm for AI. Judgment about it.

That’s what I teach before anyone in my class touches a tool. Build the muscle for knowing when to keep AI out, and the rest of the framework does its job.

The podcast was deliberately low stakes. A bad episode is annoying, nothing more. So below, I run the same six layers on a product where being wrong means a hospital visit, an allergy-aware restaurant finder I built for a Demo Day. Same framework, real stakes, and the actual output each layer should produce so you can run it on your own product.