An idea that once needed weeks before anyone could click through a demo can now be running by the end of the day. A capable model and a few good prompts can get a team surprisingly far.
That part is real. The bad inference is that product has become easy.
AI has made the cost of producing possibilities collapse. It has not made judgment cheaper. When almost any team can create a convincing demo, the hard question is no longer, “Can we build this?” It is, “Does this deserve a place in a user’s life?”
TL;DR
- AI has shifted the loop from spec-first to prototype-first, but it has not replaced the work of choosing the right problem.
- A prototype proves that something can exist. A product proves that people will return to it.
- When features become cheap, curation and coherent experience become the advantage.
- Do not measure success through demo speed or sign-ups alone. Measure the core action, outcome quality, and return behavior.
AI changed the order of the loop, not the standard for a product
Josh Elman’s Product Management is Still All About Telling Stories captures the inversion well. Teams once wrote specs, negotiated scope, and settled design before they spent scarce engineering time building. The rituals existed because every loop was expensive.
Now a team can build a rough version, use it, put it in front of a few people, and only then understand what the feature really is. That is a meaningful improvement. A prototype reveals the feel of an experience in a way that slides, roadmaps, and polished specs rarely can.
But build-first is not ship-first. A prototype should help a team ask a better question, not serve as proof that its first idea was right. If the first version does not change an assumption, reduce the scope, or sharpen the user problem, the team has learned little beyond the fact that AI can generate code.
Build speed is not proof of product value
Two studies on AI coding show why this conversation needs context. In GitHub’s controlled experiment, 95 professional developers using Copilot completed a narrowly defined JavaScript HTTP-server task 55% faster on average. It is strong evidence that AI can reduce time on a bounded task with a clear brief and clear evaluation.
METR’s RCT studied a different setting: 16 experienced contributors completing 246 real tasks in large repositories they had known for years. When they were allowed to use early-2025 AI tools, completion time increased by 19% in that setting. METR does not conclude that AI is useless. It explicitly warns against treating that result as representative of all software work.
Put those findings together and the lesson is not “AI is fast” or “AI is slow.” It is that AI output depends heavily on how clear the task is, how much implicit context exists, and whether “done” means a working demo or a change that meets the standard of a real product.
A prototype answers “Can it work?” A product answers “Should it exist?”
A prototype can answer every prompt beautifully. It can also make a team feel proud because it took only two days to build. Users do not live in the demo, though.
They arrive with a specific job to do and limited patience. If they do not know where to start, cannot trust the result, or use the tool once out of curiosity and never return, that is not product fit.
Take an AI assistant for website work. A demo may show it rewriting a homepage headline, creating a landing page, and fixing a visual bug in minutes. The harder product questions are different: What permissions will a business owner give it? Where do they review changes? When a website affects revenue, what makes an output safe enough to publish? Those questions are not solved by adding another capability. They require workflow, guardrails, and an experience that makes risk legible.
That is why product teams should keep two questions separate:
First: does this feature work?
Second: does it resolve a real enough tension that people will choose to come back?
The Product Manager is becoming the system’s editor
In many companies, PMs have been pulled into roadmap administration, requirements gathering, and protecting engineering capacity. AI makes part of that work lighter. It also creates a new problem: too many ideas now look reasonable enough to build.
That makes a good PM closer to an editor than a backlog manager. The job is not to write every line of code or invent every idea. It is to keep what matters and remove what makes the product’s story blurry.
An editor does three things AI cannot do on its own for a product:
They choose a user tension specific enough for the team to focus on.
They decide which capabilities serve the core promise and which ones are simply noise.
They make the landing page, onboarding, UX, and measurement tell the same story.
If a product promises, “Handle website work without technical expertise,” its first screen cannot be an empty prompt. It should lead the user to a real job: fix the homepage, check a checkout issue, update prices, or create a campaign landing page. The first valuable outcome should use the user’s own context. That is how a tool gets a chance to become a habit.
An empty prompt box is not onboarding
Many AI products start with “Ask me anything.” It sounds open. For most users, it is a hard assignment. They do not know what the product does well, what prompt to write, or what output they should trust.
Strong onboarding does not try to show every capability on day one. It teaches one concept at a time and leads the user to a core action. For an AI tool, that might mean uploading a file, choosing a goal, reviewing an initial result, and deciding whether to use or revise it. Users do not need a promise that the product can do everything. They need a clear reason to take the next step.
AI also creates a new form of product evidence: the transcript. A team can see what users ask for, how often they rephrase, where they stop, and what they expected instead. AI can surface patterns. It cannot decide which pattern deserves a product decision. A vague prompt could signal weak education, weak UI, weak positioning, or a need the product should not serve at all. Judgment is still the difficult work.
A new operating system for AI product teams
Step 1: Start with tension, not capability. State the job the user is trying to complete and the friction that keeps getting in the way.
Step 2: Prototype to learn. Write down an assumption that could be disproved. Do not build simply to validate your own idea.
Step 3: Watch the core action. Do not only ask whether people like the demo. See whether they can complete the important job without help.
Step 4: Cut before you scale. Remove anything that makes the product harder to explain, even if AI made it quick to build.
Step 5: Measure return, not applause. Sign-ups, demo reactions, and feature count do not tell you whether the product has entered a user’s routine.
This changes the PM’s role from writing a spec for the system to making clear what the system deserves to focus on. The work is not lighter. The responsibility is simply harder to avoid.
Conclusion: AI makes execution cheaper. Direction is still expensive.
AI will help more teams ship more software. It will also help more teams ship the wrong thing faster if they confuse a smooth prototype with a product that has earned a real place in a user’s life.
The durable advantage will not belong to the team with the most AI features. It will belong to the team that understands the moment a user is trying to solve, removes everything that distracts from it, and turns technical capability into an experience clear enough that people return.
When building becomes cheap, judgment is not a side task in product management. Judgment is product management.
FAQ
Does AI make Product Managers less important?
No. AI can reduce time spent on prototypes, early research, and documentation. Choosing the problem, understanding user context, making trade-offs, and maintaining a coherent experience still require clear ownership.
Should teams prototype before writing a spec?
Yes, when the goal is to reduce uncertainty about an experience. Before production, teams still need to define scope, reliability, data permissions, guardrails, operating cost, and a success measure.
Which metrics matter for an AI product?
Start with the core action tied to the product promise. Then track completion, outcome quality, return behavior, and why users abandon the flow. Adoption and prompt count are rarely enough on their own.