If you are building AI products, one of the most important design decisions is also one of the least glamorous: what your product does when it does not know the answer. Most general-purpose models lean toward a confident reply. As a founder, you inherit that tendency unless you design against it.
This guide treats "I don't know" as a product requirement, not a model flaw, and walks through why it matters for trust, how to specify it, and how to test it before launch.
Why your model defaults to guessing
Four forces push a model toward a confident answer, and all four apply to whatever you build on top of it:
- Next-word prediction. Models write by choosing a likely next word. "I don't know" is rarely the likeliest continuation.
- Scoring that rewards guesses. OpenAI research in 2025 argued that many AI tests work like a quiz where blanks lose and lucky guesses win.
- Human preference for confidence. In training, raters usually favor a sure-sounding answer over "I'm not sure."
- No built-in self-awareness. Sounding sure and being right are different things, and the model cannot reliably tell them apart.
The practical lesson: do not assume honesty is the default. You have to build it in.
Two kinds of "I don't know" to design for
The first is off-topic. A health companion asked who leads a country should stay in its lane and say it has no good answer. This is the easy case, and a narrow, well-defined knowledge scope handles it.
The second is on-topic but missing pieces, and it is where products earn or lose trust. Three examples:
| Scenario | Product that guesses | Product that admits limits |
|---|---|---|
| Patient asks an insulin dose | Gives a confident number | Says it cannot work that out safely and points to the care team |
| Marketer asks a competitor's pricing | Invents plausible prices | Says it found no current pricing and shows where to check |
| User asks a local sales tax rule | States an out-of-date rule | Says it is unsure and points to the official source |
In each case the honest product has to recognize what inputs are required, notice they are missing, and stop or hand off.
A practical build checklist
Good products tend to combine four habits:
- Ground answers in trusted sources. An open-book approach lets the system tell when a question falls outside what it has.
- Permit and reward abstention. Write evaluation criteria so a correct "I don't know" scores better than a wrong guess.
- Double-check. Ask the question twice and compare the answers. Disagreement is a signal to hold back.
- Hand off to a person. Define clear escalation paths for high-stakes questions.
For the grounding concept in more depth, see the original explainer, Why won't AI just say "I don't know"?.
Turn it into an evaluation habit
Before launch, build a small test set that includes questions your product should refuse or escalate. Score abstentions explicitly rather than treating them as failures. If your only metric is the share of questions answered, you are training your team, and possibly your system, to reward guessing, the same trap that benchmarks create.
Also watch for sycophancy. A Stanford-led study in Science in 2026 found that across eleven models, chatbots backed the user's actions 49 percent more often than people did. If your product advises users on decisions, test whether it simply agrees, and build in a path to a human second opinion.
Why this matters for fundraising and go-to-market
Buyers in regulated or high-stakes settings ask the same three questions you should be able to answer about your own product:
- What does it do when it does not know?
- Does it show its sources?
- Can it hand off to a person?
Clear answers make a stronger pitch than a claim of being always right, which sophisticated investors and customers will not believe anyway. Treat honest uncertainty as a feature you can demo.
If you want a broader view of how responsible practices fit into AI development, the team behind this resource shares more on the main website.
The takeaway
A product that says "I don't know" at the right moment is not a weaker product. It is one users can rely on. Specify it, reward it in testing, and make it part of your launch criteria from day one.