Blog

AI requirements gathering invented three things nobody said

Four habits that turn AI from a writer of invented requirements into a structurer and gap finder for your interview notes.

Samip Shah Nov 19, 2026 6 min read
ai-requirements business-analysis hallucination requirements-traceability source-grounding stakeholder-interviews
A person writing in an open notebook with a pen, taking handwritten notes
Photo by Kristin Hardwick via Wikimedia Commons (CC0).

The requirements looked perfect, and three of them were never said in the room

I handed AI the raw notes from a stakeholder interview and asked it to write the requirements. What came back looked clean, numbered, and professional. Three of those requirements had never been said in the room. The model had filled the gaps with what requirements usually look like, not with what the stakeholder meant.

That is the whole story, and I am leaving it bare on purpose. The output read like something I would have been happy to write myself. Every line had the right verbs and the right tone. Nothing on the page warned me which three were invented. AI requirements gathering is easy to start and easy to get wrong. Here is what I learned about the second part.

Why would a tool invent things when I handed it the notes myself?

That was my first question too. The source was right there, pasted in. Why did it need to make anything up?

The answer, as I understand it, is pattern completion. A gap in my stakeholder interview notes is not a blank to the model. It looks like a spot waiting to be filled, and the model fills it with the most likely text, which is what requirements in that situation normally say. Fabrication is the same family of failure I wrote about in the hallucination post, but this version hides inside a document you supplied yourself.

Turning interview notes into requirements is exactly where this bites. Notes are thin in places. Four habits fixed it for me, and they come after the diagnosis.

It behaves like a note-taker who has sat through a hundred similar meetings

Picture a minute-taker who has attended so many meetings of one type that they can predict the decisions before anyone speaks. Hand that person your sketchy stakeholder interview notes and ask for the formal minutes. They will not leave blanks. They will write what such meetings usually decide.

The minutes will be fluent and tidy. They will also be wrong in exactly the places where your notes were thin, because that is where the habit takes over from the record. The model has no memory of your room. It has a memory of a hundred rooms like it.

The research on fluent but unfaithful text explains the pattern

Three pieces of reading helped me, and each one maps to a habit later in this post.

First, unrequested content is a known failure class. In the abstract of a survey of hallucination in natural language generation, natural language generation is prone to hallucinate unintended text, which degrades the system performance. So this was not my bad luck. It is a documented behavior of the technology.

Second, long input hurts. The Lost in the Middle paper reports for language models working over long contexts that performance is often highest when relevant information occurs at the beginning or end of the input context. I will not pretend that gives a magic note length. It does tell me not to hand over a whole interview in one pass. I covered the same drift in the information-overload post.

Third, vendors build pointers back to the source into their own products. In its announcement of the Citations feature, Anthropic says: Claude can now cite specific passages from your documents, delivering verifiable responses with built-in source tracking. That is a vendor describing its own feature, so treat it as a design intent, not independent proof. Still, the idea is useful. If the tool builders think a pointer to the source is worth building, a human BA can demand one by hand, with no special feature at all.

If I have to check every line anyway, what is the AI even saving me?

This is the strongest objection, and I take it seriously. If I must verify everything, the time saved might vanish. Ethan Mollick's guide to practical AI use puts the burden bluntly. You will need to check it all. I will not argue with that. The cost is real, and I have no measured time saving to wave at you.

My narrow answer is that the checking changes shape. Reading a fresh document against my memory of the room is slow. Matching each requirement to a quoted line is fast, like ticking a list. And the real value of business analyst AI work, for me, is not prose. It is structure and gap finding. Those are jobs where a second pair of eyes helps, and where I stay the one who decides.

Four habits turned the same job into a structured and checkable one

I did not rerun the original three requirements with these. The habits below are what I do now, and the sample wording is illustrative, not a replay of that day.

1. Ground it. Paste the notes and tell the model to use only those notes. If a point is not in them, it must not appear. This is the same idea as the data-is-the-fuel post, applied to interview notes to requirements work.

2. Split the job. Do not say "write the requirements." Run three steps. First extract every statement the stakeholder made. Then group them by theme. Only then draft requirements from the groups. Each step is short enough to inspect.

3. Add a traceability column. Every requirement carries the exact quoted line from the notes that supports it. No quote means the requirement goes back. This is how you make ChatGPT quote the source for each requirement, and it is the same instinct behind requirements traceability in any project.

4. Run an open-questions pass. Ask what the notes leave unclear, unsaid, or contradictory. This is how to use AI to find gaps in stakeholder interview notes. The model's urge to fill gaps becomes useful once you point it at the gaps and tell it to list them instead of filling them.

An illustrative prompt, using Role / Objective / Background / Data lightly:

  • Role: You are a business analyst assistant.
  • Objective: List requirements, each with a quoted line from my notes.
  • Background: These are my own notes from one interview.
  • Data: Use only the notes below. If a point is missing, write "not stated".

Even a quoted line can mislead you if you stop reading, or paste the wrong notes

Two honest warnings.

First, a traceability column can be trusted blindly. A quote can be real and still be stretched into a requirement it does not support. The line says the stakeholder wants a report, and the requirement says the report must be exported hourly. So I still match each line to the notes, and the quote does not replace my reading.

Second, data hygiene. Notes that go into a cloud model must be stripped of sensitive customer data first.

I stay behind the wheel. I have never pasted sensitive customer data, IRB submission IDs, or other regulated material into the cloud. That rule is non-negotiable in pharma IT.

Generalise the notes, remove names and identifiers, then paste.

The sign-off is still mine, and that is the point

I still read every requirement against the notes, because I stay behind the wheel and the sign-off is mine.

So here is my question for you. The next time you finish a stakeholder interview and reach for AI to write it up, which three lines will you check first?

← All posts