Why "It Depends" Is the Wrong Answer to a Trade-Off Question

I was recording a demo video for our own product last week. My tool, my question bank, no stakes at all. The interviewer asked me about a trade-off I have genuinely made dozens of times, and I gave one of the weakest answers of my professional life. Here is the transcript, unedited, because the failure is more useful than a tidy example would be.

The question:

You've shipped RAG pipelines across clinical support and internal knowledge surfaces. Tell me about one of those. Walk me through a specific situation where you had to make a real trade-off between retrieval accuracy and latency or cost. What did you choose and why?

My answer:

So it depends. The trade-offs between accuracy and latency, it really depends on the type of problem you're trying to solve. In some places you want speed, in some use cases you want accuracy, like in medical, you don't want the wrong data. But let's say you're playing some game, sometimes you just want speed. So it really depends.

The follow-up:

That's a good instinct, but I want to pin down the actual decision. So you had a RAG system where tuning up top-k improved recall, but added latency and cost. Walk me through that specific call. What was the retrieval setup, how did you measure the trade-off, and what numbers did you actually see when you changed it?

That follow-up is the whole article. Everything I said was true, and none of it answered the question.

Why the "it depends" reflex is so strong

It is not laziness, and it is usually not ignorance. It comes from three real places.

It is genuinely true. Engineering answers are contextual, and senior people are trained to resist context-free advice. Saying "always choose accuracy" would be worse. So the honest instinct is to establish the context first.

It buys thinking time. A trade-off question is open-ended and slightly threatening. Talking about the general shape of the problem while you search your memory for a specific example feels safer than silence.

It is what technical writing rewards. Blog posts, documentation, and conference talks all open by framing the space before narrowing. That habit transfers straight into the interview, where it is exactly wrong.

The problem is that the interviewer is not evaluating whether you understand that trade-offs exist. They asked about a specific situation, in the past tense, with "what did you choose" attached. They are evaluating whether you have actually stood at that fork and picked a direction.

What they are really probing

Read the follow-up again and notice how precise it is:

What was the retrieval setup, how did you measure the trade-off, and what numbers did you actually see when you changed it?

Three separate probes, and each one is checking a different thing.

"What was the setup" checks whether the situation is real. Invented examples get vague at exactly this level of detail, because inventing a plausible architecture on the spot is hard.

"How did you measure it" checks whether you are an engineer or a person with opinions about engineering. If you changed something and did not measure it, you did not make a trade-off, you made a guess.

"What numbers did you see" checks whether you were still around after the decision shipped. People who leave before the consequences arrive never have this data.

You do not have to answer all three perfectly. But an answer that touches none of them tells the interviewer the same thing regardless of how confident you sounded.

The four-part structure

Here is the shape that works, in the order that works.

1. The decision, in one sentence. Start with what you chose. Not the framing, not the context, the choice. "I capped top-k at five and put the effort into chunking instead."

2. The constraint that forced it. One sentence on what made this a real trade-off rather than a free choice. Budget, a latency SLA, a compliance requirement, a deadline. If nothing was constraining you, it was a preference, not a trade-off.

3. What you measured. Name the metric and, ideally, the number. Recall at k. P95 latency. Cost per thousand queries. This is the part candidates skip and it is the part that lands.

4. What it cost you. The downside you knowingly accepted, and how you covered for it. This is the strongest sentence in the whole answer, and almost nobody includes it.

Notice this is not the STAR method. STAR is built for behavioral questions about people and situations, and it front-loads context. A trade-off question wants the decision first and the context second, because the decision is the thing under test.

The answer I should have given

Same example, same facts, restructured:

I capped retrieval depth at top-five on the clinical support system, and spent the effort on chunking and metadata filtering instead of pulling more documents.

The constraint was latency. We had clinicians using this between patients, and anything over about two seconds meant they stopped using it and went back to searching the intranet manually.

Going from top-five to top-twenty measurably improved recall on our eval set, so on paper it was the better system. But it pushed us past that latency budget and roughly tripled our token spend per query, because every retrieved chunk goes into the prompt.

So I took the recall hit deliberately and tried to win it back somewhere cheaper. Better chunk boundaries and filtering on document metadata got most of it back without adding any latency at all. The part I did not get back was long-tail queries where the answer genuinely was in the fifteenth document. We logged those and reviewed them weekly rather than pretending they did not exist.

That is roughly ninety seconds. It contains one decision, one constraint, one measurement, one accepted cost, and one honest limitation. It is not a better story than the one I actually had. It is the same story, told as a decision instead of as a category.

This is not only a technical interview problem

The same failure shows up everywhere the question contains "how do you decide":

• How do you prioritize a roadmap? "It depends on the business goals" is the "it depends" of product management.
• How do you handle an underperformer? "Every situation is different" is the managerial version.
• How do you balance quality and speed? This one practically invites the non-answer.

The fix is identical in all of them. Pick one real instance, lead with what you actually did, and name what it cost.

How to practice this specifically

Reading about it will not move you, because the reflex fires under mild pressure and you cannot simulate mild pressure by nodding at an article. Two things that do work:

Build a decision inventory. Separately from your story bank, write down six decisions where you picked one thing over another and something was lost. For each, force yourself to write the number you measured and the cost you accepted. Most people discover they remember the decision but not the evidence, which is exactly the gap the interviewer is going to find.

Rehearse against something that follows up. This is the part static prep cannot give you. Reading a list of common questions never produces the second question, and the second question is where "it depends" gets exposed. You need to give the weak answer out loud, get pinned on it, and feel that specific discomfort at least once somewhere it does not count.

I built the thing that pinned me, and it still pinned me. That is not a testimonial, it is just the mechanism working: a question generated from a real job description, an answer that dodged it, and a follow-up that would not let the dodge stand. The uncomfortable version of that conversation is much better to have on a Tuesday than on the day it decides your salary.

---

FAQ

Is "it depends" ever an acceptable interview answer?

As an opener, briefly, yes. As the whole answer, no. If you genuinely need to establish a fork before answering, take one sentence to name the two conditions and then immediately commit to one and answer for it. "It depends on whether latency is user-facing. In our case it was, so I capped retrieval depth." That takes four seconds and then delivers an actual decision, which is what was asked for.

What is an interviewer actually testing with a trade-off question?

Whether you have personally made the call and stayed around for the consequences. Naming the competing concerns shows you have read about the problem. Naming what you measured, what you chose, and what it cost shows you have shipped it. The question is designed so that only the second kind of experience produces a detailed answer.

What if I genuinely do not have a specific example?

Say so plainly, then give the closest real thing you have, clearly labelled. "I have not made that call on a production system. The nearest I have is a side project where I hit the same wall, and here is what I did." An honest smaller example beats an invented larger one every time, because the follow-up questions will find the invention. This is also worth reading alongside what to say when you have no experience.

Why do interviewers keep asking follow-up questions after I answer?

Usually because the first answer described a category rather than an instance. A follow-up that asks for setup, measurement, or numbers is the interviewer trying to move you from the general to the specific. It is not hostility and it is not a trap. Treat a follow-up as a signal that the useful part of your answer has not arrived yet.

Should I use the STAR method for trade-off questions?

Not directly. STAR front-loads situation and task, which delays the decision to the third beat, and the decision is what is being assessed. Lead with the choice, then the constraint, then the measurement, then the cost. Keep STAR for behavioral questions about conflict, failure, and collaboration.

How do I include numbers if I cannot share confidential metrics?

Use ratios, orders of magnitude, or direction with a shape. "Roughly tripled our token cost", "cut p95 by about a third", "recall improved by a few points and latency doubled". No interviewer expects exact figures from a previous employer, and a well-shaped approximation demonstrates you knew the numbers at the time, which is the actual point.

What is the fastest way to fix this habit before an interview next week?

Pick your three most likely trade-off questions for the role, write a four-line answer for each in the decision, constraint, measurement, cost order, and then say each one out loud to something that will ask you a follow-up. The writing fixes the structure and the follow-up is what stops you reverting to "it depends" under pressure.