Test-Driven AI Coding Interview Prep

How three approaches to testing AI-generated code behave on the things that actually break in production

Sample interview questions

1. An engineer asks an AI tool to "add tests" for a function it just wrote. The tool calls the function, captures the value it currently returns, and asserts the result equals that value. Coverage jumps to 95%. What is wrong with this test?
   • A. It is circular: it asserts the implementation matches itself, so it can only fail if the function later changes — it never checks the behavior was correct in the first place. ✓
   • B. Nothing is wrong; pinning the current return value is exactly what a regression test does, and 95% coverage confirms the behavior is verified.
   • C. The only problem is line coverage being below 100%; raising coverage to 100% would make the assertion meaningful.
   • D. The test is too slow because it executes the real function instead of a mock, so it should stub the return value instead.

   Option A is correct because a test generated by observing the implementation's current output and asserting equality to that output is a tautology. The assertion is "the function returns what the function returns," which is true by construction. It detects a future change to the output, but it can never tell you the original output was correct — if the function was wrong on day one, the test enshrines the wrong answer as the expected one. This is a clear example of why test quality, not coverage, is what matters when a model writes tests against its own code: coverage rises while verification does not. Option B is wrong because it confuses a characterization test with a correctness test. Pinning the current value is legitimate when you are deliberately freezing known-good behavior before a refactor, but here the value was never independently established as correct — it was simply whatever the freshly generated, unverified code happened to emit. A regression test is only meaningful once the behavior it pins has been validated against the requirement, not against the implementation. Option C is wrong because line coverage measures which lines executed, not whether they did the right thing. Pushing a tautological suite from 95% to 100% multiplies the same empty assertion; it adds no behavioral guarantee. Option D is wrong because executing the real function is the right call, not a performance flaw — stubbing the return value would make the circularity worse by removing even the chance of exercising real logic. The defect is the source of the expected value (the implementation), not the speed of the test.

2. A teammate argues that test-driven AI coding works well for application code but is "philosophically the same problem" as evaluating an AI-written marketing summary. Why does the code case behave differently?
   • A. Code has an oracle — a test executes and returns pass or fail deterministically — whereas summary quality has no oracle and every verdict is a fallible judgment that must itself be justified. ✓
   • B. There is no real difference; both require a human reviewer to read the output and decide whether it is acceptable.
   • C. Code is easier only because language models are better trained on code than on prose, so the generated output is more often correct.
   • D. Summaries are actually easier to verify because an LLM-as-judge gives an objective score, while code correctness can only be guessed at from the diff.

   Option A is correct because the distinction is the presence of an executable oracle — a procedure that mechanically checks the expectations you encoded, without an opinion. For code, the test is that oracle: it runs and yields an unambiguous pass or fail on the cases it encodes, and you can run that same check across many cases automatically (at the cost of writing and maintaining it). Summary quality, translation faithfulness, or design taste have no comparably crisp procedure; correctness there is a claim that some fallible human or model has to argue for, and that argument can be biased or wrong. This is exactly why test-driven AI coding is tractable: code is the regime where acceptance can largely be mechanized — though a passing test confirms only the encoded expectations under the test environment, not total correctness or that the requirement itself was right. Option B is wrong because it erases the property that makes the code workflow reliable. A human is needed to decide whether the test encodes the right requirement, but once that test exists, accepting or rejecting a given diff against it is automated and repeatable — not a fresh human judgment per change, the way a summary is. Option C is wrong because model training quality is irrelevant to the argument. Even a model that is excellent at prose still leaves you without a deterministic check on the prose; even a mediocre code model still produces output you can subject to an executable test. The asymmetry is structural, not a function of how good the generator is. Option D is wrong because an LLM-as-judge is not an executable oracle — it is another fallible model with documented position, verbosity, and self-enhancement biases, strongest precisely on specialist content. Treating its score as objective is the mistake. Code correctness, by contrast, is not "guessed from the diff"; whether the code meets the encoded expectations is determined by running the test.

3. You are fixing a bug an AI agent introduced. You want a regression test that genuinely guards against the bug recurring. What sequence actually proves the test is doing its job?
   • A. Write the test, confirm it FAILS against the buggy code, then apply the fix and confirm it PASSES — the fail-before/pass-after transition proves the test detects this specific defect. ✓
   • B. Apply the fix first, then write a test against the now-correct code and confirm it passes; a green test on fixed code is sufficient proof.
   • C. Write a test that passes immediately against the current code so the build stays green, then note the bug in the commit message.
   • D. Increase coverage on the surrounding module until the line containing the bug is executed by some existing test.

   Option A is correct because a regression test only earns trust if you have watched it catch the bug. The fail-before step proves the test actually exercises the defective behavior and discriminates against it; the pass-after step proves the fix resolves exactly that behavior. A test that was never observed failing might pass for an unrelated reason — wrong assertion, wrong code path, a typo in the expectation — and silently provide no protection. The fail→fix→pass transition is the cheap experiment that converts "I wrote a test" into "I have evidence the test guards this bug." Option B is wrong because writing the test only after the fix means you never saw it fail, so you have no evidence it would have caught the original bug. This is the same circularity trap as a self-generated assertion: the test is shaped by the corrected implementation and may pass for reasons that have nothing to do with the defect. Option C is wrong because a test deliberately written to pass immediately guards nothing — it cannot fail on the bug it is supposed to cover, so it is decorative. Keeping the build green is not the goal; encoding a failing condition that the fix flips to green is. Option D is wrong because executing the buggy line is not the same as asserting the correct behavior of that line. Coverage tells you the line ran; it does not encode what the line should produce, so raising coverage around the bug does not create a regression guard.

4. An AI-generated date-parsing utility passes every example-based test you wrote. In production it crashes on leap-day inputs and empty strings you never thought to enumerate. Which testing approach most directly targets this class of failure?
   • A. Property-based testing: specify invariants the output must always satisfy and let the framework generate many inputs, surfacing edge cases neither you nor the prompt enumerated. ✓
   • B. Add more example-based tests by hand until you happen to list leap days and empty strings explicitly.
   • C. Raise the line-coverage threshold so CI forces more of the parser to execute.
   • D. Re-run the same example tests under a mutation-testing tool to mutate the production code.

   Option A is correct because the failure is an enumeration gap: both you and the model only thought of the inputs you thought of. Property-based testing attacks that directly. Instead of listing inputs, you state invariants — for a date parser, something like "parsing then formatting round-trips to the same value," or "any valid ISO string parses without throwing" — and the framework generates and shrinks hundreds of inputs hunting for a counterexample. That is exactly the regime where AI-generated code is weakest, because the prompt that produced it shared the human's blind spots about leap days and empty strings. Option B is wrong not because it is useless but because it does not scale to the unknown: hand-enumerating examples can only cover cases you already imagined, which is the very gap that bit you. You would have to predict leap-day and empty-string handling in advance — the thing you failed to do the first time. Option C is wrong because coverage forces lines to execute but says nothing about which inputs exercise them. You can hit 100% coverage with inputs that never include a leap day, so the threshold does not target the edge-case class. Option D is wrong because mutation testing measures whether your existing tests would catch injected faults in the code — a test-quality metric — not whether your input space covers unenumerated edge cases. It would flag that your example tests are weak, but it does not generate the leap-day or empty-string inputs that property-based testing does.

5. You let a coding agent run an autonomous red-green-refactor loop: it generates code, runs the suite, reads failures, patches, and repeats. What is the single most important thing this loop needs to behave safely?
   • A. A trustworthy stop condition tied to the test suite — a green bar on a suite that actually encodes the requirement — so the agent terminates on real success rather than churning or halting on a guess. ✓
   • B. The largest possible context window so the agent can hold the entire codebase in memory at once.
   • C. A faster model so each generate-run-patch iteration completes in fewer seconds.
   • D. Permission to skip running the suite on iterations where the diff looks small, to save time.

   Option A is correct because an autonomous loop is defined by how it ends. The test suite supplies both the feedback signal that steers each iteration and the exit condition that tells the agent it is done. If the suite genuinely encodes the requirement, including the regression and edge cases, then "all tests pass" is a trustworthy stop condition. If the suite is weak, flaky, or circular, the agent can terminate on a false pass, having produced a confident, wrong diff faster than a reviewer can catch it — a passing suite confirms only the expectations it encodes, not total correctness. The quality of the stop condition is the quality of the loop, and other guardrails (iteration budgets, sandboxing, human review) still matter; the test suite is the one this question turns on. Option B is wrong because context size addresses how much the agent can see, not whether it knows when to stop. An agent with the whole repo in context and no trustworthy termination signal still loops forever or halts on a guess. Option C is wrong because speed multiplies whatever the loop already does. A faster bad loop reaches a wrong answer sooner; iteration latency is an efficiency concern, not the safety property that governs termination. Option D is wrong because skipping the suite removes the only signal the loop runs on. "The diff looks small" is exactly the visual judgment that AI-assisted workflows fail at — small diffs introduce async timing bugs, type mismatches, and hidden coupling. Skipping verification turns the loop's feedback off precisely when it is needed.

6. In review, two engineers disagree about how to vet a large AI-generated pull request. One wants to read the prompt that produced it; the other wants to read the diff and the tests. Which is the sounder review discipline and why?
   • A. Review the diff and the tests: the merged artifact is the code, and a well-crafted prompt is no guarantee the generated code matches it — what ships is what must be audited. ✓
   • B. Review the prompt: if the prompt was specific and correct, the generated code is correct by extension and the diff can be skimmed.
   • C. Review neither in depth; trust the green CI badge, since passing tests mean the change is safe to merge.
   • D. Review only the prompt and the commit message, because the diff is too long to read for AI-generated changes.

   Option A is correct because the prompt is not what runs in production — the diff is. A precise prompt raises the odds of good output but does not certify it; models routinely produce code that diverges from a clear instruction in ways visible only in the diff (a flipped condition, a dropped edge case, an unintended dependency). Reviewing the diff and the tests checks the actual artifact: does the change do what is claimed, do tests exist, do they cover the stated behavior and plausible failure modes, and does the implementation avoid hidden coupling. Treating AI output as a first draft subject to ordinary review discipline is what keeps the process honest. Option B is wrong because "good prompt implies good code" is the false-confidence assumption restated at the review layer. The mapping from prompt to generated code is not faithful enough to skip auditing the result; the gap between intent and implementation is exactly where AI bugs hide. Option C is wrong because a green badge is only as strong as the suite behind it. If the tests are circular or thin — common when a model wrote them — CI passes while the change is unsafe. The badge is an input to review, not a substitute for it. Option D is wrong because length is not a license to skip the artifact that ships. If a diff is too large to review, the correct response is to split the change into reviewable units, not to certify it from the prompt and message alone.

7. A team adopts an "acceptance-criteria-first" workflow for AI-assisted features instead of mandating strict test-before-code on every line. A purist objects that this "is not real TDD." What is the strongest defense of the team's choice?
   • A. The load-bearing element is an executable acceptance check that gates the generated code; strict line-by-line test-first ordering is a contested practice, and the gain comes from small steps plus a binary acceptance check, not from the exact moment the test is typed. ✓
   • B. Strict test-first is always optimal, so the team is simply wrong and should be overruled.
   • C. Tests are unnecessary overhead with AI codegen; acceptance criteria in a ticket are enough and the suite can be skipped.
   • D. TDD and acceptance-criteria-first are unrelated; the team has abandoned testing discipline entirely.

   Option A is correct because the evidence for strict test-first ordering as a universal rule is mixed — studies attribute the benefit largely to working in small, verifiable steps and to having a clear, executable acceptance check, rather than to the literal sequence of writing the assertion before any production line. An acceptance-criteria-first workflow keeps the part that does the work: before you trust generated code, an executable check that encodes the requirement must exist and pass. That check is the binary gate on the AI's output. Whether the engineer wrote it thirty seconds before or after prompting is far less important than that it exists, fails for the right reason, and the change stays small enough to reason about. Option B is wrong because it states a contested practice as settled fact. Asserting strict test-first is "always optimal" overclaims beyond what the research supports and ignores that the measured gains track small steps and acceptance criteria. Option C is wrong because it throws out the gate entirely. Acceptance criteria in a ticket are prose, not an oracle; skipping the executable suite removes the one mechanism that deterministically settles whether the generated code is correct. Option D is wrong because acceptance-criteria-first is a form of test-driven discipline, not its abandonment — the acceptance check is expressed as an executable test. The team kept the binary gate and relaxed only the contested ordering dogma.

8. An engineer reviews an AI-generated diff by eye, judges it "obviously correct," and merges without running the suite. Within an hour, production throws an unhandled promise rejection. Which best explains why visual review missed it?
   • A. Generated code frequently carries defects that are invisible on inspection — async timing bugs, type mismatches, dependency side effects — that only surface when the code actually executes against the test suite. ✓
   • B. The diff was simply too short; longer diffs are easier to review visually because there is more context.
   • C. Visual review failed because the engineer did not read the original prompt that generated the code.
   • D. The bug was unavoidable; no test could have caught an unhandled promise rejection before production.

   Option A is correct because whole classes of defect are simply not visible by reading. An unhandled promise rejection, an off-by-one in a boundary, a subtle type coercion, or a dependency that mutates shared state all look fine on the page and only manifest when the code runs. AI-generated code is especially prone to these because the model optimizes for plausible-looking structure, which is exactly what survives a visual scan. Running the suite — even a quick unit pass on the changed function — converts these from production incidents into local test failures. The skipped step, not the reviewer's eyesight, is the root cause. Option B is wrong because diff length is not why visual review fails here; an async rejection hides in a two-line change as easily as a two-hundred-line one. The failure mode is executing-vs-reading, not short-vs-long. Option C is wrong because reading the prompt would not have surfaced a runtime rejection either — the prompt is intent, not behavior. The missing artifact is the test run, which exercises behavior; the prompt does not. Option D is wrong because an unhandled rejection is precisely the kind of thing a focused test exercising the failing path would have caught before merge. Calling it unavoidable excuses the skipped verification step that would have caught it.

Frequently asked questions

What is test-driven AI coding?

Test-driven AI coding is the practice of defining expected behavior through tests before or alongside AI-assisted implementation. The test serves as a formal contract: the AI produces code, and the test determines whether that code is correct. This prevents generated code from satisfying a vague prompt while silently breaking real behavior. Common tools for test-driven AI coding include AI code assistants like GitHub Copilot or ChatGPT, and widely used testing frameworks such as pytest or unittest for Python, Jest for JavaScript, or JUnit for Java. Choosing familiar tools helps candidates connect these practices directly to real-world development environments.

Why is test-driven AI coding useful for production work?

AI code generation is fast but unverified. Without tests, a plausible-looking implementation can pass visual review while introducing off-by-one errors, missing edge cases, or subtle regressions. Tests turn acceptance into a binary, reproducible check rather than a best guess.

Can AI write tests for me?

AI can draft test skeletons, but the engineer must verify that they test real behavior rather than just echo the implementation. A test that asserts the current return value without checking boundary conditions or failure modes can be misleading. Review AI-generated tests the same way you would review AI-generated logic.

What verification steps should one follow for AI-generated code?

Run the smallest meaningful verification layer first: unit tests for the changed function, then integration tests for its callers, then type checks and linting. For more complex changes, add end-to-end or contract tests. The specific stack matters less than the habit of running something before merging.

How does test-driven AI coding differ from traditional TDD?

Traditional TDD is a design discipline; tests shape the API. Test-driven AI coding adds a verification layer specific to generated output. The tests still define behavior, but the emphasis shifts to constraining and auditing what AI produces rather than using tests purely to drive design decisions.

What types of tests are most useful when working with AI-generated code?

Focused unit tests on the specific function or component, snapshot or contract tests for interfaces, and regression tests for any behavior that previously worked. Property-based tests are especially valuable because they stress-test inputs that AI prompts may not have considered.

How should a team handle AI-generated code in code review?

Review the diff, not the prompt. Reviewers should check that tests exist, that they cover the stated behavior and plausible failure modes, and that the implementation does not introduce hidden coupling or silent assumptions. Treating AI output as a first draft that needs standard review discipline keeps the process honest.

Related topics