Prompting for Cited, Multi-Call Answers

Where we left off

naive_rag Lesson 15 already taught citing a single retrieval's source. This lesson's twist is specific to the agentic case: when an answer draws on two separate search_notes() calls, each returning a different document, a naive citation instruction ("cite your source") tends to produce one citation covering an answer that actually needs two, silently implying the second fact came from the same place as the first.

The code, piece by piece

SYSTEM_INSTRUCTION = """... When you give your final answer, cite the
source file ... each fact came from, right after that fact. If you
used more than one source, every fact needs its own citation, don't
cite only the first one and assume it covers the rest."""

The last sentence is doing the real work here, and it exists because of a failure mode you can reproduce by removing it: with a plain "cite your sources" instruction, a model synthesizing two retrieved facts into one fluent paragraph will often cite only once, typically at the end, in a way that reads as covering the whole answer even though only the last fact actually came from that source.

return "\n\n---\n\n".join(f"[{r['source']}]\n{r['text']}" for r in top_k)

Unchanged from Lesson 10, included here as a reminder: citation is only possible because search_notes() already tags every passage with its source filename. A tool result with no source information gives the model nothing to cite, no matter how the prompt is worded.