Query Planning and Decomposition

Where we left off

Lesson 10 showed the loop handling multiple calls, but left the decision to decompose entirely up to the model's unguided judgment. That worked, but relying on it unprompted is fragile: some questions get correctly split, others get a single broad query that quietly misses one part. This lesson makes decomposition an explicit instruction instead of an implicit hope.

The code, piece by piece

SYSTEM_INSTRUCTION = """You answer questions using a search_notes tool ...
If a question has more than one distinct part ... call search_notes
once per part, with a separate, focused query for each part ..."""

CONFIG = types.GenerateContentConfig(tools=[TOOLS], system_instruction=SYSTEM_INSTRUCTION)

system_instruction is a GenerateContentConfig field that sets standing behavior for the whole conversation, separate from any single user turn, the same conceptual role as a system prompt in other APIs. Here it does one specific job: tell the model, explicitly, what "decompose a compound question" means in terms of this tool specifically (one focused query per part, don't combine early).

sub_query = call.args["query"]
sub_queries.append(sub_query)

This lesson also collects every query the model actually searched for, not just the final answer, so you can see the decomposition happen, not just infer it happened from the answer being correct.

Why an explicit instruction, not just a better tool description

Lesson 10's tool description already hinted at multi-document retrieval ("call this again with a different query"), and often that was enough. But a hint living inside one tool's description competes for attention with everything else in the prompt, and doesn't generalize if a second or third tool is added later. A system_instruction states the strategy once, for the whole conversation, independent of any one tool's wording, more robust, and the right place to add general reasoning strategy instructions as this course's toolset grows.