Chat vs Generate

Two APIs, one model underneath

Lesson 3 used ollama.generate(), the simplest possible call: a flat string in, a written answer out. Ollama also offers ollama.chat(), which takes a list of role-tagged messages (system, user, assistant) instead of one string, the same message shape you'll recognize from LangChain's messages in the other courses.

Both APIs end up talking to the exact same model. The difference is who's responsible for formatting the prompt correctly.

Why chat template formatting matters

Lesson 2's ollama.show() printed a template field, a small program (in Go's templating syntax) that wraps your text in special tokens like <|start_header_id|>system<|end_header_id|> before the model ever sees it. Every instruction-tuned model was fine-tuned to expect its input wrapped exactly this way; get the wrapping wrong and answer quality quietly degrades; the model doesn't error, it just performs worse, in a way that's easy to miss.

generate() skips that template entirely unless you build the wrapping yourself; chat() applies the model's own template automatically from a plain list of messages. That's the whole reason chat() exists as a separate method, not just a convenience wrapper.

The code, piece by piece

response = ollama.generate(
    model="llama3.2",
    prompt=f"You are a terse assistant. Answer in one sentence.\n\n{PROMPT}",
)

This hand-glues a system instruction onto the front of the prompt with a newline. It often works well enough for small models, which is exactly the trap: it looks fine in a quick test, but there's no guarantee the model reliably distinguishes "instruction" from "question" this way, especially as prompts get longer or more complex.

response = ollama.chat(
    model="llama3.2",
    messages=[
        {"role": "system", "content": "You are a terse assistant. Answer in one sentence."},
        {"role": "user", "content": PROMPT},
    ],
)
response.message.content

messages is a list of plain dicts with "role" and "content" keys, no special classes required. Ollama applies the model's chat template to this list before sending anything to the model, so system really is treated as an instruction, not just text that happens to come first. The reply comes back as response.message, an object with its own .role (always "assistant" for a reply) and .content.

From here on, every lesson in this course uses chat(), exactly like every lesson in the langchain course uses message-based calls, not raw string prompts.