Playground and Reading Results

What we're building

The same RAG experiment from earlier lessons, run once more, then pulled back down as a pandas DataFrame using client.get_test_results, so its inputs, outputs, and scores can be inspected and analyzed directly in Python instead of only through the UI's tables.

What this reveals

LangSmith also has a Playground, a no-code part of the UI for trying a different prompt or model against a dataset by hand, without writing any Python. It's genuinely useful for quick, exploratory iteration, but this course is code-first, so instead of a screenshot walkthrough, this lesson covers the SDK-side equivalent of "getting results out to look at them": pulling a finished experiment (whether it came from evaluate(), as in every earlier lesson, or from the Playground) back into a DataFrame.

This matters once you want to do something the UI's built-in views don't offer directly: computing a custom aggregate, plotting a distribution of scores, joining an experiment's results against some other dataset you have locally, or just skimming everything in one table without clicking into each row.

The code, piece by piece

results = evaluate(target, data=DATASET_NAME, evaluators=[keyword_overlap], experiment_prefix="rag-readback")
experiment_name = results.experiment_name

Nothing new, the same evaluate() call from Lesson 10, kept here so this lesson is runnable on its own without depending on a previous lesson having already created an experiment.

dataframe = fetch_test_results_with_retry(experiment_name, "feedback.keyword_overlap")

Pulls the experiment named experiment_name back as a pandas DataFrame, one row per example, with columns for every input field, every output field, and every evaluator's score, all prefixed (input., outputs., feedback.) so they don't collide.

evaluate() returning doesn't mean the experiment (or, separately, its per-run feedback scores) has finished writing on LangSmith's side yet. Calling get_test_results immediately can raise, or come back missing the feedback column you actually want. fetch_test_results_with_retry retries with a short, growing wait until that specific column shows up, an ordinary eventual-consistency gap you'll hit again anywhere a script reads data it just finished writing.

print(f"Average keyword_overlap: {dataframe['feedback.keyword_overlap'].mean():.2f}")
print(dataframe[["input.question", "outputs.answer", "feedback.keyword_overlap"]])

Ordinary pandas from here on, this experiment's results are just rows and columns now, no different from data loaded from a CSV.