← Learning

LangSmith

A linear, one-concept-per-lesson path through LangSmith: tracing, evaluating, and monitoring the agents built in the LangChain and LangGraph courses, in production. 23 lessons · 3 tiers.

Prerequisites: Completion of LangChain & Agent Building and LangGraph, or already comfortable with agents, tools, and graphs. This course traces and evaluates those agents rather than re-teaching them.

Lessons use Google's Gemini free tier (gemini-3.5-flash-lite), the same model as the other two courses. LangSmith itself is free to sign up for at smith.langchain.com and has a generous free tier for personal learning.

Built on LangSmith, the platform this course teaches.

Course source

Every lesson's README and lesson.py for this course live in the ai-agent-engineering repo.

View on GitHub →
LangSmith

Beginner

Tracing every call, no evaluation yet.

  1. 01Setup and First TraceGitHub
  2. 02Tracing BasicsGitHub
  3. 03Types of RunsGitHub
  4. 04Alternative Tracing MethodsGitHub
  5. 05Conversational ThreadsGitHub
  6. 06Tracing a LangGraph AgentGitHub
  7. 07Checkpoint: Traced Review PipelineGitHub

Intermediate

Datasets, experiments, and prompt iteration.

  1. 08Dataset UploadGitHub
  2. 09Running an ExperimentGitHub
  3. 10Custom EvaluatorsGitHub
  4. 11Summary EvaluatorsGitHub
  5. 12Pairwise ExperimentsGitHub
  6. 13Prompt HubGitHub
  7. 14Prompt Engineering LifecycleGitHub
  8. 15Playground and Reading ResultsGitHub
  9. 16Checkpoint: Evaluated RAG PipelineGitHub

Advanced

Feedback, monitoring, and production concerns.

  1. 17Publishing FeedbackGitHub
  2. 18Filtering and DashboardsGitHub
  3. 19Online EvaluationGitHub
  4. 20Regression Testing in CIGitHub
  5. 21Tracing the Agent in ProductionGitHub
  6. 22Cost, Latency, and ScaleGitHub
  7. 23Capstone: Production-Ready Agent EvaluationGitHub
Start at Lesson 1

Contact

Book a Call