Controlling Generation

Module 3, Lecture 3.3 | Working with LLMs in Practice

This lecture covers the parameters that control how an LLM generates its response — temperature, top-k, top-p, and max tokens. Temperature is the most important lever for agent development: low values produce reliable, deterministic behavior; high values introduce creative variation. The lecture explains how each parameter works, demonstrates their effects with concrete examples and runnable scripts, and provides practical defaults for agent work. The key takeaway is one of proportion: generation parameters are worth understanding and setting correctly, but the real work of agent engineering happens in the context.

Read the full lecture narrative

Additional Resources

• Lecture slides
• LLM Settings — Prompt Engineering Guide — Comprehensive reference covering temperature, top-p, and max tokens with clear explanations of how each parameter affects generation behavior.
• LLM Sampling Parameters Explained — Let's Data Science — Detailed walkthrough of temperature, top-k, and top-p sampling with probability distribution visualizations showing how each parameter reshapes token selection.
• How to Tune LLM Parameters for Optimal Performance — phData — Practical guide to tuning generation parameters with use-case-specific recommendations and guidance on combining temperature with top-p and top-k.
• Understanding Temperature, Top-k, and Top-p Sampling — Codefinity — Beginner-friendly explanation of sampling methods with intuitive analogies and visual examples of how probability distributions change at different parameter values.