Controlling Generation

Lecture 3.3

• Temperature: controlling randomness (0.0 = deterministic, 1.0 = creative)
• Top-k and top-p (nucleus) sampling explained intuitively
• When to use which settings: coding tasks vs. creative tasks vs. agent decisions
• Max tokens: controlling output length
• Stop sequences: telling the model when to stop
• The implications for agent reliability: why agents need low temperature

Additional Resources

• Lecture slides
• generation_config.py (Python demo)
• temperature_demo.py (Python demo)