In-Context Learning and the Limits of Prompting

Lecture 3.4

• In-context learning: the model "learns" from examples in the prompt without weight updates
• Zero-shot vs. few-shot prompting with concrete examples
• When in-context learning works vs. when it fails
• The context rot problem: quality degrades as conversations grow
• The agent-specific challenge: loops generate unbounded context
• The shift from "prompt engineering" to "context engineering"
• Context engineering defined: curating the entire context state, not just the prompt

Additional Resources

• Lecture slides
• zero_vs_few_shot.py (Python demo)