Module 3, Lecture 3.4 | Working with LLMs in Practice
This lecture covers in-context learning — how LLMs "learn" new patterns from examples in the prompt without any weight updates. It compares zero-shot and few-shot prompting, demonstrating how examples improve both accuracy and output format consistency. The lecture then addresses the limits of prompting: context rot as conversations and tool results accumulate, conflicting instructions, and the single-context constraint. These limits motivate the shift from "prompt engineering" to "context engineering" — curating the entire context state so the model has exactly what it needs and nothing it doesn't. This reframe defines the rest of the course.
Language Models are Few-Shot Learners (Brown et al., 2020) — The GPT-3 paper that demonstrated in-context learning at scale. Showed that a sufficiently large model can learn new tasks from just a few examples in the prompt, without any weight updates. The foundational reference for understanding zero-shot, one-shot, and few-shot prompting.
Prompt Engineering — Lilian Weng — A thorough survey of prompting techniques from an OpenAI researcher, covering in-context learning, chain-of-thought, and their variations. Rigorous and well-cited.
Andrej Karpathy on Context Engineering — Karpathy's widely referenced post arguing for "context engineering" over "prompt engineering," explaining that industrial-strength LLM applications require filling the context window with the right information.
Context Engineering for Agents — LangChain Blog — An overview of the four core strategies for managing agent context: writing, selecting, compressing, and isolating context. Directly relevant to the shift from prompt engineering to context engineering.
Prompt Engineering Overview — Anthropic Docs — Anthropic's official guide to prompting Claude effectively, organized from broadly effective techniques to specialized strategies. Includes concrete examples and best practices.