class: center, middle, inverse count: false # Retrieval Beyond Vectors ??? ~15 minutes. The conceptual closer for Module 8. RAG is a category, vectors are one implementation, this lecture covers the rest. Sets up the upcoming project. --- # RAG Was Never About Vectors Lecture 8.1 set up the framing. Lecture 8.2 went deep on vectors. This lecture is the rest of the picture. Vector search is one retrieval mechanism. There are several others, and most production agents use more than one. The choice depends on what kind of information you need and how it is structured. ??? 30 seconds. Re-state the framing from 8.1. This lecture is the conceptual unification. --- # Web Search as Retrieval Web search APIs return current results from the public web. The agent issues a query, the API returns ranked results, the agent injects them into context. Two ways to get it: - **Built into the model API** — Claude web search, OpenAI web search, Gemini grounding with Google Search. The provider runs the search inside the API call. - **A search API called from your own tool:** - **Exa** — search built for AI; can return page text - **Perplexity Search API** — ranked pages with relevant excerpts - **Brave Search API** — independent web index The pattern: send a query string, receive structured results (title, URL, snippet). ??? 90 seconds. Connect to Lecture 8.1. A built-in search tool is run by the provider: the model calls it, and the result comes back in the same response, with no tool code to write. A search API is called from a tool you write, so your code runs the search and returns a tool_result. The upcoming project builds the second kind. Older tutorials point at Google Custom Search (shuts down January 1, 2027) and the Bing Search APIs (retired 2025). Free tiers change often; check current pricing. --- # Web Search: Strengths and Limits **Strengths:** - **Live information** — anything indexed by the search engine is available - **Broad coverage** — no upfront ingestion of a corpus - **Cheap to start** — most providers have a free tier **Limits:** - **External dependency** — your agent depends on a third-party service - **Per-query cost at scale** — pricing past free-tier limits - **No control over indexing** — you take what the search engine has crawled - **Latency** — each query is a network round trip Right choice when info changes faster than you can re-index, or scope is broader than any private corpus. ??? 60 seconds. The tradeoffs frame what kinds of agents benefit from web search. --- # Database Queries as Retrieval When information lives in a database — customer records, inventory, transaction logs — the right retrieval mechanism is a database query. The agent generates a query (often from natural language), the database returns rows, the result enters context. **Why this beats vector search for structured data:** - **Exact precision** — no similarity threshold, no fuzziness - **Aggregation** — counts, sums, joins across tables - **Cheap and fast** — milliseconds per query, no embedding cost - **Always current** — reflects database state at query time, no re-indexing ??? 90 seconds. Vector search would treat structured data as text and try to find "similar" records. That works poorly. Database queries handle structured questions natively. --- # Database Tool Safety Three things to enforce at the tool boundary: - **Read-only credentials** — give the agent a read-only user, not a write-capable one - **Allowed-tables list** — same pattern as the approved-folders list in your coding agent; restrict which tables the tool will query - **Result size caps** — add `LIMIT` clauses or row caps in the wrapper; a query can return millions of rows .warning[Don't give the agent your production database credentials. Give it a read-replica or a constrained view.] The agent's database tool is usually narrowly scoped. Constrain at the boundary, like with file operations. ??? 60 seconds. Safety pattern matches the approved-folders concept in the coding agent. Same principle: enforce at the tool boundary. --- # File Reads and API Calls Are Retrieval Too The `read_file` tool from Module 6 is a retrieval tool. The agent retrieves a known file's contents and injects them into context. Any tool that fetches information from outside the model and returns it is retrieval: - `read_file(path)` — retrieves a file from disk - `fetch_api(url)` — retrieves a JSON response from an HTTP endpoint - `get_weather(city)` — retrieves current weather from a service - `read_email(message_id)` — retrieves an email body What these have in common: they bring external information into the model's context for the current generation. ??? 60 seconds. The point: students have been building RAG since Module 6 without calling it that. Naming the pattern unifies what they've already done. --- # Parametric Retrieval The model itself is a retrieval system, in a loose sense. Training compressed an enormous text corpus into the model's weights. Asking a question without providing context is retrieval from the model's parametric memory. This kind of retrieval: - No API call, costs nothing extra, fastest available - Stale (training cutoff), opaque (no source attribution), prone to hallucination Fine for general knowledge. For anything specific, recent, or auditable, use one of the explicit mechanisms. ??? 30 seconds. One-slide aside. Important to recognize that "the model knows" is also a kind of retrieval. --- # Real Agents Use More Than One .split-left[ Production agents combine retrieval mechanisms. Claude Code is a good example: - **File reads** — for the user's codebase (`read_file`, `glob`, `grep`) - **Web search** — for documentation, current API specs - **Bash execution** — for tools that themselves retrieve (`git log`, `psql`) - **Parametric** — for general programming knowledge Each mechanism handles what it's best at. The agent's intelligence is partly in choosing the right one for each question. ] .split-right[
]
??? 90 seconds. Composition is the most important takeaway from this lecture. Image prompt for `retrieval-composition.png`: "A central rounded rectangle labeled 'Agent' with five arrows extending outward in different directions to five different rounded rectangles, each labeled and with a small representative icon. (1) Top-left: 'Vector store' (with stack-of-documents icon). (2) Top-right: 'Web search' (with globe icon). (3) Right: 'Database' (with cylinder/database icon). (4) Bottom-right: 'Files / APIs' (with file or HTTP icon). (5) Bottom-left: 'Training data (parametric)' (with brain or neural network icon, drawn in lighter color to indicate it's not external). Each arrow labeled with the tool used (e.g., 'search_docs', 'web_search', 'query_db', 'read_file'). Clean flat design, white background, sans-serif labels, teal/blue color palette." --- # When to Choose What | Need | Mechanism | |---|---| | Semantic search over unstructured docs | Vector RAG | | Current public web information | Web search | | Structured data with exact answers | Database query | | Known specific resource | File read or API call | | General world knowledge | The model itself | Most agents need at least two of these. Few need all five. ??? 60 seconds. The decision table is the practical reference. --- # The Unifying Principle .callout[**Retrieval is any way to get external information into the context.**] Strip away the implementation details. Every retrieval mechanism does the same thing: brings information from outside the model into the prompt for one specific generation. - Vector search retrieves from an embedded corpus - Web search retrieves from a search engine - Database queries retrieve from a database - File reads retrieve from disk - API calls retrieve from external services All of them are RAG. The R is the only thing that varies. ??? 60 seconds. Land the principle one more time. This is the takeaway students should leave with. --- # Key Takeaways 1. **Vector search is one retrieval mechanism** — web search, database queries, file reads, API calls, and parametric "retrieval" are others 2. **Match the mechanism to the data** — structured data wants databases, current info wants web search, semantic search wants vectors 3. **Real agents compose** — production systems blend mechanisms based on what each does best 4. **The unifying principle** — any tool that brings external information into the context is a retrieval mechanism 5. **The upcoming project brings this home** — build a web search tool and integrate it into your coding agent ### Next: Memory systems. Persistent, agent-owned, accumulated over time. ??? Transition to the upcoming project and memory. Memory is another way to get information into context — persistent and accumulated rather than queried fresh.