🅿️ Parking Lot — Participant Q&A

Q1 LLM Selection — What is the guidance on selecting the best model?

There's no single "best" model — the right choice depends on your task complexity, quality threshold, and budget. AWS recommends a tiered approach:

Practical strategy: Use a tiered model approach — route simple tasks (classification, extraction) to cheaper models (Nova Micro/Lite), moderate tasks to mid-tier (Nova Pro, Haiku), and complex reasoning to frontier models (Claude Sonnet/Opus). Amazon Bedrock's Intelligent Prompt Routing can do this automatically with up to 30% cost reduction.

Q2 How to get web search capabilities on AI agents?

LLMs have a knowledge cutoff date — they don't know about events after their training data ends. Three approaches:

1. MCP + Web Search Tools — Connect your AI agent to a web search MCP server (e.g., Brave Search MCP, Tavily MCP). The agent searches the web in real-time. This is what Kiro uses.
2. RAG (Retrieval-Augmented Generation) — Index your own documents and let the agent retrieve relevant chunks before generating. Covered in Day 2 Module 4. Best for internal/proprietary information.
3. Tool Use / Function Calling — Give the agent access to APIs (news feeds, market data, regulatory databases) via tool definitions. The agent decides when to call which API.

Q3 TPU from Google vs GPU — Will that produce different quality models?

Short answer: No. The hardware does not affect the quality of the trained model. Given the same architecture, data, and hyperparameters, a model trained on TPUs produces the same quality output as one trained on GPUs. The differences are in efficiency:

Why it matters for you: As a consumer of models via Bedrock, you don't choose the hardware — the provider already trained the model. Google uses TPUs for Gemini; Anthropic uses GPUs for Claude. The quality difference comes from architecture, training data, and RLHF — not the chip.

Q4 LLM-as-Judge — Which model should be the judge? Reasoning model? Bigger model?

Use a stronger model than the one being judged. The judge should be at least as capable as the model being evaluated. Using a weaker model to judge a stronger one produces unreliable scores.

Reasoning models as judges: Extended thinking models can be effective because they "show their work" — but they're slower and more expensive. For most use cases, a frontier model (Claude Sonnet 4) with a well-designed rubric is sufficient.

Q5 When should we use lower vs higher temperature?

For AnyCompany Finance:

• Merchant risk assessments → Low (0.1-0.3): Consistent GREEN/AMBER/RED ratings every time
• Invoice data extraction → Low (0.0-0.2): Numbers and fields must be deterministic
• Executive summaries → Medium (0.4-0.6): Some phrasing variation is fine, facts must be grounded
• Brainstorming features → High (0.8-1.0): You want diverse ideas

Q6 Gemini vs Claude — How do different models produce different quality?

Key takeaway: No single model wins everything. For structured finance outputs, Claude follows complex formatting more precisely. For research with large documents, Gemini's context window is an advantage. Test your specific use case on 2-3 models and pick the best value.

Q7 P2P Invoice Data Extraction — How to handle dynamic invoices at scale beyond OCR?

Traditional OCR struggles with dynamic layouts because it extracts characters but doesn't understand document structure. The modern approach:

Recommended architecture for AnyCompany:

1. Amazon Textract for initial extraction (fast, cheap, handles standard layouts)
2. Multimodal LLM fallback for complex invoices — send the image to Claude/GPT-4o Vision, returns structured JSON
3. Validation layer — programmatic checks (totals = sum of line items, GST, currency)
4. Human-in-the-loop for low-confidence extractions

At scale: 3,000+ invoices/month — route standard invoices through Textract (~$0.01/page), complex ones to Vision LLM (~$0.01-0.03/page). Total: ~$30-90/month vs manual processing cost.

Q8 Bedrock vs direct LLM provider endpoint — What's the difference?

Q9 Grounding vs Relevance in Bedrock Guardrails — What's the difference?

Amazon Bedrock Guardrails has a Contextual Grounding Check with two distinct scores:

How it works: You provide a grounding source (reference docs), a query (user's question), and the model response. The system scores both grounding and relevance (0–0.99). You set thresholds — responses below are blocked as hallucinations.

Limits: Grounding source max 100,000 chars, query max 1,000 chars, response max 5,000 chars.

Q10 Cursor auto-summarizes after ~10 conversations and loses context. How to improve?

This is a known limitation of all AI coding assistants caused by context window limits. When conversation history exceeds the model's context window, the tool summarizes older messages — and important details can be lost.

Q11 How to save a file in Markdown (.md) format?

Markdown files are just plain text files with a .md extension. You don't need any special software — any text editor works.

touch my-file.md
open -e my-file.md