Prompt engineering and structured output for finance

Learning objectives

By the end of this module, the reader will be able to:

• Design role-based system prompts (sell-side analyst, risk manager) with an Identity → Instructions → Examples → Context structure, an explicit as_of date, and negative grounding instructions.
• Select the appropriate prompting technique for each task type (zero/few-shot, chain-of-thought, self-consistency, decomposition, grounding) using criteria based on empirical evidence, including the documented cases where chain-of-thought degrades the result.
• Implement anti-numeric-hallucination pipelines: verbatim citations with provenance, two-step verification with exact-match in code, and a fail-closed policy.
• Define institutional-grade Pydantic schemas (InvestmentThesis, TradingSignal) with closed enums, mandatory unit and scale, semantic validators, and per-field null under uncertainty.
• Operate reasoning-cost routing (reasoning_effort, budget_tokens) and decide between prompting and fine-tuning with explicit criteria.

Module 5 built the retrieval layer: the context that feeds the prompts. This module addresses the second half of the problem: how the model is instructed to turn that context into reliable, auditable outputs consumable by front-office systems. It closes Block II (understanding information) and sets up Block III (risk and portfolios). Two principles run through it, both with direct empirical backing: the LLM never calculates —all arithmetic is delegated to deterministic tools— and every cited figure is verified by exact-match against its source, not by another LLM's judgment (Financial Wisdom TV) .

6.1 Techniques by task type

The central lesson of the 2023–2026 evidence is that there is no "best prompt": there is a technique × task correspondence, and choosing badly not only raises costs —in some cases it actively degrades accuracy (Day Trading) .

6.1.1 Role-based system prompts: sell-side analyst and risk manager

Role prompting (assigning an expert role in the system message) is well supported in finance. OpenAI's official guide recommends structuring the developer message into four sections —Identity, Instructions, Examples, and Context— with the context at the end of the prompt and delimited with XML tags (towardsai.net) . In financial sentiment, the role-playing format outperforms flat prompts (arXiv.org) , and in multi-agent decision architectures the pattern is formalized into [ROLE], [GOAL], [RULES], [FORMAT] blocks, with a Strategist → Critic → Moderator separation that improves the detection of evidentiary gaps and numeric inconsistencies relative to a single agent (Learnsignal) .

Template 1 — sell-side analyst system prompt (production):

# Identidad
Eres un analista sell-side senior de renta variable (sector tecnología) con 15 años
de experiencia. Escribes para inversores institucionales.

# Instrucciones
- Toda cifra que cites debe proceder EXCLUSIVAMENTE del bloque <filing_excerpt>.
- Si un dato no está en el contexto, responde "NO_DISPONIBLE" en ese campo.
- No calcules ratios mentalmente: para cualquier cálculo, invoca la herramienta
  `calculator`. La aritmética en texto está prohibida.
- Distingue siempre entre hechos (del filing) e inferencias (tu juicio); etiqueta
  cada párrafo como [HECHO] o [INFERENCIA].
- Horizonte del análisis: 12 meses. Indica siempre la fecha "as-of" de los datos y
  el period_end de cada cifra. No uses información posterior a {as_of}.

# Formato de salida
Devuelve ÚNICAMENTE un objeto JSON conforme al schema InvestmentThesis (adjunto).

# Contexto
<filing_excerpt source="10-K AAPL FY2025" as_of="2025-11-01" page="31">
...
</filing_excerpt>

Template 2 — risk manager system prompt:

# Identidad
Eres un risk manager de un fondo multi-estrategia. Tu prioridad es la preservación
de capital y el cumplimiento normativo; ante la duda, sé conservador.

# Instrucciones
- Evalúa cada propuesta con el esquema: evidencia → regla/política → conclusión.
- Verifica la coherencia aritmética (P5 ≤ P50 ≤ P95, rangos y totales) antes de
  aprobar; las comprobaciones mecánicas las ejecuta el código, no tú.
- Rechaza cualquier recomendación sin cuantificación de incertidumbre o sin citas.
- Nunca modifiques límites de riesgo; solo los evalúas contra la política adjunta.

Three details are not decorative. The negative instruction ("if it is not in the context → NO_DISPONIBLE") is the primary defense against hallucination: without it, the model fills gaps with potentially stale training knowledge (IDEAS/RePEc) . The as_of date is a point-in-time defense: standard models exhibit look-ahead bias, with out-of-sample alpha decay of more than 15 percentage points, because they memorize outcomes instead of predicting them (metricgate.com) . And the ban on in-text arithmetic anticipates the evidence in section 6.2.

Few-shot: three examples are enough

Few-shot prompting (including input-output examples in the prompt) is the technique with the most robust evidence for financial sentiment classification. The seminal study by Lopez-Lira and Tang (2023) showed that headline sentiment scores predict daily returns, and that predictability is an emergent capability of large models (GPT-3.5/4 yes; GPT-1/2 and BERT no), with a Sharpe of 3.8 for GPT-4 in a 2021–2022 long-short without costs (arXiv.org) . On dosage: in lightweight LLMs (Qwen3-8B, Llama3-8B) the jump from 0-shot to 3-shot raises average accuracy from 0.67 to 0.74, but from 3-shot to 5-shot the improvement stops being substantial (arXiv.org) . The additional lever is not more examples but verbose category descriptions and an escape "Other" class (metricgate.com) .

Template 3 — 3-shot sentiment classifier (AD-FCoT format, with explicit causality and no look-ahead):

Eres un analista financiero. Lee la noticia y razona paso a paso sobre su impacto
en la acción; después emite Positive/Negative/Neutral.

<ejemplo>
Noticia: "ACME retira del mercado su producto estrella por defectos de seguridad."
Razonamiento: La retirada implica costes directos, riesgo de litigios y pérdida de
ingresos del producto principal → impacto negativo en próximos trimestres.
Sentimiento: Negative
</ejemplo>
<ejemplo>
Noticia: "ACME supera expectativas de beneficio y eleva la guía anual."
Razonamiento: Beat de EPS + guía al alza señalan momentum operativo;
históricamente el mercado reacciona con re-rating positivo.
Sentimiento: Positive
</ejemplo>
<ejemplo>
Noticia: "ACME anuncia que cambiará el auditor del ejercicio 2027."
Razonamiento: Cambio rutinario de auditor sin implicación directa en resultados.
Sentimiento: Neutral
</ejemplo>

Noticia objetivo: {titular}
Razonamiento:

Chain-of-thought: where it helps and where it destroys accuracy

Chain-of-thought (CoT; inducing the model to verbalize intermediate steps) is the standard for multi-step reasoning, and in finance its value is real but bounded: in a multimodal filings benchmark, adding CoT raised calculation accuracy from 60.0% to 76.9% and reduced hallucination from ~15% to ~5% (🦜️🔗 LangChain) ; on FinQA/TAT-QA (numeric questions over annual reports), decomposition takes GPT-3.5-turbo from 28.4 to 51.4–55.1 points (metricgate.com) . Methodological warning: the verbalized chain is not necessarily faithful to the model's internal computation; it is not explainability (ryanoconnellfinance.com) .

The counter-evidence is the most counterintuitive finding in the area. On tasks where "thinking" hurts even humans, CoT degrades the model: in implicit statistical learning an absolute drop of 36.3 percentage points is observed, and in classification with exceptions to generalizable rules CoT increased the required iterations by up to 331%, because it biases the model toward the general rule and makes it ignore the contextual tokens carrying the correct answer (Day Trading) . In classification with human ground truth, enabling CoT improved no model and slightly degraded some; Chain-of-Verification reduced accuracy by 1–2 points by retracting correct classifications (metricgate.com) . In the legal domain —a structural analogue of the financial one— CoT helps contractual entailment (+8.5 pp) but hurts holdings identification (−16.0 pp) and multi-label classification (−3.8 pp) (arXiv.org) . Explicit CoT also degrades instruction following (A.L. Capital Advisory) .

Self-consistency samples \(k\) chains and takes the majority answer:

Combined with program generation it raised GSM8K from 72.0% to 80.4% with \(k=40\) (Financial Wisdom TV) . Its documented limit: if the prompt or the context induces the same bias, the samples agree on the error (majority hallucination); the technique presupposes that the model has access to the necessary information (arXiv.org) . In financial practice it is used with \(k=5\)–\(11\) over extractions of the same metric, sending the minority answers to an audit queue.

Least-to-most decomposes the problem into subproblems solved sequentially, reusing previous answers; it solved the SCAN benchmark with ≥99% accuracy versus CoT's 16% (arxiv.org) . A 2025–2026 warning: LLM-assisted decomposition can omit subtasks or introduce irrelevant steps; the mitigation is hybrid —the human fixes the skeleton (revenue → margins → cash flow → debt → valuation) and the LLM fills in the sub-questions (arXiv.org) .

Table 6.1 — Prompting techniques: when to use each one, and the evidence

Interpretation. The table condenses the module's routing rule: the technique is chosen by the cognitive structure of the task, not by fashion or by model capability. The cross-cutting pattern in the evidence is that verbalized reasoning pays when the task is compositional and numeric (FinQA, ratio calculation), and costs when it is pattern recognition or classification with exceptions —there the model "talks itself into" the wrong answer through excessive reasoning (ryanoconnellfinance.com) . Few-shot occupies the center: cheap, robust, and with diminishing returns beyond three examples, which simplifies maintenance in production. Self-consistency and decomposition are second-layer techniques: they multiply cost and latency, and are only justified where the error is expensive (a figure that will feed a risk limit) or where the task exceeds the length of the examples. Grounding, by contrast, is not optional in this domain: it is the condition of auditability, and it is developed in the next section.

6.2 Grounding and anti-numeric-hallucination

6.2.1 The "answer only with data from the context" instruction and its documented limits

The strict grounding pattern has five components observed in documented financial architectures: (1) context delimited with XML tags and metadata (source, as_of, page); (2) an explicit negative instruction ("if the answer is not in the context, answer NO_DISPONIBLE; never use training knowledge"); (3) a mandatory citation with provenance for every figure; (4) a confidence field and an evidence_quote per claim in the schema; (5) subsequent two-step verification (Learnsignal) . The canonical control block from the multi-agent literature is verbatim: [CONTROL] Do not introduce new data; rely exclusively on the CONTEXT (Learnsignal) . Citations with source and date reduce hallucinations and ease auditing (Learnsignal) ; in real pipelines over FinanceBench, the RAGAS Faithfulness metric —the fraction of claims supported by the retrieved context— sits at ~86–87% (CallSphere) .

The limits are structural, not stylistic. NumericBench documents that even frontier models (GPT-4o, DeepSeek-V3) fail at simple arithmetic, number retrieval, and multi-step reasoning, because they treat numbers as discrete tokens without numeric semantics; trivial perturbations of figures significantly reduce accuracy (博客园) . The paradigmatic case is 9.11 vs 9.9: "9.11" tokenizes as ["9", ".", "11"] and the model compares the 11 and 9 fragments as integers, reinforced by software-versioning patterns in the training data; the serious part is that Claude Sonnet 4 and GPT-4o articulate the decimal-comparison procedure with textbook precision while executing it wrong —the so-called computational split-brain (langchain.js) . At domain scale, a 2024 study cited by FailSafeQA estimates hallucination in up to 41% of financial queries, and minor wording differences ("net income" vs "earnings") alter the outputs (博客园) . In filing extraction, the dominant failures are: retrieval failure (42%), multi-state gap (23%), value-scale error (15%) —reading "millions" as "thousands"— and figure fabrication (10%) (🦜️🔗 LangChain) . That is why scale and unit are mandatory schema fields. The difficulty context: GPT-4 with retrieval scored ~19% on FinanceBench in the original configuration, and GPT-4 sits at 62.87% on FinQA versus 91.16% for the human expert (the full benchmark figures, in §5.1) (arXiv.org) .

PAL and tool calling as mitigation: the LLM never calculates

The mitigation with the most quantified evidence is delegating computation. PAL (Program-Aided Language Models) showed that the dominant failure mode of CoT in mathematics is not reasoning but arithmetic —in 16 of 25 cases, CoT generated nearly identical "thoughts" with different numbers— and that executing the reasoning as a program lifts GSM-Hard from CoT's ~20% to 61.2% (Financial Wisdom TV) . Toolformer had already shown that with access to a calculator the model solves arithmetic that pure models consistently fail (稀土掘金) . The tool calling contract is explicit: the LLM does not execute the function; it proposes which function to call and with what arguments, and the code validates and executes (supermemory.ai) .

from langchain_core.tools import tool
import ast, operator as op

_OPS = {ast.Add: op.add, ast.Sub: op.sub, ast.Mult: op.mul,
        ast.Div: op.truediv, ast.Pow: op.pow, ast.USub: op.neg}

@tool
def calculator(expression: str) -> float:
    """Evaluates an arithmetic expression. The ONLY authorized path for ANY
    calculation; the LLM must not do arithmetic in text."""
    def ev(node):
        if isinstance(node, ast.Constant): return node.value
        if isinstance(node, ast.BinOp): return _OPS[type(node.op)](ev(node.left), ev(node.right))
        if isinstance(node, ast.UnaryOp): return _OPS[type(node.op)](ev(node.operand))
        raise ValueError("expresión no permitida")
    return ev(ast.parse(expression, mode="eval").body)  # restricted AST, never eval()

@tool
def xbrl_lookup(ticker: str, concept: str, period_end: str) -> dict:
    """Returns a point-in-time XBRL concept from a SEC filing with citation (doc, page)."""
    ...

Two-step verification with fail-closed

The second pillar is deterministic post-generation verification: a code verifier checks that every figure appears literally in the declared verbatim quote. If it fails, the record is not persisted: it retries, returning the error to the model, and once retries are exhausted, it rejects (fail-closed). It is the application of institutional insight I6: for figures, exact-match in code > LLM-as-judge, because asking another LLM to judge a figure is circular.

def extract_with_retry(prompt: str, max_retries: int = 2) -> FilingMetric:
    last_error = None
    for _ in range(max_retries + 1):
        out = extractor.invoke([
            {"role": "system", "content": SYSTEM},
            {"role": "user", "content": prompt + (f"\nError previo: {last_error}" if last_error else "")},
        ])
        if out["parsing_error"] is None:
            metric = out["parsed"]
            # Two-step verification: the figure must appear literally in the quote
            if metric.value is None or str(metric.value).replace(",", "") in \
               metric.evidence_quote.replace(",", ""):
                return metric
            last_error = "evidence_quote no contiene la cifra literal"
        else:
            last_error = str(out["parsing_error"])
    raise ValueError(f"extracción fallida tras reintentos: {last_error}")

Note the deliberate asymmetry: value = null passes verification (the model abstains), but a figure that is not in the quote never passes. Abstention is a legitimate result; fabrication is a system failure.

In institutional practice. The exact-match pattern over figures is not an academic eccentricity: Kensho (S&P Global) operates in production "Grounding", a multi-agent framework on LangGraph whose multi-stage evaluation uses exact-match routing and tool-calling metrics over real financial data (IDEAS/RePEc) —precisely because an LLM's judgment about a risk figure is not auditable before a regulator. The desks running these pipelines apply the same division as this module: syntactic validation is resolved by constrained decoding; semantic validation —ranges, monotonicity \(P5 \le P50 \le P95\), unit consistency, citation verification— is the team's responsibility in defensive code (A.L. Capital Advisory) . The retry budget is set at 2–3 and every rejection feeds a human review queue, never a silent default value.

6.3 Pydantic schemas for the front office

6.3.1 InvestmentThesis and TradingSignal ready for OMS/risk

The reliability hierarchy of output mechanisms is documented: JSON mode only guarantees syntactically valid JSON (truncation is its leading cause of failure (博客园) ); function/tool calling types the arguments with ~86% schema compliance; and Structured Outputs (strict json_schema) reaches ~100% via grammar-constrained decoding (A.L. Capital Advisory) . In LangChain, with_structured_output is the integration point: in langchain-openai the default method moved from function_calling to json_schema in 0.3.0, and with strict=True Pydantic models cannot use Field metadata (min/max) or default values (oneuptime.com) ; in agents, create_agent chooses ProviderStrategy (native structured output) if the provider supports it and ToolStrategy otherwise, with handle_errors for retries and the result in structured_response (Red Hat Developer) . Essential design fact: the class name, the docstring, and the field descriptions are injected into the prompt —the schema IS part of the prompt (Quant Memo) .

Full schema — investment thesis:

from pydantic import BaseModel, Field
from typing import Literal, Optional
from datetime import date

class Evidence(BaseModel):
    """Mandatory citation for every factual claim."""
    doc_id: str = Field(description="Identificador del documento, p.ej. 'AAPL_10K_FY2025'")
    page: int = Field(description="Página del documento fuente")
    quote: str = Field(description="Extracto VERBATIM (<= 40 palabras) que soporta la afirmación")

class MetricValue(BaseModel):
    """A financial metric with explicit unit, scale, and provenance."""
    value: Optional[float] = Field(
        description="Valor numérico. null si NO aparece en el contexto. NUNCA inventar.")
    unit: Literal["USD", "EUR", "shares", "percent", "ratio", "other"]
    scale: Literal["units", "thousands", "millions", "billions"] = Field(
        description="Escala tal como aparece en el filing (evita errores 10^3/10^6)")
    period_end: date = Field(description="Fecha de cierre del periodo reportado")
    as_reported: bool = Field(description="True si la cifra es literal del documento; "
                                          "False si se derivó de una tool call")
    evidence: Optional[Evidence] = None

class InvestmentThesis(BaseModel):
    """Structured investment thesis generated exclusively from the context."""
    ticker: str
    as_of: date = Field(description="Fecha de corte de la información (point-in-time)")
    rating: Literal["strong_buy", "buy", "hold", "sell", "strong_sell"]
    horizon_months: Literal[3, 6, 12, 24]
    revenue_ltm: MetricValue
    eps_ltm: MetricValue
    net_debt: MetricValue
    bull_case: list[str] = Field(max_length=3)
    bear_case: list[str] = Field(max_length=3)
    key_risks: list[str]
    confidence: Literal["low", "medium", "high"]
    reasoning_summary: str = Field(description="<= 100 palabras; marca [HECHO]/[INFERENCIA]")

Full schema — trading signal for OMS/risk integration:

class TradingSignal(BaseModel):
    """Signal consumable by an OMS; no field computed by the LLM."""
    instrument_id: str = Field(description="Identificador canónico (ISIN/FIGI/ticker interno)")
    as_of: date
    direction: Literal["long", "short", "flat"]
    strength: float = Field(ge=-1.0, le=1.0, description="Intensidad normalizada [-1, 1]")
    signal_family: Literal["sentiment_news", "fundamental", "technical", "composite"]
    valid_from: date
    valid_until: date = Field(description="Caducidad de la señal (point-in-time safety)")
    max_position_pct_nav: float = Field(ge=0.0, le=100.0)
    rationale_codes: list[str] = Field(description="Códigos de la taxonomía interna, NO texto libre")
    model_version: str = Field(description="Snapshot del modelo + hash del prompt, para auditoría")
    compliance_flags: list[Literal["restricted_list", "mnpi_risk", "liquidity_low"]] = []

The design notes for consumption by an OMS or risk engine are five: closed enums and numeric ranges on the decisional fields; explicit expiry (valid_until) as a point-in-time defense; traceability (model_version, prompt hash) per record; typed compliance flags; and the separation of powers from insight I1 —the LLM proposes and the risk engine disposes: the signal never executes directly. For diagnostics, include_raw=True returns {'raw', 'parsed', 'parsing_error'} and allows capturing errors without an exception (oneuptime.com) ; self-repairing parsers in the OutputFixingParser style retry with an LLM until valid or until the limit (博客园) ; and it is wise to set max_tokens at 2–3× the expected length against truncation (博客园) .

Risk note — "structured output guarantees form, not truth". Near-100% schema compliance can raise the rate of confidently wrong answers. In one documented fintech case of loan-PDF extraction, the fully conformant run had the highest rate of confident errors: when the data was not in the document, the constrained version emitted {"income": 75000} because the grammar required income to be a number —the model could no longer refuse (metricgate.com) . The front-office equivalent is a fabricated but perfectly typed net_debt crossing the OMS without raising alarms. The mitigations are structural: making null/"unknown" part of every field's schema (abstention must be representable), pairing with an unconstrained self-consistency check, and keeping two-step verification before persisting (metricgate.com) . The residual risk lives in semantic validation, not syntactic: "structured outputs eliminate the class of parsing failures, but they do not eliminate the need for defensive code in production" (supermemory.ai) .

EXPANDE

Common pitfall: mistaking "valid schema" for "true data"

The jump from ~86% (function calling) to ~100% (strict structured output) eliminates an entire class of failures — parsing — and for that very reason creates a new trap: everything that comes out is well-formed, including invented data. The fintech case in the risk note is canonical: by having the grammar require income to be a number, the model emitted {"income": 75000} when the document did not contain that data point, and the fully conformant run had the highest rate of confident errors.

The design lesson has two parts. First: abstention must be representable in the schema (Optional[float] with null documented as "does not appear in the context"), because a mandatory and guessable field is an implicit order to fabricate. Second: since constrained decoding only guarantees form, truth is defended in later layers, none optional in the front office:

The residual risk lives in semantic validation: that is why exercise 2 asks for a @model_validator (monotonicity, evidence present when there is a value) and not just Field(ge=…) — which, moreover, strict=True does not even allow.

6.4 Reasoning models and technique routing in 2026

6.4.1 reasoning_effort / budget_tokens: when to pay for reasoning

Since 2025–2026, reasoning depth is a governable API parameter: OpenAI's o-series exposes reasoning_effort (low/medium/high; GPT-5 adds minimal) and Claude exposes thinking.budget_tokens (typically 1024, 4096, or 16000) (Github) . OpenAI's official guide frames the distinction: reasoning models generate an internal chain of thought, excel at complex tasks and multi-step planning, and are slower and more expensive; the official analogy is senior colleague (high-level objective) versus junior (explicit instructions) (towardsai.net) . The routing rule is symmetric: raise the effort for mathematics, code, and tool-call planning; lower it for simple classification, extraction, and formatting —"reasoning depth is not the higher the better" (Github) . On multi-capability financial benchmarks reasoning models lead (o1: 67.3% on XFinBench), but on trading-opportunity judgments every LLM failed the buy component of FinTradeBench (Github) .

Production details that break real systems: budget_tokens is a soft cap, but max_tokens must be strictly greater or the API returns a 400 error —a bug shipped to production by real teams—; the response is multi-block (thinking + text) and you must iterate response.content (arXiv.org) . Interleaved thinking allows continued reasoning between tool calls, useful in long agentic chains (Github) .

Table 6.2 — Technique × task routing with evidence

Interpretation. The table operationalizes the economics of reasoning: effort is paid per thinking token, so misrouting has a double cost —monetary and accuracy—. The classification row contradicts the 2023 intuition: for a desk's typical volume (thousands of headlines a day), the optimal configuration is the cheapest possible —lightweight model, zero temperature, three examples, no chain of thought— and adding sophistication degrades the result (Day Trading) . At the opposite extreme, multi-step analysis justifies the reasoning model at high effort, with a governance caveat: the internal thinking is not auditable, so the output must still pass through schema, citations, and exact-match verification. The calculation row has the greatest institutional impact: the PAL evidence (+41 percentage points on GSM-Hard) turns "the LLM never calculates" from a heuristic into a policy with measurable ROI (Financial Wisdom TV) . Routing is, in synthesis, a cost-risk function: reasoning is spent where the error is compositional and expensive, and saved where the task is recognition.

EXPANDE

The routing map in one image

Table 6.2 condensed as an operational decision tree — the question is never "which model is better?" but "what structure does the task have?":

And the magnitude of what is gained or lost by routing well, with the figures from sections 6.1–6.2:

Two readings: the few-shot gain is real but modest and is exhausted at the third example (+7 pp from 0 to 3-shot, flat from 3 to 5); those of PAL and decomposition are of another order (+27 to +83 pp) because they attack the dominant failure mode — arithmetic and composition —, not the wording. Misrouting not only raises costs: in classification, adding CoT moves the bar downward.

Prompting vs fine-tuning: decision criteria

The decision is not ideological. With good prompting you reach 80–90% of fine-tuning performance at zero training cost and instant iteration; fine-tuning is justified for strictly consistent format, domain vocabulary, cost at high volume, and latency (財經無界) . In regulated domains there is an additional argument: "prompt engineering does not give full control over the model's output; in high-risk, regulated domains such as legal, finance, and health, fine-tuning must be used" (Elastic) . Economic viability already exists: fine-tuned lightweight LLMs (Qwen3-8B, Llama3-8B) are competitive in financial sentiment even with 5% of the data (arXiv.org) . The operational criteria are four: (1) if the bottleneck is format or vocabulary, fine-tuning; if it is access to information or reasoning, prompting with grounding and tools; (2) if the token cost of few-shot exceeds the amortized training cost, fine-tuning; (3) if the regulator demands demonstrable control of outputs, fine-tuning plus structured output; (4) in both cases, continuous evals —snapshot changes alter prompt behavior, and OpenAI recommends pinning snapshots in production with evaluation suites (towardsai.net) .

There are, moreover, limits that neither technique resolves: look-ahead bias is not fixed with prompts (it requires point-in-time data and evaluation) (metricgate.com) ; numeric incompetence is representational and is mitigated with tools, not instructions (langchain.js) ; and Lopez-Lira-style sentiment predictability is arbitraged away as it spreads —prompting gives access to the capability, not a defensive moat (arXiv.org) . Systematic edge requires proprietary data, execution, and risk management; prompting alone was never an investment strategy.

Exercises

1. Role-based system prompt. Write the complete system prompt for a credit analyst evaluating a 10-K for a risk committee, with the Identity → Instructions → Examples → Context structure. It must include an as_of date, a negative grounding instruction, [HECHO]/[INFERENCIA] tagging, and a ban on in-text arithmetic. Justify each line by citing which failure mode from section 6.2 it mitigates; remove those that mitigate none.

2. Schema with semantic validators. Extend InvestmentThesis with: (a) a target_price: MetricValue field whose scale and unit are validated against revenue_ltm; (b) a @model_validator that rejects theses with an extreme rating and confidence="low" simultaneously; (c) a validator requiring non-null evidence whenever value is non-null. Explain why (c) cannot be guaranteed by constrained decoding.

3. Routing experiment. Implement the Template 3 classifier in two variants —direct zero-shot and with CoT— over 200 labeled headlines. Measure accuracy, latency, and token cost; contrast with the evidence of −36.3 pp and +331% iterations (Day Trading) and identify on which subset of headlines CoT fails first.

4. Two-step verification pipeline. Build the pipeline from the Mermaid diagram: extractor with with_structured_output(..., include_raw=True), normalized exact-match verifier (punctuation, thousands separators, scale), and a fail-closed policy with max_retries=2. Inject a filing where the requested figure does not exist and document whether the system returns null (correct) or a fabricated figure (failure); report the confidently wrong rate before and after making null part of the schema.

5. Reasoning budget. Configure three routes of the same research agent —news classification (minimal effort), metric extraction (low effort + tools), multi-step thesis (high effort)— and measure cost and quality per route. Add the max_tokens > budget_tokens guardrail and a regression test that fails if it is violated (Github) .

6. Prompting vs fine-tuning decision. Write a one-page memo recommending prompting or fine-tuning for a high-volume corporate-events classifier in a regulated domain, applying the four criteria from section 6.4 and quantifying the cost crossover point with your provider's current prices.

Module 6 of 15 — LangChain for Quantitative Trading. Data and versions verified as of July 2026; prices and market figures subject to change. LangChain 1.3.14 / langchain-core 1.5.2.