Production

What is Output Validation?

Output validation verifies that LLM outputs conform to expected schemas, types, and constraints before passing them to downstream systems. It's the guardrail that catches malformed JSON, out-of-range values, and unexpected formats.

Why It Matters

LLMs can produce any text—including invalid JSON, unexpected field names, or values outside allowed ranges. Without validation, these errors propagate to databases, APIs, and UIs, causing crashes and data corruption.

Validation Example

validation.py
from pydantic import BaseModel, validator
from typing import Literal

class RiskAssessment(BaseModel):
    severity: Literal["HIGH", "MEDIUM", "LOW"]
    quote: str
    confidence: float

    @validator("confidence")
    def check_range(cls, v):
        if not 0 <= v <= 1:
            raise ValueError("must be 0-1")
        return v

# Validate LLM output
result = RiskAssessment.model_validate_json(llm_response)

Validation Strategies

  • • Schema validation — Pydantic, JSON Schema, Zod
  • • Type checking — Ensure correct data types
  • • Enum constraints — Restrict to allowed values
  • • Range validation — Check numerical bounds
  • • Retry on failure — Re-prompt with error details

Related Terms

← Back to Glossary