Grounding Financial Claims: Building an Editorial Fact-Checking Pipeline with Gemini and LangGraph

Publishing financial content carries unique responsibility. If an article reports an outdated Annual Percentage Yield (APY), quotes an expired promotional credit card bonus, or misinterprets regulatory lending disclosures, readers make real financial decisions based on inaccurate data.

Historically, safeguarding accuracy required painstaking manual fact-checking. Human editors cross-referenced every numerical rate, eligibility restriction, and policy claim against primary sources, issuer websites, and government regulatory databases. While rigorous, this manual review process created a significant editorial bottleneck.

To accelerate editorial turnaround without compromising accuracy, we built an automated fact-checking engine using LangGraph, Google Gemini on Vertex AI, and parallel search grounding.

Draft Article Text
        │
        ▼
 ┌──────────────┐
 │ Node 0:      │ (Sentence normalization & structural markup)
 │ Ingest       │
 └──────┬───────┘
        │
        ▼
 ┌──────────────┐
 │ Node 0B:     │ (Extracts article context, entity focus & intent)
 │ Topic Brief  │
 └──────┬───────┘
        │
        ▼
 ┌──────────────┐
 │ Node A:      │ (Structured JSON output via Gemini 2.5 Flash:
 │ Claim Extract│  isolates numerical facts, rates, rules, & mechanics)
 └──────┬───────┘
        │
        ▼
 ┌──────────────────────────────────────────────────────────┐
 │ Node BC: Parallel Verification Orchestrator (N Workers)  │
 │                                                          │
 │  Worker 1: Claim 1 ──> Search Grounding ──> Verify Node  │
 │  Worker 2: Claim 2 ──> Search Grounding ──> Verify Node  │
 │  Worker 3: Claim 3 ──> Search Grounding ──> Verify Node  │
 └────────────────────────────┬─────────────────────────────┘
                              │
                              ▼
 ┌──────────────────────────────────────────────────────────┐
 │ Incremental Persistence & Real-Time Editorial Review UI  │
 │ (Verdicts: Supported, Contradicted, Unverifiable)        │
 │ (Actions:  KEEP, REPLACE, CLARIFY, HEDGE, REMOVE)        │
 └──────────────────────────────────────────────────────────┘

---

Architectural Philosophy: Assistive AI over Autonomous Publishing

When applying AI to regulated or high-trust domains, the most common trap is aiming for full autonomy. Large language models can hallucinate, misinterpret nuances in financial terms of service, or cite cached information.

We deliberately designed this system around human-alongside-AI assistance:

• The AI never modifies or publishes live content directly.
• The system acts as a high-speed research assistant: it isolates testable claims, fetches authoritative live citations, calculates confidence scores, and generates suggested inline edits.
• The human editor retains full decision authority, accepting, tweaking, or rejecting suggested revisions with a single click.

---

The LangGraph State Machine

We modeled the verification pipeline as a directed acyclic graph in LangGraph:

1. Ingestion and Topic Briefing (node_0_ingest.py & node_0b_topic_brief.py)

Before analyzing individual sentences, the pipeline builds a high-level TopicBrief. This summarizes the core subject (e.g., “High-Yield Savings Accounts vs. Certificates of Deposit in Q2 2026”) and provides grounding context so subsequent search queries don’t misinterpret ambiguous terms.

2. Atomic Claim Extraction (node_a_extract.py)

Articles are decomposed into discrete, verifiable assertions using structured Pydantic outputs:

from pydantic import BaseModel, Field
from typing import List, Literal

class FactualClaim(BaseModel):
    claim_id: str
    original_sentence: str
    extracted_claim: str
    claim_type: Literal[
        "numeric_rate", 
        "product_mechanic", 
        "eligibility_rule", 
        "fee_threshold", 
        "definition"
    ]
    primary_entity: str = Field(
        description="The institution, product, or regulatory body named"
    )
    verification_query: str = Field(
        description="Optimized search query targeting primary source documentation"
    )

By constraining the model to output granular assertions rather than broad paragraphs, we avoid compound verification errors.

3. Parallel Search Grounding and Verification (node_bc_parallel.py)

Long-form financial guides often contain 40 or more factual claims. Evaluating them sequentially would take minutes.

The orchestrator distributes claims across a pool of concurrent worker threads. Each worker executes two sub-steps:

1. Authoritative Evidence Retrieval: Queries Google Vertex AI Gemini with Search Grounding. The prompt enforces strict domain priority, preferring primary financial institutions, SEC filings, and federal reserve databases over aggregator blogs.
2. Verification & Action Recommendation (node_c_verify.py): The model compares the original claim against the retrieved evidence and returns a structured verdict:

{
  "claim_id": "claim_04",
  "verdict": "contradicted",
  "confidence_score": 0.94,
  "suggested_action": "REPLACE",
  "suggested_text": "The standard introductory APY is 4.25%, not 4.75%.",
  "source_urls": [
    "https://example-bank.com/disclosures/savings-rates-2026"
  ],
  "reasoning": "The issuer lowered the introductory APY by 50 basis points on May 1st."
}

---

Infrastructure: Running Safely on Kubernetes (EKS)

The fact-checker runs as an asynchronous microservice on Amazon EKS. Because verifying a full article can take 30 to 60 seconds of sustained parallel I/O, we had to handle node scaling events safely.

When using Karpenter for Kubernetes node consolidation, nodes can be cordoned and terminated when cluster utilization shifts. If a node running a fact-check worker is abruptly evicted, the editorial user experience breaks mid-review.

To solve this, the application annotates its own pod with Karpenter disruption protection when a job begins, and safely removes the annotation when finished:

import os
import requests

def set_karpenter_disruption_lock(protected: bool):
    """
    Annotates the running pod to prevent Karpenter node consolidation
    during active verification workflows.
    """
    pod_name = os.getenv("HOSTNAME")
    namespace = os.getenv("POD_NAMESPACE", "default")
    
    patch_payload = {
        "metadata": {
            "annotations": {
                "karpenter.sh/do-not-disrupt": "true" if protected else "false"
            }
        }
    }
    # Apply JSON merge patch via local Kubernetes API service account
    apply_k8s_pod_patch(namespace, pod_name, patch_payload)

---

Incremental Persistence and the Feedback Loop

Instead of forcing editors to wait for the entire article to finish processing, every verified claim is committed immediately to PostgreSQL. The frontend single-page application polls the run state, rendering verified claim cards in real time as workers complete their tasks.

The Feedback Engine

In the web interface, editors have thumbs up/down controls on every verdict and suggested rewrite. If an editor flags a false positive (for example, where a promotional rate had special geographic restrictions that the search missed), the interaction is logged to a curated evaluation dataset.

This active feedback loop allowed us to refine system prompts, domain filtering rules, and verification thresholds weekly based on real editorial usage.

---

Outcome

Deployed across more than 50 editorial staff members, the system cut initial claim verification research time by more than half while maintaining the rigorous accuracy standards required for national financial publishing.

Building effective AI tools for critical workflows is rarely about full end-to-end automation; it is about eliminating research friction and giving human experts clear, citation-backed evidence to make confident decisions faster.