EvidenceLens: A Claim-Evidence Matrix for Auditing Financial Question Answering
Jump to Discussion
Presentation
- Session
- Is the Model Even Thinking?
- Time
- Thursday, Nov 12, 08:45 – 08:54 (US/Eastern) · session 08:00 – 09:30
- Room
- Hall America south
Links
Sign in to access the Paper PDF.
- Preprint PDF(opens in new tab)
Abstract
Large language models are increasingly used to answer questions over annual reports, earnings decks, and analyst notes, yet their outputs remain difficult to verify in high-stakes financial workflows. A fluent answer can blend directly grounded statements, weak synthesis, and unsupported claims across narrative text, tables, and charts. We present EvidenceLens, a visual analytics prototype that treats financial question answering as a claim-evidence alignment problem. The system decomposes an answer into atomic claims, summarizes support composition and confidence--support gaps, and coordinates claim-level inspection with source passages, table cells, and chart regions. Its core representation is a multimodal claim-evidence matrix that exposes coverage, contradiction, and modality imbalance. To support reproducibility, we specify a JSON-based artifact schema, an inspectable multimodal alignment pipeline, and a deterministic review-priority ranking that maps backend signals into an auditable visual structure. Through two report-auditing scenarios, we illustrate how the interface makes grounded, weakly supported, and contradicted claims easier to inspect than in a linear chat transcript.
For Practitioners
Relevant practitioners include financial analysts, data scientists, fintech teams, and compliance/risk professionals. They can use this work to audit LLM-generated financial answers by checking each claim against text, tables, and charts.