LIVE
OpenAI Partners with Chilean Ministry to Advance Public Sector AI Ambience Healthcare Expands Clinical AI Suite for Hospital Operations AI Models Vulnerable to Manipulation in Simulated Warfare Scenarios, Research Shows AWS Highlights Enterprise Governance and AI Workspace Quick Ahead of re:Invent 2026 LoanPro Integrates Amazon Bedrock to Automate Loan Servicing Workflows Google Brings Wallet App to iOS Users Across the United States Microsoft Reveals Cause of Global Outlook Outage AWS Outlines Phased Roadmap for Responsible Agentic AI Governance Google updates Gemini settings to separate chat history from AI model training Uber Eats and Zipline launch autonomous drone food delivery in Houston OpenAI Partners with Chilean Ministry to Advance Public Sector AI Ambience Healthcare Expands Clinical AI Suite for Hospital Operations AI Models Vulnerable to Manipulation in Simulated Warfare Scenarios, Research Shows AWS Highlights Enterprise Governance and AI Workspace Quick Ahead of re:Invent 2026 LoanPro Integrates Amazon Bedrock to Automate Loan Servicing Workflows Google Brings Wallet App to iOS Users Across the United States Microsoft Reveals Cause of Global Outlook Outage AWS Outlines Phased Roadmap for Responsible Agentic AI Governance Google updates Gemini settings to separate chat history from AI model training Uber Eats and Zipline launch autonomous drone food delivery in Houston

Claimseed Launches Synthetic US Medical Documents to Benchmark AI Document Extractors

🧠 AI Analysis 2 min read 👁 0

A new developer tool called Claimseed provides 1,022 synthetic US medical billing documents with planted errors and exact answer keys to evaluate AI extraction models safely.

Claimseed Launches Synthetic US Medical Documents to Benchmark AI Document Extractors🧠AI Analysis

Photo: Hacker News Front Page

Key points

  • Claimseed provides 1,022 fully synthetic US medical billing documents with precise error keys.
  • The package includes degraded copies simulating scans, faxes, and low-resolution photos across 34 scenarios.
  • It operates locally or as an MCP server to score AI extractors on field accuracy and error detection.

A new open development package named Claimseed has been released to help developers test and score AI document extractors, medical bill-review tools, and autonomous agents. The toolkit generates fully synthetic United States healthcare paperwork, including Medicare Summary Notices, hospital itemized bills, and Medicare Advantage explanations of benefits, paired with ground-truth JSON data that detail exactly which billing errors have been planted and where they are located.

The standard distribution features 1,022 blind documents spread across 34 scenarios. This collection includes 260 Medicare Summary Notices, 250 itemized bills, and 512 total preferred provider organization and health maintenance organization explanations of benefits. To ensure rigorous testing conditions, one in every five documents is rendered as a degraded copy simulating scans, faxes, photos, or low-resolution outputs, while standard variations replicate multiple layout variants and electronic remittance advice files.

Developers can use the toolkit to measure field accuracy, error recall, precision, and false alarms by comparing model reports against the provided answer keys. The project also functions as a Model Context Protocol server, enabling chat clients and agents to review page images directly without saving files locally. To prevent data contamination during model training, the release includes specific canary strings and supports generating entirely blind test sets with fresh cryptographic salts.

What to watch next: Adoption of Claimseed by evaluation benchmarks and whether developers keep the test keys strictly segregated from future training corpora.

Source Hacker News Front Page — github.com/claimseed/claimseed ↗

Contact the newsroom

Got a tip, a proposal or a partnership question? Write to us and we will reply shortly.