Claimseed Launches Synthetic US Medical Documents to Benchmark AI Document Extractors
A new developer tool called Claimseed provides 1,022 synthetic US medical billing documents with planted errors and exact answer keys to evaluate AI extraction models safely.
Photo: Hacker News Front Page
Key points
- Claimseed provides 1,022 fully synthetic US medical billing documents with precise error keys.
- The package includes degraded copies simulating scans, faxes, and low-resolution photos across 34 scenarios.
- It operates locally or as an MCP server to score AI extractors on field accuracy and error detection.
A new open development package named Claimseed has been released to help developers test and score AI document extractors, medical bill-review tools, and autonomous agents. The toolkit generates fully synthetic United States healthcare paperwork, including Medicare Summary Notices, hospital itemized bills, and Medicare Advantage explanations of benefits, paired with ground-truth JSON data that detail exactly which billing errors have been planted and where they are located.
The standard distribution features 1,022 blind documents spread across 34 scenarios. This collection includes 260 Medicare Summary Notices, 250 itemized bills, and 512 total preferred provider organization and health maintenance organization explanations of benefits. To ensure rigorous testing conditions, one in every five documents is rendered as a degraded copy simulating scans, faxes, photos, or low-resolution outputs, while standard variations replicate multiple layout variants and electronic remittance advice files.
Developers can use the toolkit to measure field accuracy, error recall, precision, and false alarms by comparing model reports against the provided answer keys. The project also functions as a Model Context Protocol server, enabling chat clients and agents to review page images directly without saving files locally. To prevent data contamination during model training, the release includes specific canary strings and supports generating entirely blind test sets with fresh cryptographic salts.
What to watch next: Adoption of Claimseed by evaluation benchmarks and whether developers keep the test keys strictly segregated from future training corpora.
Source Hacker News Front Page — github.com/claimseed/claimseed ↗
Read next
⚖️Regulation & PolicyOpenAI Partners with Chilean Ministry to Advance Public Sector AI
OpenAI has signed a non-binding agreement with Chile's science ministry to integrate artificial intelligence into public administration and support local startups.
🌐Work & SocietyAmbience Healthcare Expands Clinical AI Suite for Hospital Operations
Ambience Healthcare has launched new AI tools designed to assist hospital staff with patient rounding and discharge planning, aiming to improve both clinical efficiency a…
⚖️Regulation & PolicyAWS Highlights Enterprise Governance and AI Workspace Quick Ahead of re:Invent 2026
Amazon Web Services is spotlighting its workplace AI platform, AWS Quick, ahead of its upcoming re:Invent 2026 conference, emphasizing secure deployment across major ente…
🏢Big TechLoanPro Integrates Amazon Bedrock to Automate Loan Servicing Workflows
LoanPro has deployed a multi-agent AI assistant using Amazon Bedrock and Anthropic's Claude models to help lenders manage borrower accounts, resulting in a 15% reduction …
🏢Big TechGoogle Brings Wallet App to iOS Users Across the United States
Google has expanded the availability of its Wallet application to iOS devices in the U.S., allowing Apple users to store digital passes, tickets, and receipts.
⚖️Regulation & PolicyMicrosoft Reveals Cause of Global Outlook Outage
A latent software bug caused a nearly 24-hour global outage of Microsoft’s Exchange Online services, affecting millions of users worldwide.