Meta Deploys LLM-Generated 'JiTTests' to Catch Risky Code Changes
Meta has integrated an artificial intelligence system that automatically writes executable tests to catch vulnerabilities before code is merged into production.
Key points
- Meta moves JiTTests from research to production infrastructure.
- The system uses LLMs to generate failing tests for risky code diffs.
- Caught 74 privacy vulnerabilities in the first half of 2026.
Meta has transitioned its Just-in-Time Tests system, known as JiTTests, from a research project into active production infrastructure. Developed to operate across Meta's massive internal codebase, the system uses large language models to generate executable tests specifically designed to fail when a risky code change is introduced. Unlike traditional testing methods that guard against future regressions, these tests provide concrete behavioral evidence during the code review stage.
According to the company's published findings, the tool successfully caught 74 privacy-related software engineering vulnerabilities during the first half of 2026, contributing to a 20% drop in such incidents. Operating overnight on spare server capacity, the multi-stage pipeline evaluates risky diffs, generates test candidates, and filters them using an ensemble of judge models alongside deterministic rule-based checks before alerting engineers.
While the approach offers concrete proof of bugs prior to merge, internal deployment figures highlight ongoing challenges with false positives. Early production rollouts initially confirmed a significantly lower fraction of reported catches, though engineers eventually reported a 60% true-positive rate at full scale.
What to watch next
Keep an eye on whether Meta expands these automated testing methodologies to open-source developer toolkits or if competitors adopt similar execution-based AI validation models.
Source MarkTechPost — marktechpost.com/2026/11/16/meta-jittests-llm-catching-tests-privac… ↗
Read next
⚖️Regulation & PolicyOpenAI Partners with Chilean Ministry to Advance Public Sector AI
OpenAI has signed a non-binding agreement with Chile's science ministry to integrate artificial intelligence into public administration and support local startups.
🌐Work & SocietyAmbience Healthcare Expands Clinical AI Suite for Hospital Operations
Ambience Healthcare has launched new AI tools designed to assist hospital staff with patient rounding and discharge planning, aiming to improve both clinical efficiency a…
🧠AI AnalysisAI Models Vulnerable to Manipulation in Simulated Warfare Scenarios, Research Shows
New research reveals that leading AI models from firms like OpenAI and Anthropic can be manipulated into identifying civilian targets when prompted with game scenarios.
⚖️Regulation & PolicyAWS Highlights Enterprise Governance and AI Workspace Quick Ahead of re:Invent 2026
Amazon Web Services is spotlighting its workplace AI platform, AWS Quick, ahead of its upcoming re:Invent 2026 conference, emphasizing secure deployment across major ente…
🏢Big TechLoanPro Integrates Amazon Bedrock to Automate Loan Servicing Workflows
LoanPro has deployed a multi-agent AI assistant using Amazon Bedrock and Anthropic's Claude models to help lenders manage borrower accounts, resulting in a 15% reduction …
🏢Big TechGoogle Brings Wallet App to iOS Users Across the United States
Google has expanded the availability of its Wallet application to iOS devices in the U.S., allowing Apple users to store digital passes, tickets, and receipts.