Sarvam Deploys India-Hosted Inference Platform Supporting Eight Open Models
Indian technology company Sarvam has launched its Token Factory, providing a local, low-latency inference stack powered by NVIDIA B200 clusters.
🧠AI AnalysisPhoto: MarkTechPost
Key points
- Sarvam Token Factory offers an OpenAI-compatible API for eight open-weight models hosted in India.
- Infrastructure runs on NVIDIA B200 clusters utilizing advanced caching and separate prefill-decode GPU pools.
- Self-reported benchmarks claim significant reductions in first-token latency and improved throughput per GPU.
Sarvam has opened its Token Factory platform to the public, introducing a managed inference stack hosted directly on domestic GPU clusters in India. The system provides developers with access to eight different models through a single API compatible with OpenAI, aiming to minimize latency by keeping data traffic within the country rather than routing it through overseas cloud providers.
The initial release supports eight open-weight models, including Sarvam's proprietary 105B model alongside versions from GLM, DeepSeek, Kimi, and Gemma. Deployment options are divided into pay-as-you-go shared endpoints and provisioned throughput tiers depending on organizational needs, with on-premise solutions also available for enterprise clients.
Under the hood, the infrastructure relies on advanced caching mechanisms, separate GPU pools for prefill and decode tasks, and a control plane utilizing a cache-aware router. According to self-reported measurements from Sarvam, these architectural adjustments allow the Sarvam 105B model to operate efficiently on a single NVIDIA B200 per replica while maintaining high throughput speeds for heavy agent workloads.
Source MarkTechPost — marktechpost.com/2026/11/17/sarvam-token-factory-india-inference-pl… ↗
Read next
⚖️Regulation & PolicyOpenAI Partners with Chilean Ministry to Advance Public Sector AI
OpenAI has signed a non-binding agreement with Chile's science ministry to integrate artificial intelligence into public administration and support local startups.
🌐Work & SocietyAmbience Healthcare Expands Clinical AI Suite for Hospital Operations
Ambience Healthcare has launched new AI tools designed to assist hospital staff with patient rounding and discharge planning, aiming to improve both clinical efficiency a…
⚖️Regulation & PolicyAWS Highlights Enterprise Governance and AI Workspace Quick Ahead of re:Invent 2026
Amazon Web Services is spotlighting its workplace AI platform, AWS Quick, ahead of its upcoming re:Invent 2026 conference, emphasizing secure deployment across major ente…
🏢Big TechLoanPro Integrates Amazon Bedrock to Automate Loan Servicing Workflows
LoanPro has deployed a multi-agent AI assistant using Amazon Bedrock and Anthropic's Claude models to help lenders manage borrower accounts, resulting in a 15% reduction …
🏢Big TechGoogle Brings Wallet App to iOS Users Across the United States
Google has expanded the availability of its Wallet application to iOS devices in the U.S., allowing Apple users to store digital passes, tickets, and receipts.
⚖️Regulation & PolicyMicrosoft Reveals Cause of Global Outlook Outage
A latent software bug caused a nearly 24-hour global outage of Microsoft’s Exchange Online services, affecting millions of users worldwide.