LIVE
OpenAI Partners with Chilean Ministry to Advance Public Sector AI Ambience Healthcare Expands Clinical AI Suite for Hospital Operations AI Models Vulnerable to Manipulation in Simulated Warfare Scenarios, Research Shows AWS Highlights Enterprise Governance and AI Workspace Quick Ahead of re:Invent 2026 LoanPro Integrates Amazon Bedrock to Automate Loan Servicing Workflows Google Brings Wallet App to iOS Users Across the United States Microsoft Reveals Cause of Global Outlook Outage AWS Outlines Phased Roadmap for Responsible Agentic AI Governance Google updates Gemini settings to separate chat history from AI model training Uber Eats and Zipline launch autonomous drone food delivery in Houston OpenAI Partners with Chilean Ministry to Advance Public Sector AI Ambience Healthcare Expands Clinical AI Suite for Hospital Operations AI Models Vulnerable to Manipulation in Simulated Warfare Scenarios, Research Shows AWS Highlights Enterprise Governance and AI Workspace Quick Ahead of re:Invent 2026 LoanPro Integrates Amazon Bedrock to Automate Loan Servicing Workflows Google Brings Wallet App to iOS Users Across the United States Microsoft Reveals Cause of Global Outlook Outage AWS Outlines Phased Roadmap for Responsible Agentic AI Governance Google updates Gemini settings to separate chat history from AI model training Uber Eats and Zipline launch autonomous drone food delivery in Houston

Sarvam Deploys India-Hosted Inference Platform Supporting Eight Open Models

🧠 AI Analysis 1 min read 👁 5

Indian technology company Sarvam has launched its Token Factory, providing a local, low-latency inference stack powered by NVIDIA B200 clusters.

Sarvam Deploys India-Hosted Inference Platform Supporting Eight Open Models🧠AI Analysis

Photo: MarkTechPost

Key points

  • Sarvam Token Factory offers an OpenAI-compatible API for eight open-weight models hosted in India.
  • Infrastructure runs on NVIDIA B200 clusters utilizing advanced caching and separate prefill-decode GPU pools.
  • Self-reported benchmarks claim significant reductions in first-token latency and improved throughput per GPU.

Sarvam has opened its Token Factory platform to the public, introducing a managed inference stack hosted directly on domestic GPU clusters in India. The system provides developers with access to eight different models through a single API compatible with OpenAI, aiming to minimize latency by keeping data traffic within the country rather than routing it through overseas cloud providers.

The initial release supports eight open-weight models, including Sarvam's proprietary 105B model alongside versions from GLM, DeepSeek, Kimi, and Gemma. Deployment options are divided into pay-as-you-go shared endpoints and provisioned throughput tiers depending on organizational needs, with on-premise solutions also available for enterprise clients.

Under the hood, the infrastructure relies on advanced caching mechanisms, separate GPU pools for prefill and decode tasks, and a control plane utilizing a cache-aware router. According to self-reported measurements from Sarvam, these architectural adjustments allow the Sarvam 105B model to operate efficiently on a single NVIDIA B200 per replica while maintaining high throughput speeds for heavy agent workloads.

Source MarkTechPost — marktechpost.com/2026/11/17/sarvam-token-factory-india-inference-pl… ↗

Contact the newsroom

Got a tip, a proposal or a partnership question? Write to us and we will reply shortly.