Salesforce Optimizes Global AI Model Inference with Amazon SageMaker
Salesforce has implemented cross-Region inference routing in Amazon SageMaker to manage AI model capacity across 30+ global regions more efficiently.
🧠AI AnalysisPhoto: AWS Machine Learning Blog
Key points
- Salesforce deployed cross-Region inference routing to resolve capacity fragmentation across 30+ AWS regions.
- The new system uses SageMaker-managed routers to direct traffic based on real-time endpoint health and availability.
- The architecture supports data residency compliance while enabling rapid scaling of the Agentforce Foundation's model fleet.
Salesforce’s Agentforce Foundation team has integrated cross-Region inference routing within Amazon SageMaker to address the challenges of managing AI model capacity across more than 30 global regions. Previously, Salesforce relied on a custom-built internal gateway to handle traffic spikes, which suffered from a lack of real-time capacity signals and required traffic to traverse public networking, increasing the risk of cascading failures.
The new SageMaker-managed routing system allows Salesforce to consolidate its model serving infrastructure. By using a centralized router, the company can now direct requests to available endpoints across different AWS accounts and regions. This setup ensures that if a local endpoint reaches capacity or experiences errors, the system automatically shifts traffic to healthy, pre-deployed endpoints without requiring manual intervention.
Key operational improvements include:
- Capacity Awareness: The router evaluates the health of destination endpoints based on recent request outcomes, including timeouts and 5xx errors.
- Secure Networking: By utilizing SageMaker’s native routing, Salesforce avoids the need for traffic to leave the virtual private cloud via public gateways.
- Scalability: The system allows for the grouping of model endpoints under logical targets, preventing localized issues from impacting unrelated AI services.
This implementation enables Salesforce to meet strict data residency requirements while maintaining the flexibility to scale its AI fleet rapidly as new business units are onboarded.
Read next
🏢Big TechMicrosoft Challenges Rivals on AI Pricing at Ignite Conference
Microsoft is positioning its Copilot AI as the more cost-effective choice for businesses compared to OpenAI and Anthropic, citing a commissioned study to support its clai…
🚀Startups & FundingOpenAI CEO Sam Altman Addresses Upcoming Biographical Film
OpenAI CEO Sam Altman commented on the upcoming film 'Artificial,' which portrays his life and the 2023 leadership crisis at the company.
🔌Chips & HardwareSnapchat Integrates AI Coding Assistants into AR Lens Development
Snapchat has launched the Easy Lens CLI, enabling developers to build augmented reality experiences using AI coding tools like Claude Code and Cursor.
🚀Startups & FundingRadixArk Unveils DaisyRose-27B, Focused on Agentic Efficiency
Startup RadixArk has launched DaisyRose-27B, an AI model optimized for long-horizon agent tasks, claiming significant improvements in training efficiency and reasoning ca…
🔌Chips & HardwareUS Eyes the Philippines for Massive AI and Chip Hub Amid Domestic Opposition
President Donald Trump visited the Philippines to advance the Pax Silica initiative, proposing a 1,620-hectare hub for data centers and chip fabs to bypass local pushback…
🚀Startups & FundingBest Buy Reintroduces Physical Media After Brief Exit
Best Buy has resumed selling Blu-ray discs directly through its website and physical retail locations, reversing a policy that saw the retailer exit the physical media ma…