OpenAI speeds ChatGPT storage 12× with global caching
What is Rapidly scaling online storage to serve over 1 billion ChatGPT users (Part II)?
OpenAI upgraded its Habitat storage layer to support over a billion ChatGPT users, adding reliable multi-tenant isolation, a worldwide cache, and Azure Cosmos DB as the backing store. The changes cut read latency by a factor of twelve, making responses feel snappier for everyone.
Who it is for
Developers building AI services at scale should note the patterns for low-latency, multi-tenant data access.