Cloud storage has a lot of appeal for enterprise IT: unlimited capacity, unparalleled redundancy and stability, and the promise of transforming data storage from a capital expense into an operating expense. But enterprises also need high-performance, low-latency access to data that is completely protected, secure and accessible anywhere. In order to achieve these goals with the cloud, companies must first answer this question:
How will we move huge amounts of data back and forth between various, dispersed locations?Data needs locality
Moving massive amounts of data is no simple task. It’s so hard on the network that Amazon has a service called Snowmobile that physically transports up to 100 PB on a semi-trailer truck from a customer data center to their cloud.
But the problem is much bigger than uploading data to a public cloud provider. Today’s enterprises need to move information continuously between data centers, office locations, colocation providers, and public and private clouds. The wrench in the works, of course, is latency, because it doesn’t take much to cause applications to perform poorly or even fail, frustrating end users and negatively affecting business operations. Human beings can detect latency as low as 10 milliseconds within applications or from website performance.
Even the fastest networks can suffer from latency woes. Traditionally, when IT encounters latency issues on-prem, they tend to spend money on bigger pipes to gain higher performance. Certainly, if network congestion is an issue, then yes, additional network capacity will help solve the problem. But if an end-user or app is retrieving data from a public cloud that’s a thousand miles away, the fattest pipes on earth won’t reduce latency, because the problem has nothing to do with capacity. The problem is the speed of light.
Amazon, Google and Microsoft tend to place their facilities in the middle of nowhere for economic and security reasons. There can be hundreds and sometimes more than a thousand miles between major metropolitan areas and the nearest public cloud data center. At those distances, significant latency is inevitable. As a result, many enterprises don’t consider the public cloud for business-critical apps or primary storage because they cannot solve this latency problem. Instead, they relegate cloud storage to use cases such as archiving or backup that aren’t as sensitive to latency (though I doubt anyone who’s ever worked on a request to restore a critical file has been able to convince the user that getting their data to them quickly wasn’t mission critical).
Welcome to the data center edge
But while no one can transmit data faster than the speed of light, that doesn’t mean the speed of light problem can’t be overcome. The solution isn’t to move the data faster. The solution is to move the data closer to the edge, which is wherever applications and end-users need access.
At ClearSky, we’ve built a business that approaches this challenge head on. First, to ensure high performance, we deploy and manage a 2U flash appliance on-premises that caches “hot” data locally while also writing it to the cloud.
However, what happens when a user or an app requests data that’s not in the cache, which is commonly called a “cache miss?” Once the data is downloaded from the cloud to the cache, there’s no performance problem, but in the meantime, a user or an app is going to experience latency until it arrives. To us, that’s unacceptable, because cache misses occur about five percent of the time.
To ensure high performance even in the case of a cache miss, we store “warm data” in a metro point of presence (PoP) that’s no more than 120 miles away from the user. Warm data isn’t likely to be accessed within a week, but also doesn’t qualify as cold. In our service, whenever someone requests data that’s not in the local cache, the system retrieves the data from the nearby PoP with latency of about two milliseconds. Latency that low isn’t detectable by the user.
Cold data is generally thought of as information that’s kept for compliance reasons and can include things like old backups, archives and unused files. Latency is typically not a factor with cold data, because users are willing to wait a while when they need it. Our service stores all data, including cold data, in a backing cloud.
The benefits of a metro-based network mean that enterprises don’t have to compromise performance or availability to gain the benefits of the cloud. Our caching services on the edge make all of an IT team’s data management resources appear as if they are nearby. And even if the data isn’t cached locally, it can still be accessed from the nearby metro PoP with no discernable delay. As a result, our edge-based service cuts costs in half, simplifies data management and improves both performance and access for our customers when compared to traditional on-premises storage systems.
So, while we can’t break the speed of light, we can intelligently move data closer to the edge to provide complete data management capabilities to our customers with excellent performance. With the speed of light problem solved, there is no reason for any enterprise to invest in big iron to store and manage their data locally. And there’s no need to have Amazon come out to physically move your data. With our service, we can achieve throughput rates that beat the Amazon truck.
Interested in understanding more about how our edge-based service can help you improve performance, automate backup and DR and make data management much easier? Check out our “Life at the Edge: A cloud computing primer” or sign up for a free trial of our service.