There’s no argument that the cloud offers a wealth of benefits. From virtually unlimited capacity to extreme redundancy and stability, all delivered as-a-service as an operating expense, the cloud is now a key part of almost every organization’s IT strategy.
We also know, however, that the traditional cloud most people think of — huge amounts of storage located far from urban centers — isn’t a fit for every use case. And no category of file data is trickier for cloud than rich media: videos, images, audio files, and the like. As rich media usage grows, companies are increasingly looking for ways to leverage the cloud to manage and store it.
In a recent webinar, NAPC’s CTO, Robert Palmas, and CRO, Mike Gershowitz discussed their own experiences attempting to store rich media files directly in public cloud. The integrator had been tasked with testing whether a major consumer products company would be able to follow their corporate mandate to store all of their creative team’s data — which included a lot of large rich media files — in the public cloud.
Watch our on-demand webinar: Blazingly Fast Storage-as-a-Service For Rich Media Production Environments!
Awful performance from the cloud
NAPC’s tests found that storing the creative team’s data in the public cloud was completely unworkable. For example, it took 6 minutes to upload three 200kb files, an operation that happens in a blink of an eye on-premises. Uploading a couple of 4 GB files was even worse, taking 25 minutes. Browsing performance was just as bad. It took two minutes to draw in a folder with just 30 items, and folders with more than 200 items were essentially inaccessible — the operation never completed.
The cause of such poor performance is latency due to distance. The cloud data center was half a continent away, and at that distance, not even the speed of light can travel fast enough to provide good performance. And once you start dealing with very large rich media files, latency issues become even worse. Additionally, the SMB and AFP protocols they were using were simply not up to the task. They’re chatty, requiring a lot of back and forth communications, and don’t respond well to lots of packets arriving in a jumbled order, which added further inefficiency and delay.
Using the cloud alone to store rich media files would have ground the creative team’s productivity to a virtual halt. Clearly, this type of performance was beyond unacceptable, and the test results shut down the conversation about cloud. No cost savings could compensate for the loss of productivity.
The next option NAPC considered was a gateway. Unfortunately, gateways have similar issues. If there’s a cache miss at the local gateway, you have to go to the cloud, and all the performance issues caused by latency emerge once again.
So, is it even possible to get real-time access to large production files — with no delay — and still benefit from the savings the cloud has to offer? There’s another option to consider.
Get to the edge
When latency in the cloud is an issue, the edge can provide a powerful solution. When the cloud is combined with resources at the edge close to customers, latency problems disappear.
The first “step” in ClearSky’s edge solution is an on-premises 2U flash appliance. This stores your most frequently accessed data — “hot data” — locally to provide excellent performance. Hot data accounts for about 10% of the full data set, but makes up more than 90% of the data that’s actually accessed by an organization. So, what about the problem of local cache misses? How does ClearSky avoid the “gateway” problem?
To completely eliminate latency issues, and make the cloud an option for these large, frequently accessed media files, you need another step. For ClearSky, it’s our metro points of presence (PoPs) on the edge.
These PoPs are never more than 120 miles from the user, meaning latency never gets above a couple of milliseconds, which is undetectable by humans. So, whenever a file doesn’t appear in the local cache, the system simply retrieves it from the nearby PoP, and users don’t notice any difference. We term the type of data we store in these PoPs as “warm data”, meaning files that are likely accessed less than once a week, but aren’t archival.
All data, including cold data, is stored in a public backing cloud. And ClearSky never transmits enterprise data over the public Internet. We use only dedicated, private lines, which improves both performance and security.
Read our solution brief on how ClearSky’s service provides high-performance NAS
The edge in real life
Sandy Alexander, a 450-person graphics communication company, was looking at a full SAN replacement. Their five-year-old all-flash array was quickly running out of capacity — their creative teams work with very large image and video files. Not wanting re-up with an on-premises storage system, the company looked to the cloud. But as we laid out above, though, the traditional cloud wasn’t going to get the job done, due to latency and other issues. That’s when the company looked to ClearSky.
The results have been stunning. The company was able to decommission its entire on-premises SAN and rely on ClearSky for all its primary storage, backup and disaster recovery (DR). It now manages one durable copy of its data, available to end users with flash performance.This has reduced the company’s IT burden, while significantly reducing the total cost of storage, all while keeping performance high for end users.
The cloud is a powerful tool, but it needs the edge to provide storage as a service with the performance that enterprises require. Once the two are combined, organizations can gain all the benefits of the cloud, with none of the poor performance.
Want to see for yourself how ClearSky can liberate your organization from managing and storing rich media files on-prem? Sign up today for a demo.