Pete Cheslock

DevOps, RelEng, DevTools, Automation, Randomness

But Wait, There's Less (Durability)!

Amazon recently announced a new tier of storage available within their web services cloud infrastructure.  Amazon’s current storage solution, S3, is truly the gold standard for durable cloud based storage that provides 99.999999999% durability (which if my math is right, means that for every 100 Billion objects stored in S3, Amazon “may” lose a single object every year). Amazon is listening to their customers, and now provides a lower cost (33% cheaper) S3 storage solution called Reduced Redundancy Solution (RRS).

This new service will allow companies who have systems in the AWS cloud to leverage a lower cost solution to store data that can easily be retrieved from another location or re-processed from source data.  You can now design your application to request data from S3 RRS, and if the data object does not exist, your application can retrieve it from another location or recreate the object (maybe from source data on the more resilient S3 storage).   This may be a great solution for companies that are using EC2 just for computing or processing large data sets, S3 RRS could reduce your monthly costs by quite a bit.

The only problem with this new storage service is that the cost is still pretty high for what you get.   Also, AWS did not decrease the cost for the data transfer to the RSS service (it’s the same as the standard S3), so they can potentially lose your data and the worst of it is you have to pay them to replace it.  But with a 99.99% reliability over the course of the year we’re probably not talking about a lot of data.

My company makes its living by storing an unlimited amount of our customers' data indefinitely.  We can do this because we can leverage the highly resilient and low cost S3 storage to keep email, IM’s and social media messages on our system forever without making huge capital outlays in storage systems, colo’s, etc…   I think it’s great that AWS has brought in this lower tier of storage to differentiate itself with the other up and coming and established cloud computing providers by not forcing everyone to use the same “expensive” long term storage.  But for my money, it’s worth it to pay such a small amount more (5c per GB) for such a large improvement in resiliency.