Arthur Posted August 21, 2012 Posted August 21, 2012 (edited) Now everyone can afford to backup off-site. Source: Amazon Web Services Blog / Website: aws.amazon.com/glacier http://i.imgur.com/qG0Xm.png We introduced Amazon S3 in March of 2006. S3 growth over the past 6+ years has been strong and steady, and it now stores over one trillion objects. Glacier builds on S3's reputation for durability and dependability with a new access model that was designed to be able to allow us to offer archival storage to you at an extremely low cost. To store data in Glacier, you start by creating a named vault. You can have up to 1000 vaults per region in your AWS account. Once you have created the vault, you simply upload your data (an archive in Glacier terminology). Each archive can contain up to 40 Terabytes of data and you can use multipart uploading or AWS Import/Export to optimize the upload process. Glacier will encrypt your data using AES-256 and will store it durably in an immutable form. Glacier will acknowledge your storage request as soon as your data has been stored in multiple facilities. Creating a vault in Amazon Glacier. Glacier will store your data with high durability (the service is designed to provide average annual durability of 99.999999999% per archive). Behind the scenes, Glacier performs systematic data integrity checks and heals itself as necessary with no intervention on your part. There's plenty of redundancy and Glacier can sustain the concurrent loss of data in two facilities. At this point you may be thinking that this sounds just like Amazon S3, but Amazon Glacier differs from S3 in two crucial ways. First, S3 is optimized for rapid retrieval (generally tens to hundreds of milliseconds per request). Glacier is not (we didn't call it Glacier for nothing). With Glacier, your retrieval requests are queued up and honored at a somewhat leisurely pace. Your archive will be available for downloading in 3 to 5 hours. Each retrieval request that you make to Glacier is a called a job. You can poll Glacier to see if your data is available, or you can ask it to send a notification to the Amazon SNS topic of your choice when the data is available. You can then access the data via HTTP GET requests, including byte range requests. The data will remain available to you for 24 hours. Retrieval requests are priced differently, too. You can retrieve up to 5% of your average monthly storage, pro-rated daily, for free each month. Beyond that, you are charged a retrieval fee starting at $0.01 per Gigabyte (see the pricing page for details). So for data that you’ll need to retrieve in greater volume more frequently, S3 may be a more cost-effective service Edited August 21, 2012 by Arthur
Arthur Posted August 21, 2012 Author Posted August 21, 2012 Just noticed this in the small print. Deleting data from Amazon Glacier is free if the archive being deleted has been stored for three months or longer. If an archive is deleted within three months of being uploaded, you will be charged an early deletion fee. In the US East (Northern Virginia) Region, you would be charged a prorated early deletion fee of $0.03 per gigabyte deleted within three months.
pete Posted August 21, 2012 Posted August 21, 2012 (edited) That has got me thinking. Wonder how long it'll be before it begins to creep into other backup tools. I suspect deletions are expensive in terms of resource use, so they'd like to keep them to a minimum. Edited August 21, 2012 by pete
TechMonkey Posted August 21, 2012 Posted August 21, 2012 Waiting 3-5 hours to get at your site backups when you are trying to restore may be an issue. Looks good for non-essential long term archiving though.
Arthur Posted August 21, 2012 Author Posted August 21, 2012 Wierd to be charged for deleting data? Retrieval of data too. Why does Amazon have to make it so complicated! According to this post they charge based on how long it takes them to retrieve the data. The hourly retrieval rate would be the amount of data you requested divided by how long it takes them to retrieve it (3.5 - 4.5 hours). If it takes them 4 hours to retrieve your 3TB, then your peak hourly retrieval rate would be 768GB / hour (3072 GB / 4 hours). Your billable hourly retrieval rate would be 768GB - 1.28GB (3072 * .05 / 30 / 4 hours). Total retrieval fee: 766.72 * 720 * .01 = $5,520.38 (~180x your monthly storage fee) The pricing appears to not be optimized for retrieving all your data in one fell swoop. This particular example appears to be a worst case scenario for restoration because you haven't split up your data into multiple archives (doing so would allow you to reduce your peak hourly retrieval by spacing out your requests for each archive) and you want to restore all your data (the free 5% of your data stored doesn't help as much when you want to restore all your data). (Source)
Arthur Posted August 21, 2012 Author Posted August 21, 2012 Glacier is primarily designed for the archival of data that is rarely/never accessed.
SimpleSi Posted August 22, 2012 Posted August 22, 2012 As Arthur says - they've seen a market for "just in case but prob never going to need" archiving and have priced it at the Dropbox pricing model "You'd have to be mad not to pay us a few quid a month for the fantastic service we offer" And with the access times they specify - they can afford to compress it down to 32bytes controlled by a fractal algorithm Simon
Arthur Posted August 22, 2012 Author Posted August 22, 2012 Wired has a good explanation of the fees involved... Is There a Landmine Hidden in Amazon’s Glacier? « Wired An Amazon spokesperson says “For a single request the billable peak rate is the size of the archive, divided by four hours, minus the pro-rated 5% free tier.” It doesn’t seem likely that this is Amazon’s intent for the pricing, but it has made some developers nervous. “If you wrote an automated script to safely pull a full archive, a simple coding mistake, pulling all data at once, would lead you to be charged up to 720 times what you should be charged,” one Hacker News commenter wrote.
localzuk Posted August 22, 2012 Posted August 22, 2012 (edited) All of this would make me nervous if I were to consider using it as a disaster recovery storage solution (it wouldn't be suitable for a backup solution, due to the speed at which restoration is required for files). If they could clarify the pricing with examples, that would help. The cost for storage would be cheap, cheap, cheap. But in the event of a disaster? Say you're like us and a full backup is around 4TB. You want to get your systems back up and running asap after a disaster so you request it all to download. The end result would be a bill of $7360.49 to recover all your data. Would many schools be willing to pay that? Or would they prefer to pay more evenly over time, with no direct cost to recover? Edited August 22, 2012 by localzuk
CyberNerd Posted August 22, 2012 Posted August 22, 2012 The end result would be a bill of $7360.49 to recover all your data. Would many schools be willing to pay that? We only back up to an onsite storage device. Having the extra security of an offsite backup sounds like a really good step forward. I would be MORE than happy to pay a few thousand pounds on the off chance it was needed (it never has been needed in the 20 or so years that the school has had IT) - so it would certainly be cheaper than offsite tape storage.
glennda Posted August 22, 2012 Posted August 22, 2012 The end result would be a bill of $7360.49 to recover all your data. Would many schools be willing to pay that? Or would they prefer to pay more evenly over time, with no direct cost to recover? In the Event of DR you would just have it listed in the plan - we need to pay to get it up and running again, just as in any other factor such as new hardware etc
CyberNerd Posted August 22, 2012 Posted August 22, 2012 In the Event of DR you would just have it listed in the plan - we need to pay to get it up and running again, just as in any other factor such as new hardware etc I reckon it would be negligible compared to the amount we'd pay for hardware/overtime/consultancy in the event of a disaster (fire or something)
glennda Posted August 22, 2012 Posted August 22, 2012 I reckon it would be negligible compared to the amount we'd pay for hardware/overtime/consultancy in the event of a disaster (fire or something) Exactly - to rebuilt a server room even of the basic variety costs thousands of pounds 5k for the data is nothing especially if you compared hosting that 4TB data with somebody like databarracks etc who will do the restore for free. And you could probably convince and Insurance company to pay for it!
localzuk Posted August 22, 2012 Posted August 22, 2012 Exactly - to rebuilt a server room even of the basic variety costs thousands of pounds 5k for the data is nothing especially if you compared hosting that 4TB data with somebody like databarracks etc who will do the restore for free. And you could probably convince and Insurance company to pay for it! We must be in a lucky situation here. We pay about £2k per annum for off-site backup, with no limits on our data storage or retrieval. But it is with a local college who are a partner in our trust.
glennda Posted August 22, 2012 Posted August 22, 2012 We must be in a lucky situation here. We pay about £2k per annum for off-site backup, with no limits on our data storage or retrieval. But it is with a local college who are a partner in our trust. Is that for 4TB's? From a rough guess I have a client who pays roughly £500 per month (6kpa) for a little less then that! So reducing that to sub $50 a month ($600pa) and you paying the restore fee after 2 years which you may never need.
localzuk Posted August 22, 2012 Posted August 22, 2012 (edited) Is that for 4TB's? From a rough guess I have a client who pays roughly £500 per month (6kpa) for a little less then that! So reducing that to sub $50 a month ($600pa) and you paying the restore fee after 2 years which you may never need. We have about 20 or 30TB stored up there. (We store unlimited copies of documents as they change, and then any file that is deleted is after 60 days), and it includes a 100Mb fibre between us and them. (Its our backup solution, so not just DR). Edited August 22, 2012 by localzuk
glennda Posted August 22, 2012 Posted August 22, 2012 We have about 20 or 30TB stored up there. (We store unlimited copies of documents as they change, and then any file that is deleted is after 60 days), and it includes a 100Mb fibre between us and them. (Its our backup solution, so not just DR). Then thats fair enough - I wouldn't want to put that on a normal cloud hosted solution! that's thousands a month!
Arthur Posted October 21, 2012 Author Posted October 21, 2012 Fast Glacier looks quite good. The Best Tools for Uploading Files to Amazon Glacier « Digital Inspiration http://i.imgur.com/W9FWp.png
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now