AZURE (or other Cloud Services) for School Data Backup

[Willott]

Thanks @gjseaman - had been quoted that part number but not given any clues as to what it was or what it gets us - further investigation suggests it's $100 credit on Azure, does that correlate with what you're receiving?

[gjseaman]

I am using it at present it give 5tb of storage to recovery services and or any services you wish to try and as I previously stated we pay £90/month. We are also hoping in the near future to use Azure Iaas and move a majority of our server into the cloud, does anyone here have any previous knowledge. Edited March 16, 2016 by gjseaman

[Dave_O]

We are also hoping in the near future to use Azure Iaas and move a majority of our server into the cloud, does anyone here have any previous knowledge.

 

What servers (services) were you looking to move out to the cloud?

[Michael]

What I'm struggling to get my head around is the sheer volume of data - 5TB is a lot, what kind of WAN connections do you guys have?

 

I appreciate Azure Backup works on an incremental basis, but initially it must take days surely to do a full backup?

 

Also (worse case scenario), if you needed a full restore, it also would take days which would be considered an unacceptable turn around.

[Steve21]

We're running 100Mb, but our idea (if prices were to fit) was about doing this as an off-site emergancy style backup. So even if it takes a week to upload the weekly full etc, it wouldn't be a problem as speeds would be capped during day and let loose overnight etc. (Normal backups all local still)

 

But yes it'll take a long time for sure Same in regards to restores I guess, as it's only for a oh sh* moment e.g. Loss of buildings, then we'd just get SIMs down quickly, and let rest download slower as you'd need to order hardware/mobile buildings etc in anyway

 

Steve

[Michael]

With that volume of data though, a DFS setup would probably be more appropriate - two physical/virtual servers or more in different physical locations on site.

 

I know DFS isn't bullet proof, but it's a lot cheaper and more manageable in that respect. You obviously still need to do backups. I guess once the main backup is done, incremental backups thereafter would be what - a couple of Gigabytes at most? It's hard to say. 2GB would take around 3 minutes on a 100Mbps WAN.

Edited May 4, 2016 by Michael

[Dave_O]

I appreciate Azure Backup works on an incremental basis, but initially it must take days surely to do a full backup?

 

Also (worse case scenario), if you needed a full restore, it also would take days which would be considered an unacceptable turn around.

 

MS do initial offline backups or seeding, where you can send them your initial data on 3.5" SATAs (suitably encrypted of course)

https://azure.microsoft.com/en-gb/blog/new-features-in-azure-backup-long-term-retention-offline-backup-seeding-and-more/

[Michael]

Ahh - that actually makes a lot of sense.

[Michael]

Next question - who on Edugeek has actually done this?

 

Bought a 4TB+ drive, copied data using the wizard (presumably) and shipped it to Microsoft?

[CyberNerd]

Next question - who on Edugeek has actually done this?

 

Bought a 4TB+ drive, copied data using the wizard (presumably) and shipped it to Microsoft?

 

yes, but with Cloud Platform, Azure is way more expensive. The example I gave was backup storage costing us £35 per month for EU storage yet others are paying £90 for the same thing in Azure. I DO have to pay for recovery though so I suspect the pricing differs there.

[Michael]

I suspect the more subscribers Microsoft get, the cheaper it'll get, as well as the competition element too.

 

I guess the one advantage is you're using Microsoft technologies throughout, so in terms of reliability/compatibility it should just work. I suspect Windows Server 2016 will provide more integrated services out the box once enabled from Server Management.

[drewp]

Microsoft Azure Backup Server (Data Protection Manager without tape library support) is also free but, requires and Azure Backup Vault subscription.

https://azure.microsoft.com/en-gb/blog/announcing-microsoft-azure-backup-server/

[robyholmes]

We backed up just over 1TB over our 75Mbps connection and it took around 3 days. We limited the transfer speed during the day so it didn't really affect our connection. After that it runs in under an hour most nights (One backup a day). However every now and again it does a full backup again. For us this doesn't matter, we use the Azure DPM to backup locally more often so its only the offsite backup which might get more than 24 hours out of date.

 

I look at recovery times like this, if you've lost your onsite backup your in a bad way. So waiting 3 days for a recovery to take place (You could restore key bits first faster) isn't going to be bad compared to rebuilding or buy servers etc.

[HPlum78]

https://azure.microsoft.com/en-gb/documentation/articles/backup-configure-vault/

https://azure.microsoft.com/en-us/blog/azure-egress-fee-waiver-for-the-academic-community/

 

Take a look at the above

 

[Michael]

We backed up just over 1TB over our 75Mbps connection and it took around 3 days. We limited the transfer speed during the day so it didn't really affect our connection. After that it runs in under an hour most nights (One backup a day). However every now and again it does a full backup again. For us this doesn't matter, we use the Azure DPM to backup locally more often so its only the offsite backup which might get more than 24 hours out of date.

 

I look at recovery times like this, if you've lost your onsite backup your in a bad way. So waiting 3 days for a recovery to take place (You could restore key bits first faster) isn't going to be bad compared to rebuilding or buy servers etc.

 

That's a valid point - I don't know whether under the BGfL I can throttle bandwidth to an IP or hostname for example. I suspect it's comparatively easy with the likes of Smoothwall or other.

[robyholmes]

That's a valid point - I don't know whether under the BGfL I can throttle bandwidth to an IP or hostname for example. I suspect it's comparatively easy with the likes of Smoothwall or other.

 

I did this in the Azure Backup Server (DPM) software, nothing extra needed.

[MYK-IT]

Just thought I would update this thread as I've noticed that On-line Backups is a topic of conversation on Edugeek at the moment.

 

Previously...

 

We rebuilt our Dell NX3010 with MS Server 2012 R2, along with DPM 2012 R2 (plus updates) and re-catalogued our media via our Dell TL2000 Tape Library. A procedure kindly mentioned / suggested by other Edugeek'ers (Thanks).

 

The process proved beneficial in many ways; as deduplication is now supported and we (re)configured the protection groups more efficiently (as original configuration was set-up in collaboration with a 3rd party).

 

Don't get me wrong there are still some (frustrating) quirks with DPM! but I have been persevering with it (for now) due to its v.low cost...

 

Latest...

 

Registered for the free 30 day trial of Azure, and associated and enabled Azure backups via DPM 2012 R2 to test, and to see if it is indeed a financial and viable option for us. After setting up the free account, registration of Azure with DPM was straight forward and DPM protection groups were (re)configured for On-line protection.

 

But I have now come across (yet!) another issue.

 

I (wrongly) assumed that when DPM backed up On-line (to Azure) it would be of a similar method as to how backing up to Tape Media works (copy latest short-term backup from local disk pool to tape/on-line etc.) but not so. The Azure Agent requires an additional 10-15% (of the protected data size) of storage space to be available; stored in \Program Files\Microsoft Azure Recovery Services Agent\Scratch\VHDs by default (but you can redirect to another location) to allow the Azure on-line function to work.

 

This is now proving problematic for when our DPM Server attempts to backup protection groups containing (a) higher amount of data (For example, backing up 5+ TB of data requires >500GB of additional (temp) disk space). I know that doesn't sound a lot; but when you only have 146GB (RAID 1) for OS and all the remaining HDDs are RAID'ed and allocated as the local storage pool (with no no HDD bays left) you can realise my predicament!

 

Certainly been one of those 1 step forward, 2 steps back type of project!

 

Just thought I would mention this additional required (storage) resource in case others are also contemplating using Azure on-line with DPM etc.

Edited May 10, 2016 by MYK-IT

[Michael]

I've started to trial Azure Backup and so far so good. Once I registered a cheap SSL cert with godaddy, it was relatively straight forward following the on screen instructions.

 

I suppose my only two complaints (at the moment), is on 2008 R2 (unlike 2012), there's no option for bandwidth throttling in Azure Recovery Services. Secondly there's no option to enable reporting... an e-mail would be really useful. I know I probably could create a Task in Windows Server, but the point is, I shouldn't have to!

 

For those comparing Microsoft's Azure, vs. Google's offering. When specifying GRS (Geo-Redundant Storage) - six copies of your data are created. Three times in the primary region (West Europe) and three times in another region - again, presumably West Europe, it's certainly big enough!

 

The only thing I'm not 100% sure on what exactly the difference is, block blob vs. file storage. 1TB is £18.77 per month vs. £62.56 per month. Regardless how you look at it, we're not talking ridiculous sums of money when you consider six copies of your data are created, in two different locations 100's of miles apart.

 

If I've understood correctly, the 1TB quota in practice is in fact 6TB, seeing as your data is replicated six times.

[Tallwood_6]

I've started to trial Azure Backup and so far so good. Once I registered a cheap SSL cert with godaddy, it was relatively straight forward following the on screen instructions.

 

I suppose my only two complaints (at the moment), is on 2008 R2 (unlike 2012), there's no option for bandwidth throttling in Azure Recovery Services. Secondly there's no option to enable reporting... an e-mail would be really useful. I know I probably could create a Task in Windows Server, but the point is, I shouldn't have to!

 

For those comparing Microsoft's Azure, vs. Google's offering. When specifying GRS (Geo-Redundant Storage) - six copies of your data are created. Three times in the primary region (West Europe) and three times in another region - again, presumably West Europe, it's certainly big enough!

 

The only thing I'm not 100% sure on what exactly the difference is, block blob vs. file storage. 1TB is £18.77 per month vs. £62.56 per month. Regardless how you look at it, we're not talking ridiculous sums of money when you consider six copies of your data are created, in two different locations 100's of miles apart.

 

If I've understood correctly, the 1TB quota in practice is in fact 6TB, seeing as your data is replicated six times.

 

Blob storage is unstructured object based storage i.e. you this for vhd's, filer storage is smb storage.

If your looking at using azure for backups then its worth sticking your data on Azure Cool Blob Storage that's not long gone GA https://azure.microsoft.com/en-us/blog/introducing-azure-cool-storage/ if your concerned with cost.

[Michael]

It gets more confusing, but here are the differences:

 

 

When I look at the Azure Calculator, 1TB of GRS File Storage comes to £62.56 per month, yet under Hybrid Integration > Backup, 1TB of GRS storage with no protected options comes £30.03 per month. I'm guessing it's in fact the latter?

[steve_forbes]

To be able to utilise Azure’s block blob storage, for file and system state back up, there are two options:

- Microsoft Azure Backup Server (MABs); D2D2C

- Microsoft Azure Backup; D2C

Both of these options allow for physical and virtual machines to be backed up and the way in which they are costed are essentially the same. Both MABs and Azure backup don’t have any additional licencing costs, but the size of each VM/physical machine to be backed up needs to be understood. As does the total size of the data that will be stored in Azure, either per month or over the course of a year.

Understanding the size of each VM should be a fairly simple task and the cost should remain fairly consistent. Once this is understood you can then work out your protected instances cost based on the following three tiers:

- <50GB servers (VM’s) e.g. SQL servers; £3.05 per month per server/instance

- 50-500GB servers (VM’s) e.g. general server DR backups; £6.11 per month per server/instance

- >500GB servers (VM’s) e.g. file servers; £6.11 per month per 500GB (e.g, x1 1TB files server = £12.22 per month)

The next task is to understand how much data in total is going to end up being stored in block blob storage (LRS will be fine for this). To do this a calculator of some sort will be required; as the system state backup will need to be understood both from a size and number of restore points required point of view. The user data will also need to be understood in both of these ways, as well as understanding how long you required each type of user data to be stored for. Once both of these are understood then a monthly and yearly cost can be estimated. As an example for LRS block blob it works out about £180 per TB per year.

So; now you can actually cost what backing up to azure will cost, which is great. The next task is setting it all so you put in place what you have costed. Each server or VM will need to have its own set of backup policies, including retention periods on the files for archiving. If this is set up incorrectly then all the costs work out above will be incorrect and the costs of backing up to Azure could balloon.

As with any cloud based backup service the key to getting it right is getting the data consumption under control. This where RM is going to be adding value. We are creating the calculator to help estimate the costs and then provide the installation services to ensure that the cost model from the calculator is put into practise correctly.

As a final note on storage; Microsoft have recently released cool and cold storage options. These are not currently available for using with Azure backup/MABs; the only storage options available currently are LRS and GRS storage. As soon as cool and cold storage can be utilised we will be including them in our cloud backup service for archiving purposes.

[drewp]

Been using MABS for a while with a backup vault in North Europe, was setup before the UK Datacentres were available.

 

Has anyone looked at moving their backup vaults from North Europe to the UK datacentres? Not been able to find out from MSDN forums or web searches if it can be done...

 

Would it be a case of creating a new vault in the UK region and starting again from scratch?