Server 2012 dedupe awesome: much backups! (don't compress your backups)

[chazzy2501]

I came across a blog thread that made total sense and I had to follow the premise.. It seems obvious but I'd not thought of it.

 

I created a new 2012r2 server and created a dedupped volume in which to store my VM backups. The key is to not encrypt or COMPRESS the backups. This way dedupe can work with the blocks. Traditionally I'd use incremental backups with compression on a standard windows share. This yields a lot of storage but is pretty slow especially at restoring.

 

The beauty of the dedupe is the speed. But it's getting way better results than I expected..

 

Screen shot!

 

 

that's over 7TBs of backup in less than 1.. And its faster.. (De dupe operations run during the day.) So you do need initially more storage.

 

still this is great.

Edited October 21, 2014 by Dos_Box

[3s-gtech]

Will try de-dupe on my Hyper-V host iSCSI backup folders, they're not that big so any saving will help.

[fairm010]

Perhaps you can answer me something to do with de-dupe so that I don't have to make a new thread?

 

 

I finally moved from our file server to a new physical server running 2012(Not R2). What advantages do I have of enabling de-dupe on the data volume and will it delete any existing data?

 

What does it actually do?

 

Ta

[3s-gtech]

Advantages - it essentially links duplicated files to one single master, so 30 copies of a 1GB file will take 1GB. When you move or copy this data elsewhere, it re-inflates. My file shares and home areas are all de-duped - has saved countless GB of storage and enabled me to use DAS effectively.

 

It doesn't delete data as such.

[chazzy2501]

De-dup is a little cooler than that, it looks at the block level (2012r2 has smaller blocks). So if you had 30 versions of a word document, 90% of all the documents are the same so it stores 1 version of that 90%. You shouldn't use this for your VDI solutions or any high frequency files. De-dupe is a per volume solution. But because de-dupe look at the block level encrypting or compressing the files will throw it off.

 

Also if I were to backup my backup folder, it'd need a 7+TB volume as only the server is aware of the de-dupe. So if you made a folder share with 7tb of files but it only took 1Tb of space on the host the backup server wouldn't know and would pull 7TB. (I don't know if any backup solutions are de-dupe aware). I mention this in case you have the urge to offer up massive space to your users.

[smithson83]

You can set De-Dupe to only run on file older than X days, so high frequency files would never get touched, but I would still not run it on a drive containing VDI, VM or SQL etc.

[fairm010]

In that case, I'll turn it on. This will be for shared areas and Home Drives. What de-dupe schedule do you use?

[chazzy2501]

I use the default background one.

[smithson83]

+1 for default background optimisation

[fairm010]

Thanks for your help guys!

[chazzy2501]

OMG update, same folder, same settings... 20TB! under 400GB? This seems optimistic