Data Deduplication

[Bojuexa]

Hi guys,

 

We are running short of space on our server with no option for an upgrade until April. I've been looking into data deduplication. I have tested it in a lab with good results i was just wondering if anyone has this implemented on a production server and any caveats or advise would be appreciated. .

 

Thanks

[mitchell1981]

Definitely switch it on. It works pretty flawlessly for us and saves a lot of space. I think the only thing to check it that your backup will understand the deduplication, as you don't want small data and massive backups!

[steveg]

Also worth checking if you are backing up vm's with it enabled, that the backup software can still do granular file restores - I only discovered ours couldn't when I needed to restore a file - bring a whole vhd back for 1 file was a pain!

[Ratcliffepg]

Had it on for a long time just works you will be amazed how much disk space you will save.... ! But make sure your backup solution works with duplication on drives or shares.

 

If you running server 2019, makes sure your backup server is running the same version of the OS, we had issues with this in the past, causing backups to fail on deduplicated drives, just depends on your backup solution.

[Chaniel]

Definitely enable it, we don't really see any issues and have been running dedup for like 4 years or so.

 

If you want to do any file-level restores with Veeam from a backup that has dedup enabled, the Veaam Console must be installed on a machine that has deduplication enabled as well.

 

If you make any large data deletions, you won't see the space free up immediately. You'll need to start a dedup job (powershell) to force a garbage collection task, or just wait until it does a scheduled one.

[psydii]

In my experience, if you have less than 10Gb of free space dedupe can sometimes stall. When this happened to us we were able eat into our reserves and increase the size of the disk presented to Windows 10Gb at a time.

 

vcentre shouts at me every time I open it up now. So glad covid forced us off into the cloud.

[ozydave]

Yep defo go for it, had it running on our production servers for ages with no problems

[Netwacky87]

Like others, had it enabled for years! Saves us about 3.6TB across our file server, circa 42% reduction! #winner!

[minimoo]

One point to note - when i enabled it, i almost filled up a SAN and had to pause the job for 24 hours - moving data around to save space increased the data on the san temporarily (until it cleaned up etc) - I saved 1TB, but added 2TB onto the san temporarily.

[Bojuexa]

I did wonder about this with us been very low on space. Thanks for heads up

 

[FragglePete]

Just to add - if your running VMs on this server, I know for Veeam the Hosts themselves have to have data deduplication added as a role also otherwise the backups will fail if deduplication is switched on the VM.

 

Pete

[mdrabble]

Any reason why I shouldn't run a Dedup Garbage collection manually in the day whilst users are using the system?

 

Cheers

[Chaniel]

Any reason why I shouldn't run a Dedup Garbage collection manually in the day whilst users are using the system?

 

Cheers

 

If you run a Full garbage collection, you could potentially experience some performance issues but it depends on your setup really.

 

https://docs.microsoft.com/en-us/troubleshoot/windows-server/backup-and-storage/full-garbage-collection-performance-issue

 

Difference between regular and full garbage collection here. I've run regular garbage collections during the day with no issue, never found it necessary to do a full.