+ Post New Thread
Page 2 of 3 FirstFirst 123 LastLast
Results 16 to 30 of 40
General Chat Thread, Deduplication - anyone else using it? in General; Originally Posted by sonofsanta But... but... the interface! *cough* mmc *cough*...
  1. #16


    Join Date
    Dec 2005
    Location
    In the server room, with the lead pipe.
    Posts
    4,638
    Thank Post
    275
    Thanked 778 Times in 605 Posts
    Rep Power
    223
    Quote Originally Posted by sonofsanta View Post
    But... but... the interface!
    *cough* mmc *cough*

  2. #17

    sonofsanta's Avatar
    Join Date
    Dec 2009
    Location
    Lincolnshire, UK
    Posts
    4,954
    Thank Post
    862
    Thanked 1,443 Times in 991 Posts
    Blog Entries
    47
    Rep Power
    616
    I know, I know... really I should just get on with server core and do the job properly. In my more conspiratorial moments I have suspected that this is Microsoft's intention, and that server core itself is a plan to boost the numbers taking MS certification.

  3. #18

    Join Date
    Jan 2012
    Posts
    13
    Thank Post
    0
    Thanked 1 Time in 1 Post
    Rep Power
    0
    Just to give you some quick results from some fileservers - just because some beat the 28%. Need to do some CPD on the need not to copy from the shared area to my documents! Obviously big advantages up front - has anyone come across any downsides?

    Processed files: 331578
    Processed files size: 539.95 GB
    Optimized files size: 328.72 GB
    Space savings: 211.17 GB
    Space savings percent: 39

    Processed files: 117387
    Processed files size: 200.52 GB
    Optimized files size: 126.07 GB
    Space savings: 74.43 GB
    Space savings percent: 37

    Processed files: 150567
    Processed files size: 298.56 GB
    Optimized files size: 223.50 GB
    Space savings: 75.05 GB
    Space savings percent: 25

  4. #19
    british_government's Avatar
    Join Date
    Feb 2009
    Posts
    160
    Thank Post
    24
    Thanked 20 Times in 15 Posts
    Rep Power
    36
    I'm not sure if it is a downside or not, we are looking at putting all our file shares on one drive on one server to get the most out of deduplication. We run a hyper-v environment so really this shouldn't have any performance impact compared to on 2 different virtual servers and 4 different virtual drives at the moment.

    Just feels like adding a single point of failure, but really it isn't I guess!

  5. #20


    Join Date
    Dec 2005
    Location
    In the server room, with the lead pipe.
    Posts
    4,638
    Thank Post
    275
    Thanked 778 Times in 605 Posts
    Rep Power
    223
    Just poked around on one of ours (2008R2, running SIS) and it's quite effective.

    Deduplication - anyone else using it?-quite-effective.png

  6. #21

    localzuk's Avatar
    Join Date
    Dec 2006
    Location
    Minehead
    Posts
    17,680
    Thank Post
    516
    Thanked 2,451 Times in 1,897 Posts
    Blog Entries
    24
    Rep Power
    832
    There is a downside to deduplication, and to SIS. If a sector becomes corrupt for whatever reason, and it happens to be a file (SIS) or 'chunk' of a file (Deduplication) that has multiple copies across the disk, you end up losing all those copies.

    But then, we all have backups!

  7. #22

    Join Date
    Sep 2008
    Posts
    188
    Thank Post
    6
    Thanked 71 Times in 29 Posts
    Blog Entries
    3
    Rep Power
    25
    Quote Originally Posted by localzuk View Post
    We're hitting a 25% deduplication rate here. Which is pretty good, considering we've not moved everything over to that server yet either.
    Got 24% on a 1.5Tb fileserver, was hoping for more. Obviously staff don't copy the things into different folders as much as I thought they did

  8. #23

    localzuk's Avatar
    Join Date
    Dec 2006
    Location
    Minehead
    Posts
    17,680
    Thank Post
    516
    Thanked 2,451 Times in 1,897 Posts
    Blog Entries
    24
    Rep Power
    832
    25% is about right for user files according to Microsoft's guides. Up to 50% for shared areas. 75% for software deployment areas and 90% for VHDs.

  9. #24

    FN-GM's Avatar
    Join Date
    Jun 2007
    Location
    UK
    Posts
    15,942
    Thank Post
    886
    Thanked 1,694 Times in 1,472 Posts
    Blog Entries
    12
    Rep Power
    447
    Hi,

    I have enabled this in production now. But when i look at the disks in Server Manager i can see a deduplication rate of 30% i right click and refresh it changes to say 28%. I refresh it again and it goes back up to 31%. Why is it so erratic, changing the figures in seconds? Will it settle down?

    Thanks

  10. #25
    ass17's Avatar
    Join Date
    Feb 2013
    Posts
    326
    Thank Post
    5
    Thanked 35 Times in 34 Posts
    Rep Power
    26
    Quote Originally Posted by RTFM View Post
    We have sedum on our SAN, it's fantastic
    Tell me more... Sedum

  11. #26

    Join Date
    Sep 2008
    Posts
    188
    Thank Post
    6
    Thanked 71 Times in 29 Posts
    Blog Entries
    3
    Rep Power
    25
    Quote Originally Posted by localzuk View Post
    25% is about right for user files according to Microsoft's guides. Up to 50% for shared areas. 75% for software deployment areas and 90% for VHDs.
    Checked the machine two weeks later and the 25% has gone up to 38%. Does it improve over time?

  12. #27

    localzuk's Avatar
    Join Date
    Dec 2006
    Location
    Minehead
    Posts
    17,680
    Thank Post
    516
    Thanked 2,451 Times in 1,897 Posts
    Blog Entries
    24
    Rep Power
    832
    The process is done via a pair of scheduled tasks each night. I believe they have a limit of how long they run for each night also. So, yes, the values can go up over time as it optimises more and more.

    I have come across a slight issue with backups. Backing up using BackupAssist seems to have issues with the hard links that are used, and the backups often just end up as a bunch of 4k files rather than the actual files.

  13. #28

    FN-GM's Avatar
    Join Date
    Jun 2007
    Location
    UK
    Posts
    15,942
    Thank Post
    886
    Thanked 1,694 Times in 1,472 Posts
    Blog Entries
    12
    Rep Power
    447
    Quote Originally Posted by FN-GM View Post
    Hi,

    I have enabled this in production now. But when i look at the disks in Server Manager i can see a deduplication rate of 30% i right click and refresh it changes to say 28%. I refresh it again and it goes back up to 31%. Why is it so erratic, changing the figures in seconds? Will it settle down?

    Thanks
    This seems to have stopped overnight.

  14. #29

    Ephelyon's Avatar
    Join Date
    Aug 2008
    Location
    Cheshire, England
    Posts
    1,661
    Thank Post
    288
    Thanked 318 Times in 192 Posts
    Rep Power
    141
    Our Veeam setup makes great use of dedupe for the backups. I forget the ratio now but it helps a lot, not just for storage but also (because this is source dedupe) in terms of how much data we have to transfer over the wire for our offsite backups to another school a few miles up the road.

  15. #30

    Join Date
    Jan 2010
    Posts
    62
    Thank Post
    7
    Thanked 5 Times in 5 Posts
    Rep Power
    10
    Quote Originally Posted by ghanel View Post
    Just to give you some quick results from some fileservers - just because some beat the 28%. Need to do some CPD on the need not to copy from the shared area to my documents! Obviously big advantages up front - has anyone come across any downsides?

    Processed files: 331578
    Processed files size: 539.95 GB
    Optimized files size: 328.72 GB
    Space savings: 211.17 GB
    Space savings percent: 39

    Processed files: 117387
    Processed files size: 200.52 GB
    Optimized files size: 126.07 GB
    Space savings: 74.43 GB
    Space savings percent: 37

    Processed files: 150567
    Processed files size: 298.56 GB
    Optimized files size: 223.50 GB
    Space savings: 75.05 GB
    Space savings percent: 25
    what command did you run to get that info? Also is there a way to list the most duplicated files?

SHARE:
+ Post New Thread
Page 2 of 3 FirstFirst 123 LastLast

Similar Threads

  1. Lanschool - Anyone that uses it please help
    By mattmoo in forum Network and Classroom Management
    Replies: 16
    Last Post: 7th December 2011, 11:43 AM
  2. Anyone here used IT's Learning as their VLE?
    By Sam_Brown in forum Virtual Learning Platforms
    Replies: 9
    Last Post: 13th September 2010, 09:33 PM
  3. Avatar 3D - Anyone else seen it?
    By flyinghaggis in forum General Chat
    Replies: 44
    Last Post: 1st February 2010, 01:43 PM
  4. Replies: 8
    Last Post: 10th January 2008, 09:34 AM
  5. Replies: 1
    Last Post: 11th May 2007, 09:18 AM

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •