+ Post New Thread
Results 1 to 3 of 3
*nix Thread, deduplication of mapped drives in Technical; Is it worth it? I've done some experiments, similar to this thread of 2 yrs ago: http://www.edugeek.net/forums/nix/49...mpression.html using zfs block ...
  1. #1


    Join Date
    Jan 2006
    Posts
    8,202
    Thank Post
    442
    Thanked 1,032 Times in 812 Posts
    Rep Power
    340

    deduplication of mapped drives

    Is it worth it?

    I've done some experiments, similar to this thread of 2 yrs ago:
    ZFS - Deduplication (Dedup) Vs Compression

    using zfs block level deduplication. Just using the shared drives I had a dedup ration of 1.16% (around a terabyte of data in total). If I added in staff homedrives to the pool the ratio went upto about 1.80%

    I can see some advantages:

    nearly doubles the amount of disk space available with compression
    deduplication over the wire means really fast backups
    simple to add new disks to a zfs pool and increase size
    self healing filesystem
    handy for shared files if google drive== homedrive

    and some disadvantages, mostly learning related:

    learning freebsd/solaris over linux
    different quota system
    not getting full benefit from hardware raid5 on SAN (I presented the disks separately to the server so that I'm not causing performance issues by raiding it twice)
    performance/RAM overhead
    disks are cheap anyway, saving 1TB only really saves me a couple of hundred quid.

    I keep going around in circles with it. what should I do?
    Last edited by CyberNerd; 1st May 2012 at 07:04 PM.

  2. #2

    dhicks's Avatar
    Join Date
    Aug 2005
    Location
    Knightsbridge
    Posts
    5,731
    Thank Post
    1,292
    Thanked 797 Times in 693 Posts
    Rep Power
    239
    Quote Originally Posted by CyberNerd View Post
    Just using the shared drives I had a dedup ration of 1.16% (around a terabyte of data in total). If I added in staff homedrives to the pool the ratio went upto about 1.80%
    It'd be interesting to see how that changed for media - photos, video and audio. People have a tendancy to dump stuff from camera memory cards onto shared file areas any old how, and having a deduplicated file system would avoid that becoming too much of a problem.

    We've just set up a new file server using FreeNAS, which is based on FreeBSD and uses ZFS. We don't have enough RAM in the machine to use deduplication, though. The management interface to set up file shares and so on seems to be quite usable - we seem to have file areas with appropriate Windows file permissions and so on. I seem to remember snapshot performance under ZFS being considered rather better than with LVM under Linux - LVM's write performance drops with each snapshot you do. You can still SSH in to the machine to run scripts and so on, there shouldn't be that much actual BSD stuff to deal with.

    not getting full benefit from hardware raid5 on SAN (I presented the disks separately to the server so that I'm not causing performance issues by raiding it twice)
    If you have the time to compare, you could try ZFS on a hardware RAID array and tell us the results. Would a JBOD array give you the write perfomance of the RAID array without the RAID-5 overhead/complications?

  3. #3


    Join Date
    Feb 2007
    Location
    51.403651, -0.515458
    Posts
    9,637
    Thank Post
    250
    Thanked 2,910 Times in 2,142 Posts
    Rep Power
    830
    The following article might help...?

    http://nex7.com/readme1st

  4. Thanks to Arthur from:

    dhicks (2nd May 2012)



SHARE:
+ Post New Thread

Similar Threads

  1. LoginHook to map a couple of network drives
    By Greenbeast in forum Mac
    Replies: 11
    Last Post: 12th September 2014, 06:57 PM
  2. RM CC3 - one of our mapped drives has disappeared
    By mcnallyfc in forum Network and Classroom Management
    Replies: 3
    Last Post: 11th October 2010, 12:12 PM
  3. Replies: 7
    Last Post: 27th November 2006, 12:26 PM
  4. Accssing mapped drives in word
    By Uraken in forum Windows
    Replies: 7
    Last Post: 19th October 2006, 08:24 AM
  5. Replies: 4
    Last Post: 7th December 2005, 09:09 PM

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •