+ Post New Thread
Page 1 of 2 12 LastLast
Results 1 to 15 of 18
How do you do....it? Thread, Clustered SAN in Technical; I have 2x HP Procurve ML115 Servers (Opteron 1352 (Quad 2.10Ghz 4x 512mb) 4GB DDR2 RAM with 4x WD RE3 ...
  1. #1

    Join Date
    Mar 2008
    Location
    Boston, Lincolnshire
    Posts
    189
    Thank Post
    1
    Thanked 8 Times in 8 Posts
    Rep Power
    14

    Clustered SAN

    I have 2x HP Procurve ML115 Servers (Opteron 1352 (Quad 2.10Ghz 4x 512mb) 4GB DDR2 RAM with 4x WD RE3 1TB HDD's and a 16GB Lexar Jumpdrive.

    I'm trying to set these up to run as a SAN with Load balancing and fail-over, using the USB as the boot drive and 3 of the HDD's in a RAID 5 array or an XFS+1 Array and the last drive as a hot swap.

    I was planning on using NexcentaStor 2.0.0 Develeper Edition but then realised that i could only set each server up as a seperate SAN and not cluster them... I'm now at a bit of a sticky spot so any advice or ideas would be great!

    I'm looking at Openfiler atm, but having dificulty loading it onto the USB drive. Also been looking at FreeNAS but without any joy on how to cluster them...

    Thanks in advance!

  2. #2

    Join Date
    Mar 2008
    Location
    Boston, Lincolnshire
    Posts
    189
    Thank Post
    1
    Thanked 8 Times in 8 Posts
    Rep Power
    14
    After some research and test installs freeNAS looks the most promising, i have both servers installed with FreeNAS and each have a share accessable over the network.

    SAN01/DATA & SAN02/DATA.

    I'm now trying to setup UNISION to syncronise the shared data folders across the two FreeNAS servers. Does anyone know how to do this?

  3. #3


    Join Date
    Jan 2006
    Posts
    8,202
    Thank Post
    442
    Thanked 1,032 Times in 812 Posts
    Rep Power
    339
    Quote Originally Posted by hotwired007 View Post

    I'm now trying to setup UNISION to syncronise the shared data folders across the two FreeNAS servers. Does anyone know how to do this?
    you shouldn't need to synchronise the data.
    If you set up the file system using a clustered filesystem such as GFS then they should be able to read and write to the same SAN disk simultaneously.
    clustering without GFS isn't really clustering

  4. #4

    Join Date
    Mar 2008
    Location
    Boston, Lincolnshire
    Posts
    189
    Thank Post
    1
    Thanked 8 Times in 8 Posts
    Rep Power
    14
    The filesystem is UFS (GPT and Soft Updates) the default for FreeNAS.

    All the information points to using UNISON or RSync to syncronise the two shares.

  5. #5


    Join Date
    Jan 2006
    Posts
    8,202
    Thank Post
    442
    Thanked 1,032 Times in 812 Posts
    Rep Power
    339
    ok, thats not really clustering - you won't be able to use the two servers simultaneously if you just copy the data periodically.

    I would go with rsync - it's pretty easy to set up and you can tunnel it over ssh.
    rsync

    redhat.com |
    if you were to do it again...

  6. #6

    Join Date
    Mar 2008
    Location
    Boston, Lincolnshire
    Posts
    189
    Thank Post
    1
    Thanked 8 Times in 8 Posts
    Rep Power
    14
    my understanding with unison was that it was a constant syncronisation.

    I intended on using the servers to store a number of VMs on each and for example have 8 VMs total but run 4 each from the individual NAS boxes...

    Unfortunately its a difficult situation although i'm now looking at OpenFiler to run a HA Clustered environment. I'm goign to buy a couple more HDDs for the Servers and trial them with it although they wont turn up til tommorow so i'll keep trying with the FreeNAS system.

  7. #7


    Join Date
    Oct 2006
    Posts
    3,409
    Thank Post
    184
    Thanked 356 Times in 285 Posts
    Rep Power
    148
    Quote Originally Posted by hotwired007 View Post
    Unfortunately its a difficult situation although i'm now looking at OpenFiler to run a HA Clustered environment.
    Again it isnt really clustering in how you are thinking of it, its just a backup server which is a straight mirror of the primary using DRBD. When the primary goes down the HA part kicks in and promotes the backup, which takes over the services. Works really well, but only 1 server can be a target at a time in DRBDs normal operation. (IIRC you can couple it with a cluster FS and have 2 primaries, but that certainly isnt its default operation)

    In other words; HA = yes. Backup = yes, Load balencing = No.
    For load balencing you need a cluster FS.
    Last edited by j17sparky; 29th July 2009 at 12:36 PM.

  8. #8


    Join Date
    Jan 2006
    Posts
    8,202
    Thank Post
    442
    Thanked 1,032 Times in 812 Posts
    Rep Power
    339
    straight mirror of the primary using DRBD
    I recall dhicks built a HA virtual host using DRBD
    DRBD:What is DRBD

    That is realtime, as is GFS. I'm pretty sure unison is not (but I not used it)

  9. #9

    Join Date
    May 2011
    Location
    Florianopolis, Brazil
    Posts
    4
    Thank Post
    0
    Thanked 0 Times in 0 Posts
    Rep Power
    0

    Results of this cluster?

    Hello guys!

    I'm doing the same for a FreeNAS storage. Looking for some redundancy - not necessarily fail-over.

    I will build an environment with some VMware ESX machines and I need a reliable FreeNAS with redundancy - trying to eliminate SPOF (Single Point of Failure).

    I already have a solution for the VMware, but not for the NAS.

    I thought about DRDB and an exclusive 1GB link between them, but I'm not sure if it will have the appropriated performance. Maybe kinda of DRDB asynchronous between master and slave?

    Can you both tell me about your environments? How is the final set up? How is the performance? What you think is the bottleneck? Did you guys do some reliability/performance analyzing on that?

    Thanks in advance!

    Lucas.

  10. #10

    Join Date
    May 2011
    Location
    Florianopolis, Brazil
    Posts
    4
    Thank Post
    0
    Thanked 0 Times in 0 Posts
    Rep Power
    0
    ah, I need to finish the planning of this project by wednesday. =(

    sorry for the hurry, but if you guys can help me out with some opinions, I would really appreciate.

    thanks again,

    Lucas

  11. #11


    Join Date
    Oct 2006
    Posts
    3,409
    Thank Post
    184
    Thanked 356 Times in 285 Posts
    Rep Power
    148
    Are you going to be using SSDs? If not I doubt very much that you will cause a bottleneck in the 1gb link between the SANs/NASs. It will most likely be at your HDs. I was in the same boat too when I was first setting up DRBD so I know what you are thinking. dhicks has been running DRBD now for some years, drop him a PM.

    There's absolutely tonnes of info on here about DRBD and aggregation of NICs, have a search.

    I havn't looked at FreeNAS for quite some time but when I was setting up a few years ago Openfiler seemed far better suited for the job as it was targeted at enterprise where as freenas was for home users.

  12. #12

    Join Date
    May 2011
    Location
    Florianopolis, Brazil
    Posts
    4
    Thank Post
    0
    Thanked 0 Times in 0 Posts
    Rep Power
    0
    No SSD's. SATA and SAS HD's.

    I need to use FreeBSD, and DRDB is not available for BSD, there is the equivalent, but not that reliable as I've read around.

    FreeNAS looks very reliable now, and no much difference between them as I google (I've never worked with them, my opinion is just about googling).

    I will send him a message! I would like to find someone who had set up a FreeNAS + any kinda system redundancy.

    thanks!

    lucas.

  13. #13

    dhicks's Avatar
    Join Date
    Aug 2005
    Location
    Knightsbridge
    Posts
    5,610
    Thank Post
    1,224
    Thanked 772 Times in 670 Posts
    Rep Power
    234
    Hello Lucas - I got your PM, had a look at this thread. As pointed out above, I've been running DRBD on assorted hardware for a while - at my previous school I had all our Xen VMs running on DRBD-mirrored volumes. This means you have local performance for disk read operations, only disk write operations have a delay (probably similar/identical as to what you'd get using a dedicated SAN server) as the written data has to go accross the network to the mirrored device. At the kind of scale that most schools operate at (a couple of servers hosting virtual machines), I think mirrored volumes make more sense than a central iSCISI server - you do have to buy (and power) double the number of harddisks, but then you save on a whole dedicated server and you get proper, full redundancy

    DRBD's protocol works differently than iSCISI. As I understand it, iSCISI can make better use of aggregated network links - a single DRBD connection will not use more bandwidth than is available to one network port, i.e. a DRBD connection is limited to 1Gb/s even if running over an agregated 4Gb/s link. That said, as pointed out above, it is unlikly that will actually make too much difference to you - remember, only write operations need to be sent over the network anyway, so the bandwidth required by DRBD isn't really all that great.

    As I've just recently discovered, DRBD's default setup (sensibly) assumes it is running on a general network, shared with other devices, and only uses up to 30% of the available bandwidth. If you have a dedicated storage area network (i.e. the original meaning of "SAN", although most people now seem to take that as meaninga large, central iSCISI server) you should set DRBD to use all the bandwidth it wants.

    FreeNAS looks very reliable now, and no much difference between them as I google
    I think OpenFiler actually has DRBD built-in for setting up mirrored devices.

    How much / what sort of redundancy are you aiming for here? Bear in mind that mirrored block devices are not backup - delete a file on the master volume and DRBD will quickly and efficiently delete it on the secondary volume with a minimum of delay. Your best bet might be to use rsync to take regular (hourly?) backups of your primary file server, placing the backups on a secondary server and using deduplication to store them (i.e. for identical files just store a hard link to the original file, don't copy the original). Then, just make that server a Samba server with the same file permissions as the primary server, able to serve files directly to users. That way, you not only get a backup of your files that people can go to for previous file versions, you can also use that file server as the main file server in an emergency, maybe loosing an hour's work in the change-over. I underatand FreeNAS supports acting as an rsync server, so this should be easy enough to set up.

  14. #14

    maniac's Avatar
    Join Date
    Feb 2007
    Location
    Kent
    Posts
    3,037
    Thank Post
    209
    Thanked 425 Times in 306 Posts
    Rep Power
    144
    Not set one up, but I have done a fair bit of reading. This guide seems about the best I've found using open filer as it's prefered package - it's what I intend to follow when I try a simelar project in a few months time - just doing some proofs of concept on some old hardware to start with.

    Setting up a High Availability NAS/SAN using Openfiler | Easy to Read Articles

  15. #15

    Join Date
    May 2011
    Location
    Florianopolis, Brazil
    Posts
    4
    Thank Post
    0
    Thanked 0 Times in 0 Posts
    Rep Power
    0
    Hi,

    I also think mirrored disks are better for this situation, I'm looking for redundancy, not a backup. For backup I will use Bacula.

    I just got that my hardware are not that good as I imagined, and they are not that proper also, as they support only up to 4 HD's.

    so by now I'm almost giving up on the NAS/SAN and have a local RAID with a good backup.

    my main problem with the NAS is that I would store all the system in there, so if I got a problem with the NAS, ALL THE SYSTEMS WILL SHUT DOWN! and this makes me think a lot about this decision.

    as I didn't find a professional solution case, I will give up temporarily on that.

    thanks a ton for your reply!

    Lucas.

SHARE:
+ Post New Thread
Page 1 of 2 12 LastLast

Similar Threads

  1. Building a clustered Supercomputer
    By CyberNerd in forum *nix
    Replies: 9
    Last Post: 6th March 2013, 08:29 PM
  2. SAN's - Who Has Them
    By barryfl in forum Hardware
    Replies: 67
    Last Post: 26th November 2009, 09:30 AM
  3. SAN discs
    By TronXP in forum Windows Server 2008
    Replies: 5
    Last Post: 26th March 2009, 08:56 AM
  4. SAN storage
    By Theblacksheep in forum Hardware
    Replies: 3
    Last Post: 20th November 2008, 01:08 PM
  5. Blade with SAN
    By matt40k in forum Hardware
    Replies: 7
    Last Post: 12th November 2008, 09:35 PM

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •