bossman Posted January 29, 2011 Posted January 29, 2011 We have 2 Sans, one is a sun 7110 which is now supported by Oracle but I have heard they will be dropping the hardware side of things but luckily Hitachi will support anyone with an Oracle San which is good news as we also have a Hitachi SMS 100 which gives us 6.4Tb formatted space for student and staff data is connected direct from DCs and user servers via iSCSI and and our Sun 7110 is our virtualisation backend to which we have 2 but soon to be 3 Dell 2950s attached via ethernet through 2 D-Link Gb managed switches (working ok at the moment but may change for Juniper of similar spec). 12k for the both of them which should last us for the next ten years if we get as much use out of them that we have from our previous server hardware (nearly 10 years on Windows server 2000, yes and solid as a rock) Both are working very well, very quick and will suit our needs when we use them in anger in four weeks time when we migrate over to them. I can highly recommend them if that is the way you want to go. I looked at EMCs fibre SANs but to be honest I think unless you have huge storage needs and lots of students (more than 2000) then iSCSI will suffice.
teejay Posted January 29, 2011 Posted January 29, 2011 Just to clear the support thing up, Oracle are continuing with the 7000 series unified storage range with new hadrware recently released as well, so no problems with ongoing support for those :-) What they have dropped is their high end Sun SANS which were rebadged versions of someone elses SANs I believe. They are however about to launch some new high end gear, but that really isn't in the ballpark for schools.
mister_z Posted January 29, 2011 Posted January 29, 2011 What issue(s) are you trying to solve ? Performance woes ? Manageability ? Maintenance costs ? I generally don't recommend FC for smaller shops. Yes it's the king of security and performance, but do you really need it and do you want to pay for it ? Some parts of my shops are FC ( we're big on EMC and eminently satisfied ), with multi-TB transactional databases we need the performance. But for 4 of our satellite sites -- with 100% Windows and less that 50 servers in each -- a NAS-based solution (EMC Celeras, in fact) fills the bill nicely and leaves out the complications of additional fiber work. There's already 10G and gigabit and network expertise, so it's been mostly a plug-and-play operation. Several senior leaders "thought" we needed fiber, but I talked them out of it and now they're glad I did. For FC work, if you really want uptime, you'll want two switches, with all critical servers dual-pathed to each in case one dies, and to permit regular firmware updates. That means 2x the number of servers in switch0ports (which you will pay for), some sort of multipathing (EMC PowerPath or similar for Windows or Unix, native multipathd for Linux), and connections to each FC appliance and a dual-port FC HBA in each server. Sketch it out, add up the costs and see if you think it worth it. The NAS alternative is a second GigE NIC in each server, extra GigE blades in your network switches, and a separate VLAN for the NAS traffic to reduce contention. Do you really think you'll overwhelm that ? Do you have severe contention with your existing server-based files ? Coraid makes some interesting ATAoE stuff I've been reading about, but I have no direct experience there. I keep hoping for a new site / new project to cough up some R&D money
SYNACK Posted January 29, 2011 Posted January 29, 2011 Thought I'd chip into this thread. First things first there are only two forms of connectivity for SANs - Fibre Channel or Ethernet. SAS is only used for DAS setups (Direct Attached Storage). We have a SAS based SAN servicing 3 servers, check IBMs product lineup if you don't belive me.
Soulfish Posted January 29, 2011 Posted January 29, 2011 Coraid makes some interesting ATAoE stuff I've been reading about, but I have no direct experience there. I keep hoping for a new site / new project to cough up some R&D money Coraid were interesting, but I was concerned by the ATAoE driver support in XenServer/VMWare. They certainly put forward a compelling case but the uncertainty of a small company always being able around to provide driver updates meant we ended up looking elsewhere. We have a SAS based SAN servicing 3 servers, check IBMs product lineup if you don't belive me. It may very well be a SAN, but SAS connectivity makes it more like a DAS . You can't network SAS connections, only directly connect from the SAS port to the server (well some blade chassis provide "SAS network modules", but thats more to expose internal blades to SAS connectivity). It may be that IBM have a SAN solution that is also able to provide SAS connections (as well as other more typical forms of SAN connectivity) to act more like a DAS but I'd say it's a pretty unusual setup. SAS connectivity is pretty cheap, but is unable to scale as high as fibre or ethernet and has the disadvantage that if your servers aren't within a short distance of your SAN/DAS you're unable to connect to it.
SYNACK Posted January 29, 2011 Posted January 29, 2011 It may very well be a SAN, but SAS connectivity makes it more like a DAS . You can't network SAS connections, only directly connect from the SAS port to the server (well some blade chassis provide "SAS network modules", but thats more to expose internal blades to SAS connectivity). It may be that IBM have a SAN solution that is also able to provide SAS connections (as well as other more typical forms of SAN connectivity) to act more like a DAS but I'd say it's a pretty unusual setup. SAS connectivity is pretty cheap, but is unable to scale as high as fibre or ethernet and has the disadvantage that if your servers aren't within a short distance of your SAN/DAS you're unable to connect to it. It is more like DAS but there again so is iSCSI if you look at HBA based solutions, it just looks like DAS to the local system. You are right that it is limited in range from the SAN to the server but in the lower cost end it is much more expandable than iSCSI based SANs. The aggregation of iSCSI connections is only avalible in more expencive units so you are limited to max 2GBit/s with multipathing. In comparison SAS is 3GB/s per channel which with dual pathing works out as 6GB/s withou the overhead of aggregation. This makes it vastly quicker for the money involved. Both hp and IBM are dropping iSCSI from their entry level range and replacing it with SAS as it is much quicker and cheaper which is why we went for it as a solution over iSCSI.
Duke Posted January 31, 2011 Posted January 31, 2011 Hold up, I'm confused. How are you doing Storage Area NETWORK over SAS? I could have a SAN/NAS that uses SAS disks, and has a SAS connection between the disk tray and the controller, but surely the SAN itself has to be using a network-based protocol to connect to client machines, otherwise it's not a SAN? You could have a disk array + controller that had multiple SAS ports and it could be connected to multiple clients, but you could only connect as many clients as there were SAS ports - thus it is DIRECT attached storage surely? My Ethernet based SAN can be plugged into a switch, and thus connected to as many clients as I like (via other switches) so it's a Storage Area NETWORK. Happy to be hit with a CLUE bat if I've misunderstood.
SYNACK Posted January 31, 2011 Posted January 31, 2011 Hold up, I'm confused. How are you doing Storage Area NETWORK over SAS? I could have a SAN/NAS that uses SAS disks, and has a SAS connection between the disk tray and the controller, but surely the SAN itself has to be using a network-based protocol to connect to client machines, otherwise it's not a SAN? You could have a disk array + controller that had multiple SAS ports and it could be connected to multiple clients, but you could only connect as many clients as there were SAS ports - thus it is DIRECT attached storage surely? My Ethernet based SAN can be plugged into a switch, and thus connected to as many clients as I like (via other switches) so it's a Storage Area NETWORK. Happy to be hit with a CLUE bat if I've misunderstood. Depends on your deffinition of network, it has a couple of controllers in the SAN with multiple SAS ports. It has a heafty bunch of software that came with it which allows it to show up as drives so it implements a custom protocol over the SAS connectors I think. It is caled a SAN by their sales team and although it is not connectable to a network switch it does share a resource to multiple clients over a custom protocol. It may be streaching the deffinition a bit but I still think that it fits in the SAN catagory. 1
Duke Posted January 31, 2011 Posted January 31, 2011 It may be streaching the deffinition a bit but I still think that it fits in the SAN catagory. True, just because it's not IP-based doesn't mean it can't be a network I guess. I think you're right in that a lot of the original SAN stuff wasn't necessarily Ethernet/IP based, I suppose I always just think of the 'current' definition that most SAN-resellers use, but it probably varies from vendor to vendor.
CHiLL Posted January 31, 2011 Author Posted January 31, 2011 So let me summarise; Going down the FC route. This would require: SAN > FC Switch > Server > Core Switch? (Seeminly expensive) Going down the iSCSI route. This would require SAN/NAS > Core switch over Ethernet. (Inexpesive compared to Fibre) They are two main routes from my understanding. Both will have central management, and both can backup to tape drives. I need to determine whether SAN or NAS would be the best solution for our environment. I shall do more research. From my understanding, if you want many TBs of storage, SAN is the best option to go down. Is this correct?
SYNACK Posted January 31, 2011 Posted January 31, 2011 I would still recommend a seporate switch for the iSCSI stuff as pushing it all over your core switch even in a VLAN will put a lot of load on your switch and may slow it down due to congestion. I have used a seporate VLAN on a SAN before but these were with very beefy core switches and not much usage of the SAN - only a couple of servers. SAN or NAS depends on how you want to access your storage, a SAN will offer RAW space that shows up as space you can allocate to servers, all file access is done through these servers. A NAS makes a network share itself so you don't go through the server to get the files, they are usually connected to the main network like a server. You can spec large amounts of storage for both. The SUN S7000 people keep talking about is both a SAN and a NAS and can take drive shelves for lots of storage.
Duke Posted January 31, 2011 Posted January 31, 2011 Going down the FC route. This would require: SAN > FC Switch > Server > Core Switch? (Seeminly expensive) I honestly don't know FC that well, but I think you'll also need to throw HBAs in there for every device you want to connect to the SAN directly (rather than through storage presented by a server). Unless you have significant FC structure then do NOT go for FC. FC isn't just normal networking using Fibre like you would around the rest of the network, it's a whole different protocol that you'd need to learn and support. If you already have 10Gb Ethernet infrastructure then I don't see how FC would benefit you as it normally runs at 4Gb or 8Gb anyway... Going down the iSCSI route. This would require SAN/NAS > Core switch over Ethernet. (Inexpesive compared to Fibre) Pretty much. iSCSI is SAN whereas CIFS/SMB is NAS, but it's all just standard network traffic. SAN/NAS > Core Switch > Other switches > Clients anywhere on site They are two main routes from my understanding. Both will have central management, and both can backup to tape drives. I need to determine whether SAN or NAS would be the best solution for our environment. I shall do more research. You don't want to choose between SAN and NAS, you want both - get unified storage. NetApp, EMC, Oracle, whoever - any decent vendor should offer you this, it's just case of making sure you have iSCSI and CIFS/SMB licences. From my understanding, if you want many TBs of storage, SAN is the best option to go down. Is this correct? SAN or NAS is independent of the amount of storage. Any good solution that meets what it sounds like your needs are will offer you expandable disk trays (think Petabytes) and NAS and SAN protocols. I'm in most of the day, give me a call if you want to chat. Chris
SYNACK Posted January 31, 2011 Posted January 31, 2011 Yes you will need HBAs for fibre channel but you'll also want additional NICs for your servers if using iSCSI to that you can dedicate to storage traffic. You don't want your file access being held up in the network queue behind someones funny cat video. SAS also would need HBAs its just an add in card per server, whichh ones you need depends on your chosen solution. I would second avoiding fibrechannel, it is very expencive in comparision to what you can get with iSCSI or SAS which given the cheaper interface tech can be a higher up more able model.
Soulfish Posted January 31, 2011 Posted January 31, 2011 So let me summarise; Going down the FC route. This would require: SAN > FC Switch > Server > Core Switch? (Seeminly expensive) Going down the iSCSI route. This would require SAN/NAS > Core switch over Ethernet. (Inexpesive compared to Fibre) They are two main routes from my understanding. Both will have central management, and both can backup to tape drives. I need to determine whether SAN or NAS would be the best solution for our environment. I shall do more research. From my understanding, if you want many TBs of storage, SAN is the best option to go down. Is this correct? As SYNACK and Duke have already said - don't go with FC . It's not the same sort of fibre network as ethernet based fibre - it runs its own set of protocols and will need you to learn all about FC switching and zoning. As Duke mentions you really want a unified device. Look at something like the Oracle S7000 series, any of the Netapps (make sure they give you licenses for everything up front though!), or the new range of EMC VNXe's/VNX's (depending on how big you need to be). You could also look at something like the HP P4000 series with a HP NAS gateway, or a Dell Equalogic with a Dell NAS gateway. The NAS gateway uses the underlying block level (SAN) storage and presents it as file level CIFS/NFS/whatever (NAS) storage. You may also want to look at Dells Compellent offerings. I know from when I looked at Compellent before they were bought by Dell they had a very interesting product (essentially block based, although you could get a NAS head based on their own customised Nexenta build), but they had a 100% support center satisfaction rating and certainly everyone I spoke to rated their support team very highly. How that'll change now they're a part of Dell is anyones guess 1
CHiLL Posted January 31, 2011 Author Posted January 31, 2011 Something my boss made clear is that we want something that is scalable. I've just seen on Dell's website that they sell just the chassis (Dell EqualLogic PS6000XV iSCSI SAN). This would allow us to simply buy the storage we need. However, I have noticed that it is just a SAN, not NAS or Unified. It also allows for 10GoE. The initial idea for this is that we want our servers to be servers, and our storage to be storage. We think when the kids log on and log off, at those times the servers are getting big hits and struggling to cope with the load.
teejay Posted January 31, 2011 Posted January 31, 2011 Something my boss made clear is that we want something that is scalable. I've just seen on Dell's website that they sell just the chassis (Dell EqualLogic PS6000XV iSCSI SAN). This would allow us to simply buy the storage we need. However, I have noticed that it is just a SAN, not NAS or Unified. It also allows for 10GoE. The initial idea for this is that we want our servers to be servers, and our storage to be storage. We think when the kids log on and log off, at those times the servers are getting big hits and struggling to cope with the load. So it's a waste of time then and won't do what you want as that will effectively just add extra storage space to your existing servers. In the Dell range, you'd need to be looking at the NX-4 or if you want unified storage the NS-120 or the NS-480.
DMcCoy Posted January 31, 2011 Posted January 31, 2011 Although Fibre channel is not all bad, it can be expensive, but I have found the failover to redundant channels much quicker with VMware compared to iSCSI. It also has better flow control and uses significantly less CPU than iSCSI (unless you have an also expensive iSCSI HBA). My new storage is iSCSI, although if cost wasn't an issue I'd still be using FC.
DMcCoy Posted January 31, 2011 Posted January 31, 2011 Something my boss made clear is that we want something that is scalable. I've just seen on Dell's website that they sell just the chassis (Dell EqualLogic PS6000XV iSCSI SAN). This would allow us to simply buy the storage we need. However, I have noticed that it is just a SAN, not NAS or Unified. It also allows for 10GoE. The initial idea for this is that we want our servers to be servers, and our storage to be storage. We think when the kids log on and log off, at those times the servers are getting big hits and struggling to cope with the load. I'm quite happy without having any clients talk to a storage box directly, the VMs just access it from the SAN and share the files as normal. There is one huge advantage to keeping the storage a mystery to end clients, migration. I am moving to a new SAN, I can live migrate the VM storage from one to the other without taking anything offline, and as the servers remain the same VMs, no client reconfiguration is needed. Unified/NAS features are not something I would even look at on a SAN.
Duke Posted January 31, 2011 Posted January 31, 2011 I'm quite happy without having any clients talk to a storage box directly [...] Unified/NAS features are not something I would even look at on a SAN. Sounded like CHiLL wanted home folders and profiles on the box too though - much nicer to have that direct to the NAS/SAN than the extra overhead of presenting a LUN to a Windows server (either physical or virtual) then offering that storage out to users via CIFS? 1
SYNACK Posted January 31, 2011 Posted January 31, 2011 Sounded like CHiLL wanted home folders and profiles on the box too though - much nicer to have that direct to the NAS/SAN than the extra overhead of presenting a LUN to a Windows server (either physical or virtual) then offering that storage out to users via CIFS? Not really then you just have auth overkill as the nas goes nuts hitting the servers for kerberos and you loose out on some of the features like windows file screening and shadow copies.
teejay Posted January 31, 2011 Posted January 31, 2011 Not really then you just have auth overkill as the nas goes nuts hitting the servers for kerberos and you loose out on some of the features like windows file screening and shadow copies. Unless your file server is also a domain controller you'll get the authentication traffic anyway and I would certainly never run my main file shares on a dc on any large network. Also, the latency on the ip stack on NAS boxes is generally much lower than windows file servers. Shadow copy is fully supported by nearly all decent NAS devices, file screening by some, dfs is supported. You also get extra features like de-dupe, access based enumeration, ssd caching. 1
SYNACK Posted January 31, 2011 Posted January 31, 2011 Unless your file server is also a domain controller you'll get the authentication traffic anyway and I would certainly never run my main file shares on a dc on any large network. Also, the latency on the ip stack on NAS boxes is generally much lower than windows file servers. Shadow copy is fully supported by nearly all decent NAS devices, file screening by some, dfs is supported. You also get extra features like de-dupe, access based enumeration, ssd caching. You learn something every day, NASs have definitly improved then from the dirty hacks that some of them were. 1
teejay Posted January 31, 2011 Posted January 31, 2011 Oh there are still plenty of dirty hack NAS boxes available;) Our Oracle 7310 just sits there and works very, very nicely. Shadow copies - keep a whole months worth on there from the end of every lesson, plus 1 per day for a year and never fails. No more 'oh the servers randomly decided to delete all the shadow copies' which we had on server 2003. No more disk space issues or doing complicated stuff with DFS to manage shares across 7+ file servers. No more windows update fun on it, just one update about every 3-6 months, which is generally to add shiny new features (features which some other NAS providers are charging a fortune in licensing for). I have to say, it's top of the list in pieces of kit that have reduced my stress levels and workload, including virtualising most of our servers.
Duke Posted January 31, 2011 Posted January 31, 2011 Our Oracle 7310 just sits there and works very, very nicely. [...] (features which some other NAS providers are charging a fortune in licensing for). Our 7410 hasn't been faultless (nothing is to be fair), but it's still one of the best bits of kit I've ever seen, particularly in this price range and with the feature set. CHiLL, watch out for the bit in bold above if you do look elsewhere, there are few vendors who will genuinely give you all the licensing at no additional cost like Oracle do with the S7000. £10k for NFS isn't a joke.
teejay Posted January 31, 2011 Posted January 31, 2011 Yeah, not 'entirely' faultless, but in comparison to our old Windows file servers it is in a different ball park.
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now