gshaw Posted January 9, 2009 Posted January 9, 2009 Just powered up and configured the NetApp StoreVault S500 iSCSI box we've had for a while but due to circumstances at the time haven't had time to get going until now. Took a few goes as the software isn't the best but now it's up and running on the network, static IP assigned to one of the onboard NICs and connected a LUN to a new server I've set up for SMS, WSUS and all the other management tools on the network. Question is whether I should set up another switch with separate IP addressing for the iSCSI traffic and use the 2nd NIC of the StoreVault for this. The first one would then be for management and I guess traffic for any shares I set up (it can do NAS and SAN). I've got a spare HP 2810-24 switch we originally bought for use with this box so wouldn't need any extra stuff ordered, it's just whether it's a recommended setup or not?
Butuz Posted January 9, 2009 Posted January 9, 2009 Yep your exactly right - use a dedicated switch for SAN traffic. Butuz
Zimmer Posted January 9, 2009 Posted January 9, 2009 I agree, that is how Storage Area Networks should be. Although I know a number of colleges that have their iSCSI disk array targets on their normal LAN infrastructure and just segregate the traffic using a VLAN. They have never reported any performance issues but they are using Cisco kit so performance is on their side I guess. We have two Dell/EMC AX4-5 disk arrays (iSCSI) that have dedicated switches. We use two 3Com 5500G's, Cat6 and Intel Dual PT NIC's to the servers that have LUN access. Performance is fantastic even though we are using budget hardware. All our staff, student and shared storage spaces are now going through just two servers that are SAN members. We are looking into moving our exchange mail stores over to the SAN during the next holiday break.
Dos_Box Posted January 9, 2009 Posted January 9, 2009 I have my SAN on a seperate managed Netgear GB switch, and it works flawlessly, and TBH with the amount of traffic it generates I would like to keep it off my existing infrastructure. VLAN or not.
plexer Posted January 9, 2009 Posted January 9, 2009 Just to clarify on this to see if it's what I need to do next financial year. Extra gigabit nic in each server that doesn't have a spare one. Second gigabit nics connected to a gigabit switch in server room comms cab. seperate gigabit switch in server room comms cab connected via fibre to san in remote location. That make sense? At the moment I have the san plugged into a media converter on the other end but it's still on the exsiting network as a device. Allthough I have some shares which would still need to be directly acessible from all over so I could use the second nic on the san to provide them on that ip. Ben
Zimmer Posted January 9, 2009 Posted January 9, 2009 Yeah just ensure all your servers that you want to have access to LUN's have a secondary network card that are on their own physical network separate from the current LAN. I recommend NIC's that support TOE (TCP Offloading) and absolutely have to be 1Gb/s. We use dual NIC's that is teamed to the SAN switch using IEEE 802.3AD link aggregation, but make sure your switches support that feature and LACP before giving it a try. We only do this to get the 2Gb/s throughput we need, otherwise I would have left it at single card solutions. Not to sure what you mean about accessing shares directly from the SAN? I have my SAN on a separate managed Netgear GB switch, and it works flawlessly, and TBH with the amount of traffic it generates I would like to keep it off my existing infrastructure. VLAN or not. Very true, our SAN switches are fairly busy during peak logon hours, I've seen up to 78% utilisation before now! Another good tip is to enable 'Jumbo Frames' on your SAN switches, we turned it on and got a noticeable improvement in performance
plexer Posted January 9, 2009 Posted January 9, 2009 What I mean is that as it's an openfiler server I also have smb shares that I want to still be able to access from any pc connected to the normal network. I can specify in openfiler what network addresses are allowed to access each resource so as long as the san/nas has an ip address on the normal range as well on it's second card I should be able to retain access to these other resources. Ben
Zimmer Posted January 9, 2009 Posted January 9, 2009 (edited) Ah I see, sorry I've never played with NetApp hardware so I'm not all that familiar with their features. If you have SMB shares on the NetApp disk array then yes, I'd be tempted to keep it hooked up to both the SAN and the LAN for the time being. Until you get round to moving the data in the openfiler SMB shares over to new LUN's that will be accessed via shares on the iSCSI host servers. If you go for setting up a dedicated SAN infrastructure then make sure you use a totally new IP scope. We use a 10.x.x.x on our core LAN, VLAN's in buildings around the campus are on 192.168.x.x and the SAN uses 172.16.x.x. So our iSCSI servers have a LAN IP on the 10.x.x.x network and an IP on the 172.16.x.x network. We only give the LAN NIC a gateway and DNS settings though, the SAN NIC just has an IP. We use ISNS within the SAN for iSCSI target locating, kind of like DNS for SAN's I guess. Edited January 9, 2009 by Zimmer
DrPerceptron Posted January 9, 2009 Posted January 9, 2009 Although I know a number of colleges that have their iSCSI disk array targets on their normal LAN infrastructure and just segregate the traffic using a VLAN. They have never reported any performance issues but they are using Cisco kit so performance is on their side I guess. We've got HP Kit here across the board are VLANing off our iSCSI traffic and it isn't making any impressionable negative impact on our network performance. Although, we aren't using iSCSI for our data storage, only VM's and mailboxes as well as a CCTV datastore, looking at putting WDS store on iSCSI too.
Zimmer Posted January 9, 2009 Posted January 9, 2009 Interesting you should say that mate, I have thought about moving our WDS images to SAN storage but I was worried about performance when we do mass re-installs over the summer. Summer just gone I think we got to about 60-70 workstations pulling down Vista images at the same time before the server started to refuse connections and ground to a shocking holt! I don't think it was network related as the server has a aggregated dual NIC to the server switch. If you do it please drop me a line if all goes well, I'd be interested in knowing the results
gshaw Posted January 9, 2009 Author Posted January 9, 2009 (edited) The S500 we have only allows 1GB throughput, guess the 2 NICs are as I thought, to allow SAN connection and another NIC for the NAS operations. Only thing I don't like about the StoreVault it seems that you can turn it off far too easily from the front and it doesn't exactly looks like it's gonna argue Was gonna put PC images, SQL data store for the management server (SMS database and also WSS3 database) plus a disk backup from our NAS file store Edited January 9, 2009 by gshaw
plexer Posted January 14, 2009 Posted January 14, 2009 Ok I'm looking at hp switches for this and it seems a toss up between: 2810-24G ProCurve 2810 24G Switch (J9021A) specifications - HP Small & Medium Business products or 1800-24G ProCurve 1800 24G Switch (J9028B) specifications - HP Small & Medium Business products 2810-24G is fully managed and a quick search reveals it for around £600 The 1800-24G is only web managed but is only £200 Each server will have a dedicated gig nic connected to this switch and then the switch itself will be connected via fibre straight to one gig nic in the san the other gig nic in the san will be connected to the normal network. Thoughts on the switches welcome. Ben
tmcd35 Posted January 14, 2009 Posted January 14, 2009 To be honest, I can't think of a good reason why you'd need a managed switch. We're talking about a single switch, separate from the rest of your network, that has perhaps 6 or 7 servers and an hard drive array plugged into it. I'd go for the the cheaper 1800-24G personally. So long as all the ports are 1GBps you'll be fine. Thou, having typed the above and can't be bothered to delete it, you may need the managed switch if you want to use nic bonding and have 2Gbps+ connections to the SAN. Assuming the 2810 supports nic bonding/trunnking.
Butuz Posted January 15, 2009 Posted January 15, 2009 I don't think you need to spend big money on your SAN switch. As long as its gigabit, and managed so that multiple ports can be trunked if need be - then it should be fine. Butuz
spc-rocket Posted January 15, 2009 Posted January 15, 2009 I don't think you need to spend big money on your SAN switch. As long as its gigabit, and managed so that multiple ports can be trunked if need be - then it should be fine. Butuz I would advise for a managed switch especially for as others have said for port trunking and bonding but also for jumbo frame support as on SANs these really do improve performance greatly. Ash.
SYNACK Posted January 15, 2009 Posted January 15, 2009 A managed switch may also have a faster and more robust switching fabric and so be able to handle the flow of frames more robustly and with less delays. You are also far more likely to be able to configure the error checking level on a managed switch setting it to cut through instead of store and foward based frame handling would make a large difference in responcivenes from a networking standpoint. If you get a web managed one you are usually stuck with whatever error handling is set in the firmware. The actual speed of frames through any trunks is also a product of how fast the internals are on the selected switch as the data from the two circuts must be put back together before being dumped into the switching fabric of the device to be sent out.
DMcCoy Posted January 15, 2009 Posted January 15, 2009 I would advise for a managed switch especially for as others have said for port trunking and bonding but also for jumbo frame support as on SANs these really do improve performance greatly. Ash. Also depends on the SAN. My HP 2012i doesn't support trunking, while ESX doesn't support jumbo frames with the software iscsi initiator.
gshaw Posted January 15, 2009 Author Posted January 15, 2009 If the SAN only has a single Gigabit connection then do you gain anything having multiple Gigabit connections from the server?
PRicho Posted February 9, 2009 Posted February 9, 2009 If the SAN only has a single Gigabit connection then do you gain anything having multiple Gigabit connections from the server? Id also like to know this Also where's the likely bottleneck on a San system going do be? Drive transfer speed?
Theblacksheep Posted February 9, 2009 Posted February 9, 2009 Id also like to know this Also where's the likely bottleneck on a San system going do be? Drive transfer speed? Yes failover if a NIC or switch dies, important even if you only have one link into the SAN. There are plenty o bottlenecks: drive speed and access time is often overlooked for more iscsi links but the majority of the time they doesnt carry a massive amount of traffic.
SYNACK Posted February 9, 2009 Posted February 9, 2009 If the SAN only has a single Gigabit connection then do you gain anything having multiple Gigabit connections from the server? Id also like to know this Also where's the likely bottleneck on a San system going do be? Drive transfer speed? Depends on the network cards and network stack on each device. If the nic on the SAN has a longer recive buffer and a faster network stack then yes it can give you an edge by having two of the slower smaller outbound queues to feed commands and queue up the data returned. The other thing that it gets you is redundancy incase a cable fails or nic port on your server. The bottleneck will depend on your setup but a propper SAN fully stacked with SAS disks can in general overwhelm the actual usable bandwidth of a gigabit nic. Slower drives/controllers/backplane and your bottleneck could be elsewhere.
sbigelow Posted October 6, 2009 Posted October 6, 2009 I have 2 HP storageworks aio400 storage arrays with 1 - 3 msa 50 disk arrays and 1 HP storageworks aio1200r storage array with 3 msa 60 disk arrays. All are running Windows Storage Server 2003 R2, how would i break out iscsi traffic to an HP Bladesystem c7000 that uses integrated NICs that connect to the virtual connect switch modules (1/10gb)?
mattx Posted October 6, 2009 Posted October 6, 2009 The bottleneck will depend on your setup but a propper SAN fully stacked with SAS disks can in general overwhelm the actual usable bandwidth of a gigabit nic. Slower drives/controllers/backplane and your bottleneck could be elsewhere. This is a concern of mine at the moment as we are thinking of moving everything over to a SAN. And when I say everything, I mean everything, streaming media, files, images - you name it..... Now when I first arrived at my place we had a gig backbone [ HP switch ] and 10/100 [ again all HP ] switches. Over the year I have slowly up-graded the 10/100 to managed gig switches so we are gig everywhere. Now I am thinking would I need to up-date the main gig backbone to 10gig if I go down to the SAN route ? We would of course have a different scope of address to talk to the SAN -[ looking at HP Options with Left Hand or a Dell / IBM SAN with SANmelody ] So I am left scratching my head as I also want to get up and running ESX or ESXi to get rid of around 4 servers we currently have. Just out of interest, has anyone installed the VM Ware's Virtual Centre on a virtual server ? Or is this a bad idea ??!! [ just looking to save a few quid ]
garrya100 Posted October 8, 2009 Posted October 8, 2009 (edited) I have Vsphere running on a virtual server, it runs fine. The only thing you need to be aware of is that you need to be connected to a SAN to make full use of this technique, otherwise you can never apply updates to the host that has your VS on it. This is because in order to update a host you need to be in maintence mode, which means your VM are shutdown. When the VS shutsdown the updates stop. It also means you need at least two ESX hosts. You need to Vmotion the VS of a host to enable it to update it. That said the benefits are of course you can incorporate your VS into a HA or DRS strategy so it's always available and also save buying another box Edited October 8, 2009 by garrya100 1
cr32003 Posted April 13, 2012 Posted April 13, 2012 I know this may be presumptious - but we have asimilar setup and the iscsi performance is not that good - could you please advise re the configuration of the 5500 ports for iscsi - i.e flow control etc.....
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now