Storage?

[localzuk]

We currently have a Dell MD1000 DAS array with 15 x 1TB SAS drives in it, connected to a Perc 5/E controller in our file server, and it is now 3 years old, and therefore out of warranty.

 

We have 2 options - extend the warranty for 2 years, at a cost of about £3k or come up with something else which is more cost effective.

 

My thinking here would be to start my way down the SAN route.

 

Problem is, our core switch is very short on capacity and we won't be upgrading that for some time, so I was thinking of a Fibre Channel array, direct connected to an FC controller in the storage server for now, then in the future expanding that to more devices via an FC switch.

 

If I were to suggest this route, what devices should I be looking at? Or do others have a more suitable suggestion?

[pcstru]

What work will the SAN doing? How many LUNS to how many devices and what are they doing? Given the probable cost - if the MD1000 is doing the job, then £3K for another two years sounds pretty reasonable.

[localzuk]

To start, it will be a drop in replacement for our file server and that's it. However, in the future, I plan to use it for virtualisation of our servers (we currently have 12 servers, but 4 of them are DCs and are very poorly utilised).

 

A new MD1000 is about £7k, so paying nearly half the price of a device for 2 more years for warranty seems steep to me. Or an Overland Storage SAN with 24TB raw space, 3 year warranty and FC card would come to about £8k.

Edited October 27, 2011 by localzuk

[CyberNerd]

I'd be inclined to extend the warranty unless you need to use it for something else, eg backups. Replacing a storage array after just 3 yrs isn't cost effective.

 

for alternatives, we have a DS34000 with 4GBs FC which is quite nice, and expandable

 

IBM System Storage DS3400: Overview - IBM System Storage

 

you'd still need a FC switch which isn't so cheap, so it might be worth considering upgrading the core ethernet switch and moving it out to the near edge.

[localzuk]

From what I've read and been told, an FC switch is not needed for a single device - you can direct connect 2 FC cards together, output to input on each.

 

Even for more than one device, you can use do things like 'Arbitrated Loop' etc... but that is complex and not the best way of doing things.

[SYNACK]

If you are just after a SAN that offers storage to your servers then look at SAS connectivity, depending on how many servers need a direct conection. Its very quick and way cheaper than FC. You then offer out your data from the servers that it is conected to. If you want all stations to conect to your storage solution directly over the network then you will need a SAN/NAS hybrid which is a different beast, more expencive and more prone to issues given their increased complexity.

[pcstru]

8K for 3 years, is £2.6K per year, compared to £1.5K to extend the warranty. Seems reasonable to me if the device is doing the job you want it to. The market for SAN/NAS devices seems to be moving along at a fairly rapid pace at the moment. I'd hope that in a couple of years we will be seeing many more (cheap) iSCSI devices with High Availability built in.

[CyberNerd]

The market for SAN/NAS devices seems to be moving along at a fairly rapid pace at the moment. I'd hope that in a couple of years we will be seeing many more (cheap) iSCSI devices with High Availability built in.

 

yes. I'm thinking along the lines of Fibre Channel over Ethernet - Wikipedia, the free encyclopedia using 10GB/s core switches will be the way forwards, utilising network infrastructure rather than separate FC switches or the overhead of iscsi

[pcstru]

yes. I'm thinking along the lines of Fibre Channel over Ethernet - Wikipedia, the free encyclopedia using 10GB/s core switches will be the way forwards, utilising network infrastructure rather than separate FC switches or the overhead of iscsi

 

Is FCoE better than iSCSI when it comes to overhead? Speaking personally we have a single windows 2008 VM acting as a file server which is attached through iSCSI to a lump of storage on a Thecus 8800 Pro using SATA II discs with two 1GB nics aggregated using 802.3ad. That serves as the primary network areas for 2500 students and 400 staff - with peak demand of maybe 1000 concurrent devices accessing files. I see people here specifying up 15K SAS drives on dedicated fibre channel blah-de-blah and spending much more money to do the same job for smaller schools. Maybe I'm doing it wrong?

[CyberNerd]

Is FCoE better than iSCSI when it comes to overhead?

 

I assumed it was as it isn't running over TCP/IP but it probably has its own overhead.

FCoE would make sense for us as were already using FC so integration is possible- were mostly using FC for virtual machines but also for storage. FC SAN was within our budget so it made sense to buy the best we could afford.

[localzuk]

Is FCoE better than iSCSI when it comes to overhead? Speaking personally we have a single windows 2008 VM acting as a file server which is attached through iSCSI to a lump of storage on a Thecus 8800 Pro using SATA II discs with two 1GB nics aggregated using 802.3ad. That serves as the primary network areas for 2500 students and 400 staff - with peak demand of maybe 1000 concurrent devices accessing files. I see people here specifying up 15K SAS drives on dedicated fibre channel blah-de-blah and spending much more money to do the same job for smaller schools. Maybe I'm doing it wrong?

 

Please can you download and run HDTune (not Pro) on your VM and tell me what the Min/Max/Average Transfer Rates are, and the Access Time/Burst Rate?

 

So I have something to compare with.

[pcstru]

@localzuk - I'm not very keen on installing any unnecessary software on our file server - sorry. There is a utility called HD_Speed which is a standalone .exe and is what I tend to use. Between a replication server and an 8800Pro I get transfers peaking at 85MBs (Bytes per second) averaging 70MBs over a couple of minutes. The storage network itself is not uncontended and is currently copying off some VM's to some smaller i4500's (I've had these show 120MBS on a good day).

[CyberNerd]

hdparm shows (when everything is in use)

 

/dev/mapper/Web_VLE-Web_VLE_LogVol00:

Timing cached reads: 9358 MB in 2.00 seconds = 4685.95 MB/sec

Timing buffered disk reads: 752 MB in 3.00 seconds = 250.29 MB/sec

[localzuk]

@localzuk - I'm not very keen on installing any unnecessary software on our file server - sorry. There is a utility called HD_Speed which is a standalone .exe and is what I tend to use. Between a replication server and an 8800Pro I get transfers peaking at 85MBs (Bytes per second) averaging 70MBs over a couple of minutes. The storage network itself is not uncontended and is currently copying off some VM's to some smaller i4500's (I've had these show 120MBS on a good day).

 

That's fine, same results. Those speeds are really quite low. We get 122MB/s average (200MB/s peak) on our MD1000, and I wouldn't want to go any slower, as this directly affects log in times, video editing storage, Photoshop file editing etc...

[pcstru]

That's fine, same results. Those speeds are really quite low. We get 122MB/s average (200MB/s peak) on our MD1000, and I wouldn't want to go any slower, as this directly affects log in times, video editing storage, Photoshop file editing etc...

 

Sure that may affect the times - but if logon is 2 seconds slower or a large avi takes 8 seconds to copy rather than 12, then is the cost justified? It also may not affect end user times at all at some points if they are on 100Mbs links and there is little contention for the storage. My perception is that some schools are spending tens of thousands on storage that sure is fast, but is then underutilised and poor value. Contention for funds generally more demanding than contention for 100MBs resources. "We bought the best we could afford" is poor value if the best that you could afford is much more than you ever needed.

[localzuk]

I'd never spend 10's of thousands on storage, that would be a waste - there's no need.

 

There are plenty or perfectly suitable SAN solutions for schools of all sizes, without ending up in the price range you're talking of.

[CyberNerd]

Sure that may affect the times - but if logon is 2 seconds slower or a large avi takes 8 seconds to copy rather than 12, then is the cost justified? It also may not affect end user times at all at some points if they are on 100Mbs links and there is little contention for the storage. My perception is that some schools are spending tens of thousands on storage that sure is fast, but is then underutilised and poor value. Contention for funds generally more demanding than contention for 100MBs resources. "We bought the best we could afford" is poor value if the best that you could afford is much more than you ever needed.

 

You're not considering expandability and longevity; in the context of the original post the OP is already considering replacing the existing array after 3yrs. I don't think the money we spent on FC was a waste if I can expand nearly double the amount of virtual machines than I could on iscsi. disk access time is a real bottleneck for both operating systems and remote filesystems. Clearly we're saving money on other aspects of our network (thin clients, cloud applications etc) but it makes sense to me to put the resources in where the bottlenecks are likely to be - and I think disk access time is important, especially where there is a lot of virtualisation.

[localzuk]

For me, the system we have is entirely 3 years old, as it was a managed BSF contract until a few months ago. So, what we have wasn't implemented according to the school's design from what I can tell - hence us having 4 dedicated DCs.

 

So, what I'm looking at is taking the large rack full of stuff and getting rid of stuff that is not needed, and moving to a system that is going to last and be sustainable.

[pcstru]

I'd never spend 10's of thousands on storage, that would be a waste - there's no need.

 

There are plenty or perfectly suitable SAN solutions for schools of all sizes, without ending up in the price range you're talking of.

 

What has ending up in a particular 'price range' got to do with it?! The equipment is either up to the job or it is not. If equipment is up to the job, how could it possibly be too cheap to consider?

[localzuk]

What has ending up in a particular 'price range' got to do with it?! The equipment is either up to the job or it is not. If equipment is up to the job, how could it possibly be too cheap to consider?

 

You asked if spending 10's of thousands to improve file opening/login times by 33% was cost effective. I say no. However, I also say that there is no need to spend that much money to do the task, when it can be achieved in a very reasonable budget. Context is important here.

[pcstru]

You're not considering expandability and longevity; in the context of the original post the OP is already considering replacing the existing array after 3yrs. I don't think the money we spent on FC was a waste if I can expand nearly double the amount of virtual machines than I could on iscsi. disk access time is a real bottleneck for both operating systems and remote filesystems. Clearly we're saving money on other aspects of our network (thin clients, cloud applications etc) but it makes sense to me to put the resources in where the bottlenecks are likely to be - and I think disk access time is important, especially where there is a lot of virtualisation.

 

All I have said is that my perception is that a lot of schools are spending a lot of money, sometimes on very expensive SAN equipment, and that in many cases I believe that to be overkill for what they need. It's all very well running benchmarks on kit to see what it CAN do, but the real question is what IS (or WILL) it be asked to do and what performance do you then need to meet that need. As for longevity - the example here is that kit purchased three years ago is looking to be replaced simply because the continued maintenance is approaching the cost of replacement. There is so little longevity in IT kit that buying now to meet needs you might have in even two years time, is a mugs game.

[localzuk]

All I have said is that my perception is that a lot of schools are spending a lot of money, sometimes on very expensive SAN equipment, and that in many cases I believe that to be overkill for what they need. It's all very well running benchmarks on kit to see what it CAN do, but the real question is what IS (or WILL) it be asked to do and what performance do you then need to meet that need. As for longevity - the example here is that kit purchased three years ago is looking to be replaced simply because the continued maintenance is approaching the cost of replacement. There is so little longevity in IT kit that buying now to meet needs you might have in even two years time, is a mugs game.

 

My issue is down to a lack of long term planning by the managed provider and the BSF LEP. I'd've been looking at a 5 year warranty off the bat with any system originally, and would've been putting a virtualised system in 3 years ago anyway! Both those things would've provided a much more sustainable solution than was implemented, not to mention a cheaper one!

[CyberNerd]

There is so little longevity in IT kit that buying now to meet needs you might have in even two years time, is a mugs game.

 

?? when I buy server/network hardware I think with capacity planning in mind, its basic ITIL. We bought a DS3400 3yrs ago with a projected lifetime of 5yrs. In that time we've loaded it with all manor of servers including citrix terminal servers which are processor heavy (bear in mind iscsi also has a processor overhead ; which would have slowed down our server processing power, and available network bandwidth - meaning we could have needed another physical server, and dedicated iscsi switch to compensate). knowing that our SAN will easily meet our capacity demands for another 2 yrs and possibly further (I pay an annual maintenance fee) means that I don't need to worry about replacing it any time soon (saving considerable time). Sure there is no point in buying something you don't need, but planning ahead isn't a mugs game.

[pcstru]

I didn't say that planning ahead is a mugs game. In some cases, and I think it's valid on storage, the life of the technology is relatively short and the market moves very quickly. That means that the very act of planning ahead might suggest that buying kit now that will still meet the needs in 4 years is going to be much more expensive than buying cheaper kit which will only serve for a couple of years but by the time you do replace it, you get better than twice the bang for your buck or (usually) more.

[Millgate]

We currently have a Dell MD1000 DAS array with 15 x 1TB SAS drives in it, connected to a Perc 5/E controller in our file server, and it is now 3 years old, and therefore out of warranty.

 

We have 2 options - extend the warranty for 2 years, at a cost of about £3k or come up with something else which is more cost effective.

 

My thinking here would be to start my way down the SAN route.

 

Problem is, our core switch is very short on capacity and we won't be upgrading that for some time, so I was thinking of a Fibre Channel array, direct connected to an FC controller in the storage server for now, then in the future expanding that to more devices via an FC switch.

 

If I were to suggest this route, what devices should I be looking at? Or do others have a more suitable suggestion?

 

Hi,

 

My suggestion here would be to use ISCSI but still directly connect it to the storage sever for the time being.

When you are ready you can replace your core switch and VLAN the ISCSI traffic off to give a dedicated Storage broadcast domain LAN

This will give you the same sort of dedicated fabric that you would get with FC (albeit not as fast)

You are unlikely to need FC below a user count of 500 seats

I’ve seen 2000 seat implementations of ISCSI work very well.

 

However, having said that you could buy an Array which supports both FC and ISCSI on the controllers

In a joint venture between Huawei (27Bn dollar Chinese telco provider) and Symantec, they make one (2600) which has twice the number of front end ports you would normally find on a small SAN (4 FC and 4 GbE per controller) it also has a SAS backplane which is faster than the standard 4Gbps FC loops you normally get.

Its supports snapshots / replication, mixed disk (SAS and SATA) and is VERY cost effective.

If you bought a similar array from EMC or Ntap it would cost you 50k

You can have 20TBs RAW with dual controllers and 3yrs support for sub 16k with the 2600

We use several of these in our private cloud to run our VMs and they’re superb

 

 

It’s an entry level Enterprise SAN array, which will scale to 96 drives and can support both file (SATA) and transactional (SAS) workloads concurrently.

 

The overland array like similar Netgear or Q is really just an expensive SOHO product,

Its not something you would ideally want to rely on as a core foundation to critical systems in a corporate office environment