RAID Controller SSD Caching

[Jollity]

Has anyone had experience with using an SSD card with a RAID controller to act as a large fast cache in front of an array of conventional hard drives? I have been having a look at technologies like LSI CacheCade (used by Dell) or SmartCache from HP. The reports look rather promising.

 

I was concerned that the SSD cache was a single point of failure, but I read at least for the read-only version of Cachecade (that Dell uses) failure does not cause data loss.

 

This is step 2 of my ongoing quest to look for a way to SSD up our storage speeds without SSDing up the price.

Edited March 5, 2014 by Jollity

[Jollity]

The lack of response to this is useful information in itself. I am taking away that this is not widely used.

[mattpayne]

The Cachecade is only available on the controller with 1GB NV RAM

 

Ive not used it yet, but I should have a server arriving in the next few weeks to test this out myself... I'm looking at about a TB of SAS storage with a pair of mirrored 256GB decent consumer grade SSDs - my view being life expectancy of server ~5 years, enterprise SSD's are bankers money and if I have to replace my cheap drives once a year id still be onto a winner cost wise.

 

I'm using my laptop as a benchmark - Samsung 840pro - 1 year old, ~10TB of transfers (40/60 split read/write) and its still at 95% health.

[Jollity]

Ive not used it yet, but I should have a server arriving in the next few weeks to test this out myself... I'm looking at about a TB of SAS storage with a pair of mirrored 256GB decent consumer grade SSDs - my view being life expectancy of server ~5 years, enterprise SSD's are bankers money and if I have to replace my cheap drives once a year id still be onto a winner cost wise.

I would be interested to hear what you find. I have read that the controllers look for Dell IDs on the drives and throw errors if they find 3rd party ones, but I am not sure to what extent they work regardless.

[VeryPC_Ed]

Has anyone had experience with using an SSD card with a RAID controller to act as a large fast cache in front of an array of conventional hard drives? I have been having a look at technologies like LSI CacheCade (used by Dell) or SmartCache from HP. The reports look rather promising.

 

I was concerned that the SSD cache was a single point of failure, but I read at least for the read-only version of Cachecade (that Dell uses) failure does not cause data loss.

 

This is step 2 of my ongoing quest to look for a way to SSD up our storage speeds without SSDing up the price.

 

Hi Jollity,

 

With CacheCade if I remember correctly if the SSD fails then the controller will suspend access to the array until it is resolved.

 

We have supplied LSI controllers for a long time and have supplied quite a few Cacheade setups, typically I would suggest that you use a quality Enterprise SSD which is designed for the job - something like a pair of Intel S3700 Series SSD and configure them in Raid 1 to prevent a hard stop of access to your SAN etc.

 

You can use a consumer SSD but once you have burned through it, there is a good chance you won't be able to RMA it as I know with the Intel drives they have an internal counter which can advise how much data has been written in what time.

 

Edit: Depending on how much the cache is being hammered will determine whether this is a short time or long time.

 

CacheCade also is a software license rather than a physical thing and is available on the 9260 range onwards, you just buy the license and apply it within the SAS controller.

 

let me know if you want to discuss the CacheCade / speeding up your storage more.

 

Thanks,

 

Ed

[zag]

Has anyone had experience with using an SSD card with a RAID controller to act as a large fast cache in front of an array of conventional hard drives? I have been having a look at technologies like LSI CacheCade (used by Dell) or SmartCache from HP. The reports look rather promising.

 

I was concerned that the SSD cache was a single point of failure, but I read at least for the read-only version of Cachecade (that Dell uses) failure does not cause data loss.

 

This is step 2 of my ongoing quest to look for a way to SSD up our storage speeds without SSDing up the price.

 

Hmm patch old technology with new.... Not always the best idea

 

If you insist on using RAID, why not just put 3 Intel 600GB SSD's together. The cost is not as much as you would imagine.

 

Personally I've skipped RAID completely these days and use the intel SSD's on their own. Their reliability is far greater than hard disks and they are so quick you can back them up far more regularly.

[Michael]

I agree with the above - SSD does beat SAS. The alternative is you can get hybrid drives which (for example) are SATA disks, but include 1GB of flash memory built in. Not sure if these are available in SAS form, but it would increase speeds further.

 

The only thing faster than an SSD RAID setup is a RAM drive, but you're limited in terms of capacity and if you lose power it's bye bye data! It's good for non-critical setups.

[VeryPC_Ed]

I agree with the above - SSD does beat SAS. The alternative is you can get hybrid drives which (for example) are SATA disks, but include 1GB of flash memory built in. Not sure if these are available in SAS form, but it would increase speeds further.

 

The only thing faster than an SSD RAID setup is a RAM drive, but you're limited in terms of capacity and if you lose power it's bye bye data! It's good for non-critical setups.

 

We have some of these on their way to us shortly for testing - Enterprise Turbo SSHD: SSD Performance, 600 GB Fast Hard Drive | Seagate

 

Essentially they are Enterprise versions of the Hybrid HDD, each unit has 32GB NAND RAM inside, going to be interesting to see how they perform.

 

Ed

[mattpayne]

Ive yet to see a hybrid with SAS connector and not sure if the limited flash on a DATA version will be enough to overcome the advantages of an SAS drive.

 

I was looking at SSD RAID, but I had just come up against lots of mentions about TRIM/Garbage collection issues with the drive hiding behind a real controller.

[mattpayne]

We have some of these on their way to us shortly for testing - Enterprise Turbo SSHD: SSD Performance, 600 GB Fast Hard Drive | Seagate

 

Essentially they are Enterprise versions of the Hybrid HDD, each unit has 32GB NAND RAM inside, going to be interesting to see how they perform.

 

Ed

 

I spoke too soon! Any idea on pricing/availability on the 600GB models?

[VeryPC_Ed]

I spoke too soon! Any idea on pricing/availability on the 600GB models?

 

I've not had the pricing yet as but will see if I can get an indication.

 

With the SSD Raid, I think a lot of drives have pretty good internal garbage collection now, we've supplied quite a few SSD arrays as of late to schools (including a few edugeekers)

 

The biggest concern I see with SSD Raid is the write penalty / parity writes when using Raid 5 being the issue, most of the installs we have done have been Raid 10 for VDI / Virtual Machines.

 

Enterprise SSD have come down in price quite a lot recently, Intel S3500/3700 Series, Seagate 600 Pro, Samsung 840 Pro etc are all well priced for the workloads.

 

Thanks,

 

Ed

[Michael]

Depending on the costs, I'm speculating you need this kind of speed for a Hyper-V setup? Sometimes (depending how you look at it) having several physical boxes can give the same/similar speeds for less money. High density isn't always the best method.

[zag]

Depending on the costs, I'm speculating you need this kind of speed for a Hyper-V setup? Sometimes (depending how you look at it) having several physical boxes can give the same/similar speeds for less money. High density isn't always the best method.

 

I've currently got a complete Intel 320 SSD local storage + Hyper-V setup and it absolutely flies!

 

No problems with storage space or reliability so far...

 

The difference from using RAID 5 mechanical hard disks is like night and day.

Edited March 10, 2014 by zag

[Jollity]

With CacheCade if I remember correctly if the SSD fails then the controller will suspend access to the array until it is resolved.

I know this is the case for the full CacheCade 2 read/write system, but do you know if it is the case for read-only CacheCade, as well? It is possible, but seems silly if it is.

 

Good to hear that some people are using CacheCade out there.

 

Hmm patch old technology with new.... Not always the best idea

I have read your blog post and I thought you might say that.

 

I see the benefits of your approach: it fits together and makes a fair amount of sense. But I do not have enough confidence in the approach or in my own judgement to step that far from the main stream. I am concerned about the durability of "consumer" SSDs in a server (though time may prove you right) and not having Dell/HP support due to using aftermarket disks. Though perhaps other server providers can offer more reasonably priced SSDs while still providing support.

 

 

We have some of these on their way to us shortly for testing - Enterprise Turbo SSHD: SSD Performance, 600 GB Fast Hard Drive | Seagate

I was looking at those as well, but I was concerned about how well they would behave in a RAID array. It seemed potentially problematic to have the disks all trying to cache independently of the RAID controller, which was why I started looking at the RAID controller caching. But I suppose ordinary disks do their own caching normally.

[seawolf]

SSD write caches have been a part of ZFS storage pools for many years, and yes I have used them.

 

ZFS uses three types of caches for reading and writing synchronous and asynchronous data. ARC (RAM) and L2ARC (large SSDs) for reads and Logzillas (very fast SSDs designed for massive amount of write operations).

 

Using Logzillas and L2ARC SSDs in Hybrid storage pools provides a combination of outstanding performance with large storage capacity using either SAS drives for maximum performance or large SATA drives for maximum storage space. Bothe L2ARC and Logzillas are always used in pairs to provide redundancy, further showing that SSDs are not "indestructible" and should not be trusted as superior to good RAID.

 

http://constantin.glez.de/blog/2010/04/ten-ways-easily-improve-oracle-solaris-zfs-filesystem-performance

 

Our 7320 using Hybrid storage pools with two 73GB ZeusIOPS SSD write caches and 96GB of ARC can provide a sustained 50,000 IOPS under load. Not bad for 20 x 10,000RPM SAS drives and a couple of Logzillas.

[Arthur]

going to be interesting to see how they perform.

Very well I imagine.

 

Seagate Enterprise Turbo SSHD Review « Storage Review

 

Seagate makes a bold claim with the Enterprise Turbo SSHD, "The World’s Fastest Hard Drive." Normally, storage companies like to use a bit of bravado as part of their marketing claims, but saying "World's Fastest" is pretty strong. Here's the thing though, they're right. The Turbo is absolutely smoking and the results trounced Seagate's own 10K SAS drives, which posted over 200% gains in some cases. For enterprises looking to refresh an array but don't have the budget or IOPS/latency need for an all flash solution, the Turbo is the superior choice.

[seawolf]

The biggest concern I see with SSD Raid is the write penalty / parity writes when using Raid 5 being the issue, most of the installs we have done have been Raid 10 for VDI / Virtual Machines.

 

I'm very strongly against using RAID 5 (as others on here can attest) except for secondary backup storage where a failure doesn't have much of an impact. RAID10 is best where maximum performance and reliability are needed (e.g. VDI / Core Infrastructure). If more cost-effective storage is required over RAID10, then Use RAIDZ3 or RAIDZ2 or RAID6, in that order of preference. ZFS takes care of the write penalty for parity writes BTW.

[kidpressingbuttons]

Has anyone actually done this yet?

I have set cachecade up on my new dell servers- but would like to know what impact it has had on others in terms of speed and reliability!

[zag]

Has anyone actually done this yet?

I have set cachecade up on my new dell servers- but would like to know what impact it has had on others in terms of speed and reliability!

 

As I said originally (and its even more so today), the cost of large SSD's has tumbled so I'd advise moving completely solid state based storage, thats where the technology is going.

 

These half way solutions are going to be redundant soon.