New Server, Sanity Check on Spec

[dhicks]

I think the issue is that this benchmark isn't testing what Storage Spaces actually does. I'll try a script that copies some files around, see if I can better make it mirror an actual workload.

 

I wrote the following in Python:

 

import os
import sys
import time
import shutil
import random

fileData = {}
fileData["100MiB"] = [104857600]
fileData["1GiB"] = [1073741824]
fileData["4GiB"] = [4294967296]

testSize = "100MiB"
if len(sys.argv) > 1:
   testSize = sys.argv[1]

testLoops = 5
if len(sys.argv) > 2:
   testLoops = int(sys.argv[2])

randomString = ""
for pk in range(0, 1024):
   randomString = randomString + chr(random.randint(0, 255))

def createFile(fileName, fileSize):
   outfile = open(fileName, "wb")
   for pl in range(0, fileSize, 1024):
       outfile.write(randomString)
       outfile.flush()
   outfile.close()

def clearFiles():
   for filename in os.listdir("C:\\Benchmark\\"):
       os.remove("C:\\Benchmark\\" + filename)
   for filename in os.listdir("E:\\Benchmark\\"):
       os.remove("E:\\Benchmark\\" + filename)

clearFiles()
createFile("C:\\Benchmark\\" + testSize + ".txt", fileData[testSize][0])

resultArray = []
for pl in range(0, testLoops):
   startTime = time.time()
   shutil.copyfile("C:\\Benchmark\\" + testSize + ".txt", "E:\\Benchmark\\" + testSize + ".txt")
   operationTime = time.time() - startTime
   resultArray.append((fileData[testSize][0] / operationTime) / 1048576)

result = 0
for pl in range(0, len(resultArray)):
   result = result + resultArray[pl]
print "Write: " + str(result / len(resultArray)) + " MB/s"

resultArray = []
for pl in range(0, testLoops):
   startTime = time.time()
   infile = open("E:\\Benchmark\\" + testSize + ".txt", "rb")
   randomData = infile.readlines()
   infile.close()
   operationTime = time.time() - startTime
   resultArray.append((fileData[testSize][0] / operationTime) / 1048576)

result = 0
for pl in range(0, len(resultArray)):
   result = result + resultArray[pl]
print "Read: " + str(result / len(resultArray)) + " MB/s"

clearFiles()

 

The idea was to create a file of the appropriate size that I then copy around multiple times to see what kind of performance we get. I get the results below:

 

> benchmark.py 100MiB 5

Write: 471.016959734 MB/s

Read: 214.194644203 MB/s

 

> benchmark.py 1GiB 5

Write: 383.208574217 MB/s

Read: 174.081253186 MB/s

 

> benchmark.py 4GiB 5

Write: 422.7091218 MB/s

Read: 199.71821858 MB/s

[dhicks]

So, after the test files have been created as above, if I comment out the part in the Python scripts that deletes those files so I can pin them to the SSD tier...

 

> $StorageTier = Get-StorageTier -FriendlyName "KS-TIERED01_Microsoft_SSD_Template"

> Set-FileStorageTier -DesiredStorageTier $StorageTier -FilePath "E:\Benchmark\100MiB.txt"

> Set-FileStorageTier -DesiredStorageTier $StorageTier -FilePath "E:\Benchmark\1GiB.txt"

> Set-FileStorageTier -DesiredStorageTier $StorageTier -FilePath "E:\Benchmark\4GiB.txt"

 

> python.exe .\benchmark.py 100MiB 5

Write: 383.186047684 MB/s

Read: 239.349172213 MB/s

 

> python.exe .\benchmark.py 1GiB 5

Write: 401.851234381 MB/s

Read: 181.86126146 MB/s

 

> python.exe .\benchmark.py 4GiB 5

Write: 412.107014987 MB/s

Read: 192.065022822 MB/s

 

Around about the same - hopefully my benchmark is testing the correct thing, i.e. file read / write performance on tiered storage on repeat copies of the same data where the storage tiering system can see it's the same data being copied. Pinning the test files to the SSD tier doesn't seem to make any real difference, so hopefully it's the SSD tier that's being tested in the first place.

[Arthur]

Pinning the test files to the SSD tier doesn't seem to make any real difference, so hopefully it's the SSD tier that's being tested in the first place.

Perhaps it might be worth giving Microsoft's DiskSpd a try?

 

https://technet.microsoft.com/en-GB/library/dn894707.aspx#BKMK_TestOverview

 

When you test the performance of a newly created tiered storage space, your goal should be to baseline ideal storage tiers performance by testing the performance of the SSD tier only. For optimal performance, the SSD tier should be large enough to accommodate the entire working set (all active data) of workloads that use the space. That ensures that your faster SSDs will process all or most I/O, with minimal I/O performed on the slower but less expensive hard disk drives (HDDs), which are intended for storing "cold," inactive data.

 

Storage Tiers Optimization, which is performed at 1:00 a.m. nightly by default, attempts to maintain that balance by moving the active (“hot”) data to the SSD tier and moving inactive ("cool") data off the SSD tier to less expensive hard disk drives based on I/O activity since the last tiers optimization. If you’re not familiar with this process, see Monitor Storage Tiers Performance in Windows Server 2012 R2 for an overview. You can also pin critical files to the SSD tier to ensure that all processing is performed on the SSDs.

 

In the real world, you’re not likely to maintain the entire working set on your SSD tier, but you can get close. By finding out how fast the storage space can perform when all I/O is processed by the SSD tier, you can get a baseline against which to gauge performance as you migrate workloads to the space. During a phased migration, the Storage Tier Optimization Report can provide diagnostics for determining the SSD tier capacity and Storage Tiers Optimization frequency needed to meet performance requirements of the workloads.

 

Note. There's really no way to accurately predict performance of both the SSD tier and the HDD tier working together by using a synthetic workload. For that, you will need to use the Storage Tier Optimization Report and monitor performance counters for Storage Spaces, including those for the Storage Tiers object and the Storage Write Cache object, to characterize daily activity of the deployed workloads as they become stable and predictable.

 

The test design

 

• The DiskSpd.exe command-line tool is used to test the performance of a synthetic workload in a new storage space. DiskSpd provides flexible options for emulating performance behavior of synthetic random or sequential workloads. It can be used to test the performance of physical disks, partitions, or files in a storage subsystem.
   
   
  
• To provide enough data to fully exercise the underlying SSDs, DiskSpd will create a 64 GB data file, the size of an average virtual machine. To ensure that file-level contention does not impose unnecessary performance limits, we include a test that varies the number of data files (Pre-Test 2), with guidance for adapting the number of data files, file size, and threads per file to take full advantage of your system’s capabilities.
   
   
  
• To exercise all layers of the storage stack during testing, the tests are run on a virtual machine deployed to the storage space.
   
   
  
• We test the performance of the SSD tier only, to find out the high end of potential performance of a tiered storage space. To ensure that all I/O from the tests is processed on the SSD tier, the VHDX file for the test machine is pinned to the SSD tier, and Storage Tiers Optimization is run to move the entire file to the SSD drives.
   
   
  
• To get accurate performance data for the SSD drives, each run of the DiskSpd command includes a 5-minute warmup time followed by 10 minutes of data collection. If you shorten the warmup time, you might be observing SSD drive initialization, before the drives reach a steady state. And, of course, the longer the sampling period, the greater the reliability of your results will be and the less variability if you repeat a test.
  

[dhicks]

Perhaps it might be worth giving Microsoft's DiskSpd a try?

 

Good point - and I will, but now I've read the link you posted above, I realise that just pinning a file doesn't actually place it right away on the SSD tier, you have to run the optomisation task first for Windows to spot the pinned content and shuffle it around. I've done that with the VHDX files for our test machine and re-run CrystalDiskMark. Results are:

 

 

The first result is before I ran the "pin" command on the VHDX file. The second one is immediatly after, and the third one is after I'd run the optomise schedualed task. Just based on those results, there seems to be pretty much no difference in performance between pinned and unpinned data. I'll try DiskSPD.exe instead.

[craig-schools]

Just a follow up to the original topic. I've bought a server from VeryPC with the help of VeryPC_Ed with the following spec.

 

Dual Xeon E5-2620v3 Six Core Processor

64GB DDR4 2133 ECC REG RAM (8 x 8GB DIMM Modules)

DVDRW Optical Drive

Intel 8 Port RS3UC080 SATA/SAS HBA

Intel S3500 120GB Series Enterprise SSD (2 - Raid 1)

Intel S3610 Enterprise 400GB SFF SATA SSD (4)

Seagate 900GB Enterprise SFF 10k SAS HDD (4)

Redundant 750W Hot Swap Power Supply

No Operating System

5 Year Next Business Day Warranty

Additional 8 x 2.5" SFF Cage and 8 Port HBA (for future upgrades if required)

 

APC SmartUPS 1000VA UPS (Tower)

 

Should be delivered 15th June, allowing me plenty of time to play with / prep it before my migration week of 20th July.

 

Thanks for all your help

[hallb15]

Just a follow up to the original topic. I've bought a server from VeryPC with the help of VeryPC_Ed with the following spec.

 

Dual Xeon E5-2620v3 Six Core Processor

64GB DDR4 2133 ECC REG RAM (8 x 8GB DIMM Modules)

DVDRW Optical Drive

Intel 8 Port RS3UC080 SATA/SAS HBA

Intel S3500 120GB Series Enterprise SSD (2 - Raid 1)

Intel S3610 Enterprise 400GB SFF SATA SSD (4)

Seagate 900GB Enterprise SFF 10k SAS HDD (4)

Redundant 750W Hot Swap Power Supply

No Operating System

5 Year Next Business Day Warranty

Additional 8 x 2.5" SFF Cage and 8 Port HBA (for future upgrades if required)

 

APC SmartUPS 1000VA UPS (Tower)

 

Should be delivered 15th June, allowing me plenty of time to play with / prep it before my migration week of 20th July.

 

Thanks for all your help

@VeryPC_Ed knows his stuff when it comes to servers. Looks like a good spec - please keep us posted how you get on with it.

[PotNoodleTech]

 

[craig-schools]

The server arrived on 16th June, so I've had it here for a while now.

As space is limited in the building I've had it on my desk and to be honest the only time I can hear it is when it is booting, fans on full etc, the rest of the time it is whisper quiet.

I've setup some storage on it and have begun to start spooling HyperV Servers in prep for the migration. I've even moved some of the less critical services over already like MDT / WDS and my AV.

 

It's super quick and I've popped the side off to check the inside layout, cable management is top notch. All in all I'm a happy camper.

[PotNoodleTech]

Fit like a glove!