Architecture of this server

[glennda]

your nas box would indirectly be the file server, it is worth still keeping a windows box with the drives on the nas box as in my opinion it would be easier to configure for permissions etc. ill do you a little diagram in a min and post it up.

 

Toby

[mirojr]

p.s current server is a hp proliant g5 dl380 blade with 3.35gb RAM

 

546gb hdd space currently using 233gb

 

5x146gb hotswap hdd's with 3 spare bays

 

growth on the hdd space is not too fast

[Duke]

I thought the NAS would be the file server?

 

It totally depends on how you want to do it. If your NAS solution supports CIFS (Windows shares) and Active Directory authentication then there's no reason why your users can't talk directly to the NAS box for their userdata. You'd just make shares on it and point your users to \\NAS\share

 

However, if your NAS just acts as 'dumb' network-accessible storage, or maybe you got a SAN that doesn't offer CIFS, you'd need to map storage on the NAS (over the network) to a Windows server. The Windows server will then be used to share that storage out to users and control permissions, quotas, etc.

 

My Sun 7410 has full AD and CIFS support, so my users talk directly to it. In this way I get better performance (no Windows server bottleneck) and one less single point of failure. The Sun S7000 stuff has a really nice management interface and I'm happy to use it for this.

 

However, if I was using my NetApp box and didn't have the CIFS licence, then I'd make an iSCSI LUN and use the Microsoft iSCSI Initiator to connect it to a Windows server (where it would appear as local storage, even though it's connected over the network). I'd then use Windows to make a folder structure on the iSCSI drive and share the folders as required. The Windows file server would handle all of the permissions and access, even though the data itself would be fed back to the SAN/NAS.

 

Hope that helps, it's all about what hardware you've got and what's going to work best for you.

 

Chris

[jtotheb]

The ML115 states only Raid 0,1,5 does this mean i can do 1+0 or not

 

Thankyou v much for input by the way

 

I've got an ML115 at home. It does Raid 1+0 (I've got 4 x 500 gig drives as a ~1TB striped, mirrored volume). I initially wanted to run it in Raid 5 but found the performance in Windows quite painful. 1+0 performs perfectly well for me.

[mirojr]

It totally depends on how you want to do it. If your NAS solution supports CIFS (Windows shares) and Active Directory authentication then there's no reason why your users can't talk directly to the NAS box for their userdata. You'd just make shares on it and point your users to \\NAS\share

 

However, if your NAS just acts as 'dumb' network-accessible storage, or maybe you got a SAN that doesn't offer CIFS, you'd need to map storage on the NAS (over the network) to a Windows server. The Windows server will then be used to share that storage out to users and control permissions, quotas, etc.

 

Hi

 

The idea of having a seperate storage area independent to the domain controllers sounds great.

 

I am thinking that if i have to upgrade a server or something, i then do not have to worry about migrating the data(which is one thing i am worrying about although it might be simple)

 

If my NAS is dumb(which i presume it will tell me if its CIFS or not in description), is it straight forward to map the drives through windows server, as if its on the servers local drive?

 

Thanks

[glennda]

yeah is simple, download and install the microsoft IScsi Initiatior which once set up makes it think that the NAS box is an internal hard disk dirve, rather then a mapped drive. so you would then create folders etc just as if it was a normal server with the hard disks built in. Migrating the data can be simple, check this out EduMove its an easy way to migrate files across servers will keeping permissions folder names etc the same.

 

Just make sure if you buy a NAS that it supports IScsi

 

Toby

[mirojr]

Nice diagram by the way.

 

I am looking at the NAS boxes seems that LG N2B1D NAS Enclosure with Integrated Blu-ray - Misco.co.uk

this only has 2 bays

 

this QNAP TS-419U 1U NAS Server Enclosure with iSCSI - Misco.co.uk looks better, but would it be cheaper to have another ML115 and maybe I could get this to be a decent NAS.

 

And then there is this type QNAP TS-410 4 Bay NAS Enclosure with iSCSI - Misco.co.uk

 

wondered why its suggested to use the ML115 as the esxi server over the DL380?

 

Thanks.

[glennda]

you could buy another ML115 an stick the disks in then run freenas, its up to you and depends entirely on budget and how you would like to play it, obviously setting up a prebuilt nas box would be simplier then creating your own as the software will guide you through it all by pressing next.

i would suggest running ESXI on the New Server for the few fact that its newer and less likely to fall over

[mirojr]

Cheers

 

Old server prob needs more RAM too.

 

Whats the best method of moving users folders and data from one DC to another or from DC to NAS

 

Ta

[Guest Guest]

Unless you're going to add more servers in the future i dont see the point in you having shared storage (ie a NAS/SAN). Direct access to the HDs in your new HP server will be much easier to impliment and will be faster (assuming you dont go for a sun box with a nice san switch etc etc).

 

Id then use the NAS you are looking at for backups. Grab a script off the citrix/vmware site which will snapshot and export your VMs to the backup box via NFS. Personally i think the shared storage will mean you're jumping in at the deep end with no real benifits of doing so (at this point in time). Better to start small with a couple of VMs on a baremetal host, and thats it. Believe me, thats plenty to learn (properly).

 

If you do go the NAS/SAN route, i wouldnt be looking at that LG nas. Its only got 2 HDs so will have to be run in mirror raid. So effectively you are running # VMs off of 1 consumer grade SATA HD. It will be slooowwww!

Edited February 25, 2010 by Guest

[mirojr]

I was thinking really about NAS so that i wouldnt have to migrate data if i needed to demote a server for an upgrade or any other reason. Not sure how all that works with a NAS anyway, but i thought it would be simpler if the DC didnt have the data on it, hence it wouldnt need moving about

[mirojr]

Just so the post on edumove gonna have a look now.

 

What do we think of this model:

 

QNAP TS-419U 1U NAS Server Enclosure with iSCSI - Misco.co.uk

[Duke]

Never tried Edumove personally, but most people are happy with RoboCopy, or you can even just use xcopy with a lot of switches. I don't know about other products, but Sun have a feature where you can select a location to migrate from when you create a new share on the box and it moves the data for you.

 

No personal experience with QNAP so I'll let someone better informed reply.

[mirojr]

The OS directly on the server is VMware ESXi (bare metal install). The OS's of the virtual machines running on it are a mixture of Linux and Windows. That server with 8GB RAM is happily handling 5 VMs currently and will take a lot more as they're fairly low-usage.

 

Booting from USB because it was just a bit easier. I bought the bundle deal from ServersPlus which was that server with 4GB RAM (bought an extra 4GB) and a USB stick with ESXi already installed. I did a fresh install of ESXi anyway, but as ESXi does so few read/writes the USB stick will probably work out more reliable and easier to replace than the internal HDD. I can swap out a cloned replacement USB stick with no issues, whereas if I was using HDDs I'd probably want them mirrored in RAID1.

 

I said I'm using a SAN for storage, but as it's NFS it is more accurately NAS.

 

SAN = block-level (iSCSI)

NAS = file-level (CIFS, NFS)

 

I've got a Sun 7410 Unified Storage system which does iSCSI, CIFS, NFS, HTTP, FTP, etc. NFS is the most flexible and gives best performance with VMware in my experience using this hardware.

 

Having shared storage (be it iSCSI SAN or NFS NAS) isn't so much of an issue with ESXi as it runs as a standalone host (I'm talking about the free version of ESXi here), but eventually I'll have a pool of ESX servers using shared storage. In this way, if an ESX host fails another ESX host which has access to the same shared storage can automatically take over. If you're unfamiliar with the concept, Google 'VI3 Demo' and watch the old VMware video. It basically takes your downtime in the case of a server failure from a few days to about two minutes. With VMware Fault Tolerance you can get that down to less than a second of downtime (there's a couple of limitations).

 

As far as your 'expensive' comment goes, it depends. I was looking at £100k+ for a NetApp solution. My Sun box is the top-end model and came in at less than a third of the NetApp quote and is still bigger and more powerful. Sun, EMC, Dell, etc. all do cheaper solutions, and you could even build your own SAN/NAS using OpenFiler or FreeNAS for the same price as a decent PC.

 

Hope this helps,

Chris

 

Hiya Chris,

 

Could you just clarify about this esix thing for me plz?

 

I get the impression that you dont have anything on this server its all (VMs etc) residing on shared storage?

 

Is that right?

 

If so could i install the esxi on the HDD and have the VMs etc on shared storage, i have run VMs like this before i think, but wondered if things would be slower.

 

Would it be tricky to have exsi see the external storage for the initial setup?

 

Thanks

[Duke]

Could you just clarify about this esix thing for me plz?

 

I get the impression that you dont have anything on this server its all (VMs etc) residing on shared storage?

 

Is that right?

 

If so could i install the esxi on the HDD and have the VMs etc on shared storage, i have run VMs like this before i think, but wondered if things would be slower.

 

Would it be tricky to have exsi see the external storage for the initial setup?

 

Sorry, been in a meeting all day (talking about Sun storage and VMware oddly enough).

 

If you use shared storage then the only data you'll have on your server is the ESX/ESXi install. This is the hypervisor 'operating system' that manages the virtual machines and actually does the processing. When the VMs are running, they will be doing their processing on the server and using it's RAM, but their disk files will stay on the SAN.

 

All the VMs virtual hard disk and configuration files will be stored on the SAN/NAS box, which is particularly important if you use ESX and have things like failover so one server can take over from another if one dies. This way, all your ESX servers can see the VM disk files.

 

You could install ESXi on the server's hard drive, or run it from a USB stick. Support for local storage on cheaper servers with vSphere4.0 is very good, but if you do have issues then running from a USB stick will often work. There aren't a huge amount of read/writes so performance isn't an issue.

 

No, things shouldn't run slower if your SAN/NAS and network bandwidth are both good. What we're talking about doing here is a common, standard way of doing virtualisation and it works very well as long as you use appropriate hardware. Access to remote storage can be incredibly fast if done right.

 

You configure access to the external storage once ESXi is up and running, you don't need to worry about making it available while installing ESXi. As long as you use a supported protocol (NFS or iSCSI) and your SAN/NAS works as it should then connecting the ESXi server to the shared storage is very simple.

 

Watch this and you'll find a lot of the virtualisation/shared storage stuff makes more sense: [ame=http://www.youtube.com/watch?v=qWf_WiaFedc]YouTube- VMware Infrastructure 3 demo[/ame]

 

Note that this video is now two generations of VMware old, and things are now even more impressive.

 

Hope this helps,

Chris

[mirojr]

Thanks for info and video Chris,

 

I was just put off about using central storage on a QNAP 439 after reading that post :

 

Originally Posted by adent

Depends what you want to do with them. Be VERY careful if planning to use SATA in a virtualisation project for example. We've seen a number of schools where the NAS or SAN unit collapse after about 8-12 months because the drives fail.

 

SATA is very good for file storage and archiving. Very bad for vitual server images due to the high random io.

 

If that is the case, i would be looking at keeping the VMs on local HDD or the other servers HDD

 

Thanks

[Duke]

I was just put off about using central storage on a QNAP 439 after reading that post : [...] If that is the case, i would be looking at keeping the VMs on local HDD or the other servers HDD

 

Take a look at my reply to that post too and bear in mind SATA can work as a virtualisation solution if it's done right, although obviously the Sun S7000 uses SSDs to really accelerate the performance.

 

If you're running the VMs from the ML115 server then remember it uses SATA disks anyway (don't think it takes SAS?) so there won't be a huge performance difference between your local HDD SATA disk and your networked centralised storage disks. With SATA alone you'd probably be looking at RAID 1+0 to get the performance you need anyway, regardless of whether it's local or centralised.

 

Not sure about adent's comments about units dying due to disk failures - enterprise SATA MTBFs and enterprise SAS MTBFs aren't too far apart these days and there's no reason a SATA-based device should be more likely to fail as long as it's designed properly. Bear in mind that NetApp, EMC, HP, etc. all use SATA disks in some of their SAN/NAS products where capacity is more of a requirement than speed. I think it's more a case of excessively cheap or poorly designed products rather than ones that use SATA disks.

 

Chris

[mirojr]

Hello, Thanks for the video, i liked it.

 

I was a bit that my 2 servers are different architectures so i cant move my VMs to different hardware. tut.

 

Nevermind though. I can still move the vm images to different storage areas.

 

 

My QNAP 439 will not support raid 1+0, so wahts best option for tht if i use this to store images.

 

My new server could be RAID 1+0 like that. Great

 

As for my current server DL380 g5, SHOULD this be raided like this if its virtulised, and should it be raided like this anyway or would you only do that if its going to be a virtual host?

 

Thankyou

[Guest Guest]

4 HDs? Its got to be RAID 5 then.

 

What you generally do is;

 

NAS/SAN - the most important thing here is HDs. SAS > SATA. More spindles (HDs) the better. Give it a nice bit of RAM for caching and away you go. You dont need mega processing power.

 

VM Hosts - Put the HDs in a RAID1 array so that you have redundancy. Nothing, other than the host OS, sits on these so no need for then to be big or fast. Pack your hosts out with as much RAM and CPU power as possible.

 

 

 

You really need to be aiming at having atleast 2 similar hosts. If you have a motherboard/PSU/whatever failure your whole network is going to be down. Same goes for your SAN really. A way around this may be to have a linux box as your backup box with NFS already configured so that if your SAN does go tits up you can just add the backup box as the SR.

[Duke]

NAS/SAN - the most important thing here is HDs. SAS > SATA. More spindles (HDs) the better. Give it a nice bit of RAM for caching and away you go. You dont need mega processing power.

 

VM Hosts - Put the HDs in a RAID1 array so that you have redundancy. Nothing, other than the host OS, sits on these so no need for then to be big or fast. Pack your hosts out with as much RAM and CPU power as possible.

 

+1 on that. If you are going SATA alone for the SAN then you need to look at RAID 1+0 to get the required performance. If that's not possible then you should really be looking at a 15k SAS disk solution. You can run one or two low-disk-usage (e.g. webservers) VMs off a SATA RAID5 array, but not much more than that without noticing the disk bottleneck.

 

As j17sparky said, RAID1 for the VM hosts just so things keep running if a disk dies. Failing that, you can run it off a USB stick and keep a backup copied stick ready to swap out.

 

Chris

[mirojr]

not looking good for vm ware setup at this rate.

 

Maybe the old dl380 g5 server could be raid 1+0 to hold the vmware images. Thats got 5 SAS drives in at moment.

 

Seperate question: Why do you only run the esxi on a machine and not use the harddrive to hold the images also? Bad practice?

[mirojr]

Sounds like the only way to get round this in a reasonable budget would be to build own NAS, as all qnap stuff is sata and its STILL expensive

[Duke]

not looking good for vm ware setup at this rate.

 

Maybe the old dl380 g5 server could be raid 1+0 to hold the vmware images. Thats got 5 SAS drives in at moment.

 

Seperate question: Why do you only run the esxi on a machine and not use the harddrive to hold the images also? Bad practice?

 

Not so much a bad idea, and if you're using ESXi then it's not too big a problem. The main reason for shared storage is if you're using the paid version of VMware (be it ESX or ESXi) then you can 'cluster' your VMware servers so that if one ever fails then another server will detect it and start the VMs that were running on that host back up. To do this, the VMs need to be stored in a central location that all the servers can access, so having them on one server's HDD is no good.

 

I think based on what sounds like financial limitations for you, I'd avoid going for a large virtualisation deployment. It can be a really good idea, but it does often involve putting all your eggs in one basket so you need to make sure you have the right hardware.

 

If you've got a spare server, put ESXi on it and use the local drives, but don't run anything critical on it for now, at least not until you can get a good backup strategy going. If you can get two servers and some kind of shared storage then look at the free version of Xen, which will allow you to take advantage of things like failover for free.

 

Cheers,

Chris

[mirojr]

chris,

 

I was hoping to use the virtulisation for hardware consolidation. It would be a free version. Esxi or xen or hyperv.

 

This would allow me to put my linux proxy on and save having boxes all over the comms room.

 

Currently we dont have a BDC and next to nothing in regards to any failsafes concerning the current DC, just data and sys state backups. i am stating this so you can see that i am not maybe going to be in a worse position if i dont have failover. I would still have to rely on current backup strategies. But i would poss be in better postion with a BDC.

 

I was wondering how much load a DC is on a vm. If its going to be better to have a DC directly on the hardware on its own then i will keep it this way.

 

The BDC i could run on the esxi (but as raid5 only)? Then i can use this for the linux proxy too and poss lab servers?

 

Thanks

[Guest Guest]

^ What he said.

 

 

So lets start again. What do you have? What are you planning on buying? And what services do you need to run? Between us all im sure we can come up with something which is both suitable for your school, and abit more interesting for you

 

What id probably be looking at is buying a new server to be a fileserver and main DC. And putting ESXi/Xen on the old one and running a second DC, running your image deployment from it, any www services etc...

 

 

While i remember, one thing i dont think has been mentioned is you really need a dedicated switch. Nice and fast possibly with link aggrigation, and preferably with the ability to do VLANing.