seawolf

Pete, To answer your question, I would be inclined to looks at a hybrid SAN (as suggested by glennda) or unified storage (Enterprise NAS) solution if I were in your shoes. I don't have personal experience with the Nimble product, but technically it looks to be a very good solution, just don't know the cost.Other alternatives I have used are the IX Systems hybrid SAN and NAS devices (based on FreeNAS) and the Oracle ZFS storage devices (7120, 7320...7130 coming out soon). Another potential solution is to build your own SAN based on the SAM-SD specification -http://sam-sd.john-refactored.com/doku.php - I've heard a lot of good chatter about these.

Hybrid solutions are a different story and can be quite cost-effective, I agree. They offer the best of both worlds right now. If all someone needs for their storage solution is 1-3TB then pure SSDs solutions can be had for a reasonable price. Storage needs for most are going up, not down though and in my case I need 20TB of enterprise storage as a bare minimum. A pure SSD solution for that amount of storage is FAR more expensive than a traditional or hybrid SAN solution. One day that will change, but that's not the reality today.

Enterprise-grade SSDs are VERY expensive. Read and write caches/accelerators even more so. I'm right now looking at a quote for some flash based wire accelerators and they cost $4,000 each for 73GB. Combine very expensive with the need for large data storage that most people need and you're looking at writing a cheque with LOTS of zeros in it. Sure, you can roll your own with a SuperMicro chassis, etc.and lots of 512GB consumer grade SSDs, but that misses the point of what SANs are used for. Now, change your timeframe from 3 years to 7+ years and you might be right. Just not yet.

WSUS Package Publisher http://wsuspackagepublisher.codeplex.com/ Puppet https://puppetlabs.com/puppet/what-is-puppet/

Dan, The testing you have done would seem to show that the server and server NIC itself is OK. It doesn't prove that the switch is bad, but it does appear to indicate the problem is network related. You only have a limited subset of servers (I assume you have more than 2 servers) and clients in your temporary test environment. The switch may be fine and you may actually have a client with a faulty NIC or that is compromised and flooding the network with traffic. My recommendation is to setup a network monitor (MRTG, Cacti, or free trial versions of PRTG or OpManager) so that you can see what is going on in your network. If you find nothing abnormal with that, then your switch may be faulty or the firmware may be buggy. At that point, I would reboot the switch, and run tests. Then upgrade the firmware and test again. That's what weekends are made for in IT - testing and troubleshooting....

Dan, I agree here, you need to be systematic in tracing this problem. You need to isolate it to resolve it and that's going to require focus. If I were in your shoes, I would first see if I could isolate the problem either to the server itself (it could in fact be the problem if it is the only thing affected by networking issues) or the switch (if it is not just the server that's affected). If you find that it does not seem to be isolated to the switch or server, then the job gets much harder and you will need to snoop your network (wireshark) and also use a tool to measure bandwidth and traffic on the network (source, destination, type, etc.) such as MRTG, Cacti, or PRTG. From your recent posts,I'm beginning to think the problem may be the server itself. For one thing you have an AD server also acting as a file server. That's generally not something you want to do. It could be a faulty NIC on the server. Then, there is the possibility that Sophos is causing problems on the server. We experienced so many problems with Sophos last year we abandoned it for Avast! But, only go down this road lookingnfor issues with the server if you can either isolate the issues to this server OR exclude the switch as a problem (through testing) - otherwise, you're just chasing rabbits down a hole.

Yep, Symantec have a pretty good rep for buying up smaller companies or products then running them into the ground...

What's the MTU seton the switch port the server is connected to? If it doesn't match the server, then that spells trouble. Also, I would be more inclined to leave flow control turned on at the NIC and disable it on the switch ports (all of them). However, if its off both on ALL of your NICs OR off on ALL switch ports then that won't be the cause of any problems. The sudden occurrence of your problem would seem to rule out flow control being an issue unless you have recently added any new devices (NICs) to the network that would correlate to the timing of your problems.

Have you used Wireshark to see what is going on at the time of the SRV errors and other issues? You might also want to look at using MRTG or PRTG to check the bandwidth on all of the ports (or download and install OpManager, you get full features for the trial period). Also, is this server a VM or a physical server? I have seen similar issues occurring when a server was being pushed beyond its capacity, particularly in regards to disk I/O. If you are seeing high read or write latencies, then you will want to look into that further. This will cause some big problems with VM servers in particular.

I would look at FOGand some of the forum articles in creating "universal" images http://fogproject.org/forum/threads/how-to-use-fog-to-supply-machine-specific-drivers-for-windows-7-sysprepped-machines.694/ We've used FOG for about 18 months and it works much better than our previous solution, Ghost.

I would recommend using BackupAssist (http://www.backupassist.com/index.html) for backing up your physical servers (as long as they are Windows servers) and storage and Veeam for the VMs. I've used BackupExec, NetVault, Acronis, and Zmanda in the past and found that BackupAssist just works better overall for Windows backups. It's inexpensive, dead simple, reliable, and quite flexible. The backups are also non-proprietary so no worries about being able to recover from the backup media (same goes for Zmanda). If you have a need to backup physical Linux or Mac servers as well then it won't do that and I would probably go with NetVault or Zmanda in that case. Veeam works a treat for backing up VMs, and is very flexible. It can be a finicky and can throw a wobbly occasionally with things like CBT faults and such, but these are usually pretty easy to resolve. The ability to test your backups in a sandbox and replicate to secondary storage arrays on-site or off-site makes it a real winner.

Wow, it is expensive there then. I could redo either WiFi or BYOD (not WAN though) at our medium size school for the equivalent of £10,000 - and Australia is not a cheap place for IT. The WAN would cost us closer to £40,000, but we're considered to be in the "bush" by Telstra (you know - a whole hour and a half drive from Melbourne).

That's another good idea. Setting up your school's own TV station would also be a very practical and high profile project you could do. Amazingly, my school had a TV recording studio in my elementary (primary) school back in the 70s (showing my age) that was kitted out with professional TV cameras and a fully equipped studio and editing room. I can still remember working the cameras and taking direction from the control booth ("camera 4, zoom in on interviewer..,,hold steady...cutting to you in 5,4,3,2.1.." - amazing experience that influenced me into technology. I think a local TV station had donated it all, because the school certainly didn't have that kind of money. It would have cost a mint back then.

Can't say I'm impressed with the guidance you've been given from your leadership, but it does present an opportunity. It sounds like you have the basics down, so there are a few options I suppose depending on whether any of the areas below are weak: - Backup/DR: do you have great backup systems and a comprehensive Disaster Recovery strategy? No, it doesn't have ANY of that "wow" factor, bit it might save your school from a catastrophe, and that's far more important. - WAN : What type of connection do you have? Do you have a high speed fibre connection? If not, then the school could get a lot of practical and marketing mileage out of getting one. - WiFi: Do you have a really good campus-wide WiFi for staff and students to use (e.g. Ruckus, Aerohive, Xirrus, etc.)? If not, then might be an opportunity to fill this gap. - Marketing/Social Strategy: Do you have a really good CMS-based website? Are you using Social media effectively? Do you have an app for students and parents to use (iOS, Android) to provide info on upcoming events, news, policies, direct communications between the school and parents, etc? If not, then here's your opportunity. - Distance Learning: You could use the money to setup or extend your website and LMS capabilities to provide distance learning opportunities. - BYOD program: you will find you may have many gaps to fill to properly implement one of these, including an informative parent and student portal for the program, systems such as Absolute Manage, SCCM, or Casper Suite to enrol and configure BYOD devices, additional Server CALs, upgraded WiFi, a good web filter to make BYOD less of a pain in the rear (like an iBoss 3550, which we use), etc. the costs can add up quickly. Of course, you might have enough money to do more than one of these and that would be even better (I don't know what £10,000 will buy you there). I would pick the top three ideas you come up with or are suggested, do a bit of initial costings for them to ensure feasibility, and present them to your leadership and have them decide - that's what they are paid for. Not your job to decide the entire future of the school's IT educational strategy. That's my brainstorm session for you. Good luck.

I would take a look at the firmware that is on the 4108GL. We have a 4208VL that we also had to upgrade the firmware on a few months ago due to persistent CRC/Alignment errors. The issue was resolved with newer firmware.

The 2012 errors would tend to indicate a NIC problem or an issue with the switch the server is attached to. Is the switch with the CRC errors the same the server is connected to? Have you tried to upgrade the NIC drivers? What HP switches do you have? I had an issue with 2520g-24 switches that were suddenly causing a lot of problems until firmware was upgraded. Just be aware the very latest HP firmware for some switches has a totally different (non-Java) GUI, s if you rely on the switch GUI proceed with caution.

I assume this is happening with all clients on the network? If it is, then you need to be checking the switch logs to see if you are experiencing a lot of collisions or flooding of the network from specific network ports. You could have a loop somewhere, but if you have STP enabled that's unlikely. I would be trying to isolate it to a particular switch or ports(s) because you might have a faulty switch, NIC, or a compromised client(s) flooding the network (malware or botnet). You might also see if you're switch firmware is up-to-date. We had some network problems earlier this year out of the blue that was resolved by a firmware upgrade to all of our switches.

I like the concept of making servers service-based as well to limit the possibility of an upgrade or problem with one system causing issues with another installed on the same server. However, I think 50 servers for your environment is overkill. There are many services that are complementary or very unlikely to ever conflict and can be installed on the same server. I think 25 VMs would be the absolute upper end for your requirements and you could probably do it with 20 or maybe even less and still have services very logically split across the servers. Even if you are going to do wireless as a second phase (we did the same), I would recommend getting PoE switches as that will make it much easier to roll out as many WiFi AP's can use PoE, which will mean less work and cost to install them (no mains power required for each). If you don't purchase PoE switches you could always use PoE injectors, but that's an extra cost and adds mess into your data cabinets. I know, because I have one building that I didn't buy a PoE switch for and we have to use 6 injectors to power all of the APs (and security cameras) for that building.

@Sam_narula I am a ICT manager at a P-12 school in Victoria, Australia as well and went through a very similar upgrade cycle when I took over the role in late 2009. We have 75 staff, 550 students, over 400 college owned devices (Mac, PC, and iPad) and an iPad and BYOD program. I have replaced nearly every system, both hardware and software in that timeframe. In 2009, many systems did not work very well at all, there was very limited WiFi, no working backups systems, and no remote possibility of a BYOD program being implemented. We had the virtually the same problems that you need to solve or worse and we have managed to do so for 99% of them (I won't say 100% because when does that ever happen in IT?? . We are about to undergo our next upgrade cycle at the end of this year and I would be happy to share my experiences with you and even give you an onsite tour if you want. If I can save you some headaches I'd be willing to do so. The choices in our systems and service providers has been made after significant testing with the desire to have reliable, cutting edge, and integrated systems. and it has cost me a lot of blood, sweat, and tears to make that happen. Some of the systems (hardware and software) we use is the following: - Sun (now Oracle) servers (x4170s): more expensive upfront, but fantastically reliable, high performance and very good support. Ensure you have plenty of RAM, at least 48GB and 2-3 servers. - Sun Unified Storage Array for SAN (7110, moving to 7120 soon). Excellent performance, flexibility, and GUI. These had some initial problems when they were first out - but the initial issues were fixes and they are great units to build your VM infrastructure on top of. On the other hand, you might be able to get by without a SAN if you have 3 high quality VM hosts and excellent backup and DR processes. - VMWare vSphere Essentials Plus: Good for up to 3 servers, which is more than adequate if you are specing up your servers as you should - HP ProCurve switches throughout. We use mostly 2520G-24 (PoE) for edge with a couple of 2810s in a building we don't require PoE and for the DMZ and SAN switches where we also don't require PoE. Core switch is an older HP chassis model 5304xl that we will be upgrading in the near future and a 4208vl. The HP chassis switches are great for the core as they offer great flexibility and capacity as your needs change. Unlike your situation, we have 16 separate buildings, 16 switch cabinets, and 23 network switches, so a bit more complex. - Watchguard firewall: Used these for over 6 years now at two companies. Great bang for the buck, including good multi-WAN and built-in VPN features. - iBoss web filter: Easily the best web filter you have ever seen, and GREAT value. You won't see these very often in Oz and I had to order them direct from US at the time, but the support is fantastic and everyone I've ever demonstrated it to, including companies selling other filters admitted it was unbelievably good. It is more than BYOD ready, it makes BYOD successful. - Ruckus WIFi: We have 24 APs (internal and external) across 16 buildings providing full coverage of the campus and 5 different SSIDs on 5 VLANs (web filtering tied to VLAN for WiFi networks by iBoss). I also investigated Aruba, Meru, Xirrus, Aerohive, and Cisco. If I had to do it over again, the only other provider I might consider any sort of contender to Ruckus is Aerohive, which was on my short,short list anyway. Ruckus wasn't the one I initially thought I would go with, until I tested it in our environment and saw how well it worked in the real world (Tom's Hardware agrees - http://www.tomshardware.com/reviews/beamforming-wifi-ruckus,2390.html). - VEEAM backup and replication: A must have if you are putting all your eggs in the VM basket. - BackupAssist: We use this with an LTO tape drive to take more traditional backups. It is nix pensive, easy to setup, and just works. This and VEEAM have "saved our bacon" more than once. - FreeNAS server for backup storage: you won't find a more cost effective and reliable solution IMO. - FOG: it will take you a bit of initial setup time, but it is free and it blows Ghost away. Great performance too. - Windows 2008R2, SQL Server 2008, Windows 7: I would stay away from Win8. It is not ready for a managed environment IMO. Go with Windows 7. It's proven, it works, and it's easy to manage. And I've been told that Server 2012 is a dog's breakfast by a server admin I trust. Maybe in a couple of years, but not now. - Avast! SOHO: Relatively low cost, simple to setup and manage, just works, and is minimally intrusive. Combined with the AV scanning from Watchguard, and malware site detection on iBoss it provides a great level of protection. Previously used Sophos, but tired of their faulty upgrade to v10 that created issues for us and the botched updates that caused problems for many users (http://www.zdnet.com/sophos-antivirus-detects-own-update-as-false-positive-malware-7000004565/). I used Avast! for several years elsewhere as well and it worked well. Of course there are some other good ones I'm sure, but Avast! Is the best I've personally used (I've used Trend Micro, CA, MacAfee, Symantec, AVG in the past as well). - PRTG and Splunk: a couple of the best and easiest to setup and use monitoring and logging tools available for the $. Have proven priceless on several occasions. Ping me if you want to know more.

I think the judgement on the cost depends in just how much hand holding you're getting from the engineer. If they are going to doing the following for you then it may be worth it if you have no experience in the area: 1. Determine your requirements and go through possible solutions (magic triangle with AD, OD, and WGM, or just AD and Profiles) 2. Ensure your environment is compatible and there are no issues such as having a .local domain (it could still be done, but the results wouldn't be great) 3. Configure Mac server to requirements (including DeployStudio or similar for imaging) 4. Create image to SOE requirements and upload to imaging server, test with you to ensure meets requirements, 5. Test deploy SOE image and ensure bindings, file mapping, printing and other management profiles or management preferences do what you expect and that nothing is missing or not working properly. 5. Deploy 100 Macs and ensure working as expected and to requirements. If they will be doing all of that, then it might just be money well-spent. If you have no clue about Macs or how to manage them, particularly in a Windows server environment then a good engineer holding your hand through all of this will save you many headaches. Imagine setting up an AD, DNS, file and print server, all of the GPOs, and an imaging server (GHOST OR FOG) if you had no clue about Windows or AD and that is the equivalent of what you are about to do in the Mac world. Someone with plenty of experience and knowledge will be well worth it. Of course, they do need to be knowledgeable and experienced....

I've retrofitted SSDs from OWC, Samsung, and Sandisk into over 45 Macbook Pros (2009-2012 models), Mac Pro (2007 and 2010 models) and Mac Minis (2009 and 2010 models) over the past two years with zero failures so far and no compatibility issues. In fact the only SSD we've had fail was an Intel SSD that came factory installed from Apple on a 17" MacBook Pro that was CTO. That drive died after only 11 months. For performance the Samsung and OWC Extreme models offer the best performance and value I've seen.

I have not experienced performance issues with 10.8.2 on a Windows network, either with logins, file share access, or printing. We have had no issues with the in-built AD bindings (e.g. We are not using any 3rd party solutions that would mitigate these sort of issues you're seeing). Just so you know how we are setup - Our AD and file servers are running Windows Server 2008 R2 SP1 and we have HP ProCurve Gigabit switches throughout with 12 buildings connected with a mix of fibre and CAT6. We have 170 Macs, 135 PCs and around 150 iPads and iPhones on the network. We use a non .local domain (we previously had a .local domain and DID experience the issues you refer to). We also map the Windows 7 and 10.8 user accounts to the same network home. We modified the Win7 folder redirection to match the folder names used by OSX as our users may log onto a Mac in one class and a PC the next(works perfectly). That's our setup, so what are the results? 1. Average login times are under 10 seconds after initial account creation (upto ne minute for initial creation of user account). This s better than the Win7 clients on average. 2. We see fewer problems with PHD once setup (sync errors) than in 10.6. However, we did have some initial problems getting the managed settings for mobile sync to take. Sync is also faster than before. 3.File,sharing services over SMB are quick and work well. However we do use ExtremeZ-IP to provide AFP and Windows print services, so use of SMB was limited to testing and not everyday use under load. My point is that I think something else is going on in your environment, possibly DNS issues. If you were using a .local domain I would definitely peg the blame on that. But you aren't, so the next place I would look is DNS. I would also look at the logs (console) to see what is happening during these failed logins or long login delays - you should find some answers there.

The latest Flash patches have had some stability issues (http://www.joystiq.com/2013/04/24/unity-ending-flash-support/), this could be related.

There are substantial changes from 10.6 server to 10.8 server. For one, the Server Admin and other server tools are gone with the exception of WGM, although I haven't tested WGM thoroughly enough on 10.8 to confirm it will function as expected (or as it did with 10.6). Era reusing profiles with only a couple of non-conflicting MCX settings for our 10.8 clients. The Server.app has taken over from Server Admin and while it has improved from 10.7 it still does not have and may never have the granularity of control Server Admin had. Server.app has been "dumbed down" to make it easier for SMBa and home users to set up a server. It does that fairly well, but means that you have to dive not the command line to provide the same level of control for some services that were available in the Server Admin GUI. There are also some services that are been plain broken IMO in 10.8 server, such as Mail and SMB file sharing. 10.8 uses SMBX rather than SAMBA for SMB services and it breaks for many Windows clients in my testing, but does seem to work fine for OSX clients I tested. I also found that SMB would not work for copier scans, particularly with Xerox units. I finally had to use FTP to overcome that. Mail Server is now useless for all but the most basic needs and you will want to avoid using it. That said, there are some positives. I've found 10.8 server to be more stable and faster. Profile Manager and built-in WebDAV file sharing makes managing iOS clients possible without any third part software. There are still some rough edges, but it is good value. Caching server has some real potential, but i have not tested it out yet. Wiki Server has also improved. My suggestion would be to setup a test server running 10.8 on a Mac Mini or any spare client on an isolated subnet or network and test it out. I would recommend against upgrading a 10.6 server though - just make a clean start and test, test, test.