Hardware Thread, Sun 7410 Getting Hammered - Suggestions? in Technical; Hey all, looking for any possible advice here. I've raised a case with Cutter who are looking into it.
25th November 2009, 12:40 PM #1
Sun 7410 Getting Hammered - Suggestions?
Hey all, looking for any possible advice here. I've raised a case with Cutter who are looking into it.
Weíve got a start menu share on our S7410 that has student and staff mapped/shared (via DFS) start menus on it. Everything has been fine for ages, but yesterday at just after 10:00 CIFS, Network and CPU started getting absolutely hammered, and CIFS is now showing a hurricane on the BUI.
The access is only to the staff start menu and is coming from staff PCs. It drops off overnight when no one is logged on then starts up again in the morning. The type of operation going on is NtTransact and isnít hitting the disks. Network traffic is an even split of in and out.
The only thing that changed around the time this started was that I added a new shortcut for software to the staff start menu. Itís just a shortcut to a html file but it did have a slightly unusual character in it (an e with an accent). Any idea if this could be the problem? Nothing else has changed to my knowledge. Iíve now removed that shortcut completely but so far it hasnít made a difference. It might not change until the DFS caching runs out (30 mins).
IDG Tech News
25th November 2009, 01:11 PM #2
What AV are you using?
25th November 2009, 01:17 PM #3
Sophos, installed on all PCs but there's no AV on the 7410 itself. Sophos has caused us problems before, but nothing like this. It crossed my mind, I'll investigate further...
25th November 2009, 06:13 PM #4
Time for Analytics!
Break down CPU in to process., identify which process is taking your CPU.
Break down CIFS to client access, file access.
See who? is doing what ? and when ?
25th November 2009, 06:18 PM #5
I'm working on this now and analytics is saving me. More and more this is looking like a Sophos problem, even though Sophos itself isn't reporting any issues. Gonna work until I get kicked out, will post an update once I find one.
So far (and I'm halfway through testing), it looks like you can log onto a PC fine as a member of staff, but once you open the start menu usage on the SAN pick up loads. Rather than dropping off once the menu is closed, it then constantly stays at that level. It's like Sophos is doing an on-access scan of the start menu when you mouse over a folder (which is fine), but then keeps scanning those files rather than stopping!
EDIT: Disabling on-access scanning hasn't made any difference. Next step is a complete uninstall.
Last edited by Duke; 25th November 2009 at 06:33 PM.
25th November 2009, 06:34 PM #6
Originally Posted by Duke
Can you add an exclusion in Sophos or just uninstall it on a PC to test.
25th November 2009, 06:35 PM #7
Yep, am working on it now. Should be able to give an update in 5 mins...
Originally Posted by cookie_monster
EDIT: Well, it's not Sophos. Uninstalled it on the test PC and still got the same problem. The weird thing is that it's not a bunch of files getting hammered, it's just the root programs folder on the start menu, e.g. /export/menus/Staff/Start Menu/Programs. I can see the individual files and folders pop up on the BUI analytics when I mouse over them, but they just read once then they're done. /export/menus/Staff/Start Menu/Programs sits there getting hammered after it's been opened once, even if the start menu is closed. Time to start killing services and processes...
EDIT 2: Alarms are going on site in a minute so I gotta run. No joy so far, even after killing off pretty much everything. I got SAN usage to dip a little when I killed some of the processes, but after giving it a few seconds it was back up to the normal level.
Last edited by Duke; 25th November 2009 at 06:59 PM.
25th November 2009, 06:59 PM #8
Broken link somewhere in the root and windows is try to resolve it?
If this were a folder other than a start menu, I'd be looking for a corrupt Thumbs.db
25th November 2009, 07:08 PM #9
Nothing obvious and certainly nothing that's changed recently. We do have some broken links (although not in the root) because some software is only installed in certain places. However, this has always been the case so is nothing new...
Originally Posted by pete
EDIT: Beer time, I'll let you guys know tomorrow if I get it sorted.
Last edited by Duke; 25th November 2009 at 07:12 PM.
25th November 2009, 07:30 PM #10
It seems strange that one Windows client could hammer the SAN across the network, i'm assuming when your testing that it's still killing the SAN.
26th November 2009, 09:37 AM #11
The SAN itself is actually running okay performance-wise and no one's noticed anything, probably because the traffic isn't hitting the disks, just network and CPU. I don't know exactly what NtTransact is but it doesn't seem to be doing any major reads or writes.
It looks like all staff PCs with a member of staff logged onto them (thus getting the staff start menu, students don't seem to be affected) are hitting it. Andy from Cutter's dropping by later for something else, might see if he has any ideas.
26th November 2009, 09:41 AM #12
Any mac clients? Although I have seen smb requests do the same to samba on OS X on many occasions.
26th November 2009, 10:21 AM #13
Nope, all XP SP3.
Originally Posted by DMcCoy
I've just removed all the shortcuts that were in the root of the 'programs' folder but it's made no difference so far. Fun and games...
26th November 2009, 10:21 AM #14
Give a test staff account a brand new blank start menu with just a link to notepad in, see what happens?
Originally Posted by Duke
26th November 2009, 10:31 AM #15
Has the storage box been rebooted? I found once the smbd process crashes it will spawn replacement ones constantly.
Originally Posted by Duke
By Duke in forum Hardware
Last Post: 3rd June 2010, 12:43 PM
By linescanner in forum Thin Client and Virtual Machines
Last Post: 23rd April 2009, 10:51 AM
By tmcd35 in forum Thin Client and Virtual Machines
Last Post: 19th December 2008, 01:05 PM
By localzuk in forum Thin Client and Virtual Machines
Last Post: 18th April 2007, 11:33 PM
By sidewinder in forum Wireless Networks
Last Post: 9th November 2006, 10:37 AM
Users Browsing this Thread
There are currently 1 users browsing this thread. (0 members and 1 guests)
Tags for this Thread