Spooler consuming high CPU - multiple erroneous \\PIPE\Spoolss

[pete]

One of our servers (2003 Standard SP2, configured as DC + File/Print server for admin offices) has a print spooler consuming excessive (60-80%) cpu. Users have multiple \\pipe\spoolss sessions open which immediately reopen if I force them closed, despite no print jobs being in the queue. Pipes remain open even if the user is no longer logged onto the machine. Shutting down the remote machine is the only thing that ends the session and reduces CPU use on the spooler.

 

Things I've tried so far:

 

Rebooted the server, deleted all printers, ripped out all printer drivers, checked for updates, and then replaced with new or known good versions (that work ok on a similarly patched server). Didn't reinstall the newest printer (HP3005dn) for good measure, just in case it was the culprit. For reference, the same model printer using the same drivers shared on another server works fine - no excessive spooler cpu consumption.

 

After the reboot, cpu use was normal until 10am this morning when it ramped up again and is currently sat at 93%. The event logs show a lot of print jobs where pages printed = 0.

 

i.e

Document 104, staff telephone list 0708.xls owned by USERNAME was printed on MO-BWLaser via port IP_$IP  Size in bytes: 117151; pages printed: 0

 

I know that document is 2 pages long at least, and I'm getting a fair few of these, so I'm assuming that the spooler is dropping jobs because of the load.

 

Software-wise the only changes we've made recently is updating the clients to office 2003, but it's been 3 weeks since we did this and excessive cpu consumption by the print spooler started on Friday, flagged up by Nagios.

 

Google reveals similar problems occuring on earlier patch versions of 2003 Server, specifically with XP clients, but the fix was part of XP SP1 (we're on SP2 + security/critical). http://support.microsoft.com/kb/835318/en-us.

[contink]

Is there any chance that the workstations that are locking things up have one of the following:

 

- printer monitor software installed

- old versions of drivers (or the above)

- old "printers" pointing to the same printershare but with the wrong drivers set

 

Only other thing I can think of is a clash between a recent Winupdate and the printer drivers.

 

 

Wild assed guesses all I'm afraid but maybe they'll nudge something loose.. Best of luck.

[pete]

No print monitor software and it's actually the server that's getting the high cpu, but stripping the drivers and any local printers (there shouldn't be any) from clients couldn't hurt.

[Michael]

You say you're using 2003 SP2, (which is fine), but by what method are you pushing out/distributing printers?

 

I personally use Print Management which comes part of 2003 R2. The advantage here is printers can be pushed out via GPOs and not scripts. Also, when you update the driver on the print server, the new driver is automatically pushed out to workstations on the network.

 

Is it possible you still have old drivers (as already mentioned) on workstations or the workstations have old printers pointing to the same TCP/IP port? This causes all kinds of problems.

[pete]

We're using GP deployment via pushprinters.exe. Workstations shouldn't have old drivers / printer mapping and most of the admin boxes were reimaged recently, I'm currently prowling around checking ones I'm not sure about.

 

Update: Ok, I'm satisfied it's not an old driver, all clients are using the current one - I think the culprit is HPBOID.dll and other assorted HP guff, so I'm investigating how to remove this and keep a working driver.

Edited March 17, 2008 by pete

[Michael]

Then you are using R2 Print Management. Within Administrative Tools on the Start Menu (on your print server), look for Print Management.

 

Seeing as you are using R2 Print Management, I would encourage you install the latest print driver on your print server, then check (after a workstation reboot) whether that driver is then being used on the workstation.

 

In my experience, when installing HP printers, they create a HP TCP/IP port. You need to create a standard TCP/IP port and delete the HP port. It's a load of rubbish and creates nothing but problems!

[contink]

In my experience, when installing HP printers, they create a HP TCP/IP port. You need to create a standard TCP/IP port and delete the HP port. It's a load of rubbish and creates nothing but problems!

Small addendum to that re: HP printer drivers.

 

In some cases even installing the new printer driver to the machine doesn't clear the issue and requires any and all printers to be removed and then re-added from the workstation.

 

Had this happen with an HP L7680 with the 8.0.1 version driver installed but it was still using the older one for the printer in the printer list. Weird didn't cover it

[mac_shinobi]

what event id are there errors you are getting on the server that are related to the printing ?

[Michael]

With R2 Print Management you don't have to remove printers manually. Printers are added dynamically everytime a user logs on (in my case) as I deploy on a per user basis.

 

To prove my theory, if you take a look at a workstation printers from My Network Places, the printers are not listed.

 

I agree also, the next step is taking a look at the event logs. Both server and local machine. You can do this all from MMC.

[pete]

updates again.

 

# Event logs

Other than 0-page print jobs, mentioned earlier, the event logs on client and print server show nothing out of the ordinary. As far as the client and server are concerned printing is happening ok. None of my users have complained that things aren't getting printed.

 

# Standard TCP Connections for printers

Yep, we do that anyway, excessive printer guff causes too much grief.

 

# Drivers

We're already using the latest versions of the drivers from HP - even the minimal drivers install unnecessary guff.

 

I did find this: http://forums11.itrc.hp.com/service/forums/questionanswer.do?admit=109447626+1205836342961+28353475&threadId=370850

 

the cause is different, but the symptoms are similar. We don't have HPBPRO.EXE running on the server, but one of my suspects is the BOID.dll, which was holding a lot of connections open (and is installed as part of the HP driver).

 

So I deleted the printers from the server, ripped off the HP drivers again and commented out the HPBOID.dll and HPPRO*.dll entries in the .inf. I then re-added the driver and printers.

 

My users have been in since about 7:30, cpu usage on the server has stayed below 25% and printing is going fine.

[ICT_GUY]

This might be useful

I find that a lot of problems are solved by deleting and reconnecting printers on logon. This script does just that, also has the advantage that if done in the computer settings in AD, it gets rid of printers the profile might have picked up else where.

 

Its a VBS script.

 

'printer script for non ict suite computers.

 

 

On error resume next

 

 

 

Set wshNetwork = CreateObject("WScript.Network")

 

'remove old printers

 

wshnetwork.RemovePrinterConnection "\\wykeserver\ForumPrinterOKI"

wshnetwork.RemovePrinterConnection "\\wykeserver\OKIC5100"

wshnetwork.RemovePrinterConnection "\\wykeserver\ICT_BW_OKI"

wshnetwork.RemovePrinterConnection "\\wykeserver\PPA_BW_OKI"

 

 

'add ict suite printer but do not make it the default.

 

Set wshNetwork = CreateObject("WScript.Network")

wshNetwork.AddWindowsPrinterConnection "\\Wykeserver\OKIC5100"

 

'add the forum BW printerand make it the default.

 

wshNetwork.AddWindowsPrinterConnection "\\Wykeserver\ForumPrinterOKI"

wshNetWork.SetDefaultPrinter "\\Wykeserver\ForumPrinterOKI"

 

'add the ICT BW printer

 

wshNetwork.AddWindowsPrinterConnection "\\Wykeserver\ICT_BW_OKI"

 

'add the PPA BW printer.

 

wshNetwork.AddWindowsPrinterConnection "\\Wykeserver\PPA_BW_OKI"

 

'add the PDF creator

 

wshNetwork.AddWindowsPrinterConnection "\\intranet\PDFCreator"

[Michael]

Impressive work Pete, I hope your changes work for you

[pete]

Grr, spoke too soon.

Service Warning[18-03-2008 11:57:02] SERVICE ALERT: SERVERNAME;CPU Load;WARNING;SOFT;1;CPU Load 83% (5 min average)

 

And yet again, nothing useful in the event logs. Back to the drawing board.

[Michael]

Have you tried this:

 

Change to the System 32 folder by typing cd\winnt\system32 for Win2K or cd\windows\system32 for WinXP.

 

Run the command "hpbpro.exe -RegServer".

Then run the command "hpbpro.exe -Service".

 

Then type "Exit" and then restart your PC.

[pete]

Yeah, that particular .exe wasn't present on the clients or the server. I remotely restarted all the client workstation print spoolers, which dropped cpu use down to normal levels on the server yesterday, but it's 13:49 and Nagios is about to notify me of unacceptable cpu load again.

 

It's report season here until Thursday, so I can't get at the workstations at the moment.