I was wondering how many people use Smoothwall on their own hardware vs. how many use Smoothwall on a UTM?
Basically since April this year we have been having performance issues with our Smoothwall box causing it to either to drop the Web proxy service or cause web pages to take up to 2 minutes to load a single page.
We currently run the Smoothwall box on our own hardware which is a dual quad core Xeon running at 2.33MHz (Intel L5410) 8GB of RAM, and dual 1TB Barracuda ES.2 SATA drives, along with 4 Intel(R) PRO/1000 Network cards (e1000e)
We only see these performance issues when Smoothwall reports in User Activity that there are over 500-600 users using Smoothwall. We are currently licenced for 1120 as we have roughly 1000 computers/laptops/tablets plus we allow some use of BYOD. Does anyone have Smoothwall running with this amount of users and if so what do you run it on and do you have any problems.
I have being using Putty to run Top to see what might be causing these issues and this is what I find. Dansguardian3 runs at between 500-700% and datastore runs at 100% all other processes jump up and down never running above 100%. The memory usage is very small but then I guess this is because Smoothwall is only 32bit and can never currently go over 4GB anyway.
I have had a number of support calls open with Smoothwall since April and mainly waiting for the new database update in the Summer which was going to help with these load issues but unfortunately it has not. The latest news is there are no software problems and it is down to our hardware being not man enough to cope with the load. Ok fair enough I thought there are three ways we can go with. One buy the Smoothwall UTM3000, two buy are own better specification server, three look at other products.
Please donít get me wrong I like the Smoothwall software what it does and when it works it works well in terms of functionality but we have had Smoothwall for 3 years now and although all these new updates come out bring new features it seems to need ever increasing resources. We first had the Smoothwall running on a Intel Core 2 Duo similar specs to UTM1000 which ran well for a year but then upgraded due to slow downs and now less than a year later we are looking at a massive upgrade again.
The UTM3000 sounds like a nice box albeit rather expensive at £5000 to buy but should Smoothwall really need such high specifications to run smoothly (pardon the pun) we currently have 8 virtual servers running on a server of this spec going from heavy use SQL servers to heavy use file stores and web servers it seems crazy. I donít think we would buy the UTM either way because last week we were quoted £3000 for a Dell PowerEdge R620 running 16GB of RAM and Dual Xeon E5-2640 processors with all the bells and whistles including a 3 years NBD warranty.
Please can anyone offer any advice to the situation we are in? Having just paid out over £3000 for our annual Smoothwall costs I canít see us getting another £5000 to pay for a hardware upgrade.
That's an interesting issue you have there, ours ran on a much lower specced box than that, albeit with far fewer users. I've had other solutions though on other boxes (in a previous post) with lower hardware specs and didn't have those issues. I guess rebuilding the box and reloading the config isn't an option, or have you tried that already?
@buzzard Just interested to know what are the other solutions you have had running, at this point I'm willing to try anything. Looked at Lightspeed but they only do it as an appliance I believe and seeing as we have just renewed to Smoothwall we would not have this money again until September. As to rebuilding I would gladly do this in the half term but as support have told me its a hardware issue is there any point I ask myself?
Just tried to look at the spec on the UTM3000 but gave up, wtf has happened to the smoothwall site! In one of my old posts (with about 1200 NOR) originally they had an RM Smartcache2 which ran well but was limited in capability, we then replaced it, but I'm struggling to remember with what, I have used Light speed and found it very good but there was another one we used.... hopefully it will come back to me! I have found Smoothwall the best all rounder, as I said I've only used that up to about 300-400 users but that ran on an old Dell Poweredge 840 again no reall issues on that, that (if i remember correctly) was a single quad core Xeon with about 8Gb Memory on a PERC5 controller (only one Intel 1Gbps NIC).? But it hardly taxed the system so I do struggle to see how/why your box is struggling so much, which I'd try a rebuild tbh, I did something similar to another smoothwall box because we couldn't work out a niggle, the rebuild and a restore of the config sorted out that issue.... If you're happy technically and have the time it's worth a try!
The UTM3000 spec is I believe dual Xeon E5645 processors, 12GB of RAM (only 4GB would really be used at the minute being 32bit) and 2 1TB 15RPM and 4 Gbit network cards. Nice spec but £5000 I think I could buy two. Sorry
There could be a couple of issues in play here.
First, we have seen connection tracking overloads on some sites - if you haven't already, go adjust the conntrack table (networking/advanced IIRC) - if it's set to 65000 or so (auto) double it.
If it's not that, there could be a few other things pegging out the box's processor. It does sound like you have an unusually busy system there. Would certainly be worth working out what's the bottleneck before laying out any cash either on our tin, or a 3rd party's (which may well be cheaper in some cases!)
@tom_newton We did have the connection tracking problem and this issue was sorted out via a support call at the start of September but now the problem is when we have over 500 users page loads get increasing longer upwards of 2 minutes loading per page.
I have had a support call open (ref 00045814) and the last response I have had said the problem is purely down to our hardware not being up to the job. I would love to find out the bottleneck before we change hardware etc.
Interestingly we had an ongoing issue with the web proxy failing which meant we had to restart the whole UTM1000 box fairly regularly. After a year of re-installing the box, even replacing it with a new box, Smoothwall support were unable to find a solution for us.
I never saw the processor spiking a lot though.
We "fixed" the issue by sorting our core switch out, we set up proper access lists etc. Since then the Smoothwall box didn't crash once, so it would appear there was to much traffic hitting smoothwall (a lot of it duff) which was causing the problem. We did however have our smoothwall UTM providing DHCP for our guest wireless, so it mgith have been guest wireless traffic that wasn't good.
To be fair it seems that support may have got this one right - the box is often seeing 600% CPU usage in Guardian alone, and load averages of 10+ on an 8 core box. Load average is "number of tasks waiting for CPU" - you have effectively 8 CPUs with 10/11 tasks waiting at busy times, so 2-3 are going to end up not served in a timely manner. You may just have a great deal of traffic - is there perhaps, as a previous poster speculated, one or two users applying dubious workloads?
We do have the 64bit release coming in a week or so, but I am not sure it will do a lot for you, given that your machine is already pretty effective in using its CPU cores to the max.
@tom_newton why is such a high spec machine required to do this job? I know when I've used SW and other filtering solutions it never needed such a beefy box? Has there been more features in SW recently or is it possible to turn off features not being used, I'm not disagreeing with your post, I'm just curious!
There does seem to be something amiss here, do you have any traffic management going on on the core network at all? where is all that traffic coming from? I'll see if I can look at my old place with SW setup to see what they have concurrently and compare to server performance, @tom_newton does SW have some metrics to compare this against?