Smoothwall Slowdown ? Especially in the morning.

[2097]

Any one experiancing this issue, especially since Monday this week.

 

Normal internet is fine , but.. As soon as i use Edge with the proxy applied every thing seems to take ages to run.

 

I have noticed a lot of these in the logs

 

08:56:57 Web filter ProtocolHandler prematurely aborted SSL client handshake, around the same time as the sudden decrease in speed.

 

Only seems to be an issue in the morning , After this it seems ok.

[2097]

Leeds, update 63 . I am on the latest Update.

[ibpalle]

Just a quick check - look at Network - settings - advanced. What's the size of the 'SYN backlog queue size' dropdown set to? An older default of 1024 is a bit too low for todays amount of cloud services so set this to the 32k value.

 

Otherwise, check bandwidth usage in the morning, load average and look at DNS lookup as well in the network. If you use the Smoothwall as firewall it can be useful to set your internal DNS servers to use Smoothwall as DNS forwarder for them. This reduces external DNS lookups immensely.

[fiza]

Just a quick check - look at Network - settings - advanced. What's the size of the 'SYN backlog queue size' dropdown set to? An older default of 1024 is a bit too low for todays amount of cloud services so set this to the 32k value.

 

Otherwise, check bandwidth usage in the morning, load average and look at DNS lookup as well in the network. If you use the Smoothwall as firewall it can be useful to set your internal DNS servers to use Smoothwall as DNS forwarder for them. This reduces external DNS lookups immensely.

@ibpalle Sorry to jump in on this but just checked my SYN backlog queue size and it is set to 8192. Do you recommend an increase?

[2097]

Just a quick check - look at Network - settings - advanced. What's the size of the 'SYN backlog queue size' dropdown set to? An older default of 1024 is a bit too low for todays amount of cloud services so set this to the 32k value.

 

Otherwise, check bandwidth usage in the morning, load average and look at DNS lookup as well in the network. If you use the Smoothwall as firewall it can be useful to set your internal DNS servers to use Smoothwall as DNS forwarder for them. This reduces external DNS lookups immensely.

 

Thanks for this . Correct mine was 1024 ( which is now 32k as suggested)

 

Question - can i view any logs that will show me when i applied smoothwall updates .

 

This is a recent occurance and i have been running smoothwall without issues for years . Only alterations i have done recently is to BYOD ( which i have now removed all these old settings )

 

We dont use Smoothwall as a firewall nor do we run it transparently.

[gaz350b]

See if there are any domains in the top 10 on the homepage with a massive amount of hits. We had a Microsoft domain being blocked 1 million times a day….

[ibpalle]

There is no listing for install dates for all updates but you can see the most recent date in the system - maintenance - system restore as the last system restore point will have been taken at the time the last update was applied.

 

Other than that, you would have to look through the logs manually.

[Primus]

@ibpalle Sorry to jump in on this but just checked my SYN backlog queue size and it is set to 8192. Do you recommend an increase?

 

Did you change it? I don't think ibpalle replied to your question and mine's set to 8192 too.

[fiza]

Did you change it? I don't think ibpalle replied to your question and mine's set to 8192 too.

no I didn't as I was hoping for a reply first @ibpalle ?

[ibpalle]

8192 is the new default - it should be sufficient. The queue was set low due to memory limitations when we were still running 32 bit OS. Given the memory amounts we have available now there are no issues running 32k values in that field. It's a hard one to quantify (similar to slow DNS issues at times) but test the different values, see if it gives you an improvement and go with that. I don't think you will ever see improvements going below the 8k mark though.

[2097]

Sadly still grinding in the morning, Any one able to help me compare some stats ?

 

On my homepage shows the following

 

Top domains (hits, 24 hours)

Bing 238475

Google 129789

encrypted-tbn0.gstatic.com 121794

play.google.com 62662

ib.adnxs.com 56030

hbopenbid.pubmatic.com 52168

http://www.youtube.com 48186

prg.smartadserver.com 46636

pagead2.googlesyndication.com 43897

ap.lijit.com 43229

 

Is it me or does Bing and Google seem a little excessive ?

 

Also

 

Google 4 (GBytes)

msedge.b.tlu.dl.delivery.mp.microsoft.com 3.8 (GBytes)

Bing 3.3 (GBytes)

d3acmx713t44o3.cloudfront.net 2 (GBytes)

http://www.samlearning.com 1.8 (GBytes)

http://www.gstatic.com 1.5 (GBytes)

http://www.youtube.com 1.5 (GBytes)

s3-dub-2.cf.dash.row.aiv-cdn.net 1 (GBytes)

static.parastorage.com 740.7 (MBytes)

mail.google.com 732.4 (MBytes)

[ibpalle]

Nothing there looks excessive as such. What is the DNS resolution time on the Smoothwall and on a client in your network?

[2097]

Primary DNS server latency results 8 DNS requests failed to respond.

42 DNS requests succeeded.

DNS resolution speed is a little slow or less than half the DNS requests were successful.

Average DNS request response time: 478.8381 ms.

[enjay]

Sadly still grinding in the morning, Any one able to help me compare some stats ?

 

This is ours:

 

Top domains (24 hours)

tlu.dl.delivery.mp.microsoft.com 38.1 (GBytes)

ccmdls.adobe.com 12.4 (GBytes)

1d.tlu.dl.delivery.mp.microsoft.com 9.6 (GBytes)

edgedl.me.gvt1.com 8.5 (GBytes)

3.tlu.dl.delivery.mp.microsoft.com 7.4 (GBytes)

fastdownloads2.texthelp.com 6.8 (GBytes)

11.tlu.dl.delivery.mp.microsoft.com 5.7 (GBytes)

Google 5.4 (GBytes)

cdn-ffc.oobesaas.adobe.com 4.5 (GBytes)

rr2---sn-aigzrnze.googlevideo.com 4.3 (GBytes)

 

Top domains (hits, 24 hours)

api.smoot.apple.com 265328

client.wns.windows.com 164901

Google 151952

secure.pmstats.org 145054

195ec04504ea0272771e-7c2c6dacbab7a2b2d574b53c70c1fe31.ssl.cf3.rackcdn.com 123939

outlook.office365.com 115820

files.classcharts.com 112496

tlu.dl.delivery.mp.microsoft.com 109259

cf-st.sc-cdn.net 79336

espresso-pa.clients6.google.com 76527

 

This is pretty normal stuff. The large number of Apple hits will be lots of iPhones on our BYOD network. pmstats.org is the PowerMAN software so computers are checking in for reporting but probably not sending much data. All the delivery.microsoft.com hits will be because we're rebuilding a load of computers.

 

A little surprising to see a random CDN site come up so highly, I'll keep an eye on that.

[enjay]

Is it me or does Bing and Google seem a little excessive ?

 

I see you've got Google Mail and some hefty YouTube use, so a big hit to Google doesn't seem all that odd.

 

I'd be more interested in why prg.smartadserver.com and pagead2.googlesyndication.com are so high, aren't you blocking ads?

[2097]

Any chance of running a function test on DNS ? via smoothwall.

[2097]

Ok did some playing this morning and did the following.

 

1.) Changed our DNS servers in smoothwall and our firewall to our ISP DNS servers instead of google's.

2.) GPO disabled the "News and Interests" Icon on all the computers ( we recently installed 120 new machines with the version of windows with it on )

 

Internet is running fine this morning, Very snappy.

 

So either this is a coincidence or its one of them two changes i have just made.

[ibpalle]

Primary DNS server latency results 8 DNS requests failed to respond.

42 DNS requests succeeded.

DNS resolution speed is a little slow or less than half the DNS requests were successful.

Average DNS request response time: 478.8381 ms.

 

Does not look completely healthy to me. What are your internal DNS servers using as forwarders? Or are they using root hints?

 

Take a look at this KB

https://kb.smoothwall.com/hc/en-us/articles/360003640159

especially the 'Optimize internal DNS servers' section because I think using the Smoothwall as a DNS forwarder for your internal AD DNS servers may help in this case.

[enjay]

Ok did some playing this morning and did the following.

 

1.) Changed our DNS servers in smoothwall and our firewall to our ISP DNS servers instead of google's.

2.) GPO disabled the "News and Interests" Icon on all the computers ( we recently installed 120 new machines with the version of windows with it on )

 

Internet is running fine this morning, Very snappy.

 

So either this is a coincidence or its one of them two changes i have just made.

 

DNS is the more likely, I'd be surprised if the "News and Interest" icon is generating much traffic.

 

We've got Google's 8.8.8.8 as our DNS, and don't have the problems you're having.

[TriggerHappyUK]

2.) GPO disabled the "News and Interests" Icon on all the computers ( we recently installed 120 new machines with the version of windows with it on )

 

We've tried doing this but it still shows up :-(

Do you mind sharing what GPO settings you used?

[simpsonj]

Does not look completely healthy to me. What are your internal DNS servers using as forwarders? Or are they using root hints?

 

Take a look at this KB

https://kb.smoothwall.com/hc/en-us/articles/360003640159

especially the 'Optimize internal DNS servers' section because I think using the Smoothwall as a DNS forwarder for your internal AD DNS servers may help in this case.

 

Sorry to jump on this thread, but on the DNS settings, should each reverse lookup zone be added to the domain area? Also, should anything be set as a Static DNS host?

[ibpalle]

Sorry to jump on this thread, but on the DNS settings, should each reverse lookup zone be added to the domain area? Also, should anything be set as a Static DNS host?

 

Reverse lookup is good to add - it gives you the ability to use hostnames in the locations. Apart from that, it's mainly important for Kerberos.

[2097]

Does not look completely healthy to me. What are your internal DNS servers using as forwarders? Or are they using root hints?

 

Take a look at this KB

https://kb.smoothwall.com/hc/en-us/articles/360003640159

especially the 'Optimize internal DNS servers' section because I think using the Smoothwall as a DNS forwarder for your internal AD DNS servers may help in this case.

 

Hi They are using root hints.

 

I did at the begining of the week do a support request with smoothwall, I can see someone did a remote login on "Tuesday" . But i havent had any correspondence since the intitial opening of the case .

 

Another Big change done recently ( early OCT) was the moving from our Linux based firewall to a Meraki MX .

 

Unsure if it can be related , as everything not going through the smoothwall seems fine ?

[simpsonj]

Reverse lookup is good to add - it gives you the ability to use hostnames in the locations. Apart from that, it's mainly important for Kerberos.

 

Thanks for that, I'll make that change.

 

I've got my DCs set as both Conditional DNS Forwarders and Static DNS Hosts. Wondering if I should remove them as static DNS hosts...

[2097]

After some thought process , Decided to roll back Smoothwall to our May 2022 version from backups ( Quite a few smoothwall versions behind )

 

Noticed a dramatic speed increase ! and is working as it should.

 

If anyone is noticing a problem on the latest version, go back before September, Possibly July