Smoothwall mangling URL's - Big Sur Macs.

[educontractor]

Hello,

 

We have a strange problem. Replaced a bunch of older Mac minis running El Capitan with some M1 based ones running Big Sur.

 

They have the intermediate SSL certificate installed, and trusted.

 

They are using a Proxy configuration that allows Kerberos / NTLM authentication, but falls back to a sane group - Secondary Students for a device that doesn't authenticate. Since the PERL script that authenticates Macs isn't compatible with Big Sur, we assumed this would be fine.

 

The browsing behaviour is strange, though. Some sites e.g. Google, BBC load but the images are blocked. Looking at realtime monitoring of the Web Filter for these sites shows that attempts are being made to download the images through their respective CDN but these requests aren't blocked.

 

Weirder still is that other sites have their URL rewritten, naturally enough resulting in a 504 error as the proxy can't display them.

 

e.g. https://www.yahoo.com is rewritten to http://smoothwall:813/?https://www.yahoo.com/

 

There isn't a content modification rule that is being applied so does anyone have an inkling as to what's happening here?

[ibpalle]

The http://smoothwall:813/?https://www.yahoo.com/ is a redirect to port 813 on Smoothwall where the negotiate Kerberos/NTLM is performed. If you are getting a 504 on that, then if you are using proxy settings, please make sure the option for bypass proxy for local addresses is enabled along with exceptions for your internal subnets.

 

As an aside, it may be an idea to contact your account manager and setup a call to evaluate the auth methods being used. We have some new methods available that is far better suited than Kerberos/NTLM these days.

Edited October 1, 2021 by ibpalle

[educontractor]

Thank you, that's very helpful.

 

I did try enabling a few different proxy ports with different auth methods including a redirect to a login page, but continued to get the above behaviour. Likewise the port/proxy configured for Core Authentication shows the same behaviour...

 

Just to confirm - is the bypass you describe the one set on the clients?

[ibpalle]

The exceptions should be set on the clients - if the clients are using proxy settings.

 

If you are using the proxy port for set for core auth, this should not happen - if the transparent proxy is using negotiate kerberos/ntlm then the proxy settings may not be used correctly by the client.

[educontractor]

Again, thank you for this.

 

I had thought that I had configured it this way, but perhaps I've made a mistake.

 

I wonder if you'd be so kind as to cast your eye over the attached screenshots, and see if I'm missing anything obvious?

 

 

 

[ibpalle]

Enable the option 'Exclude simple hostnames' please. That is the equivalent to 'Do not proxy for local addresses' in the Windows settings. Basically it excludes any request using just a hostname (like 'smoothwall' ) from going via the proxy.

[educontractor]

Enable the option 'Exclude simple hostnames' please. That is the equivalent to 'Do not proxy for local addresses' in the Windows settings. Basically it excludes any request using just a hostname (like 'smoothwall' ) from going via the proxy.

 

Thank you again for your ongoing assistance - I really appreciate it.

 

My heart leapt when I read your post - it seemed such a logical solution but, alas, the issue remains (see screenshot). I have requested a ticket with Smoothwall, but I feel that it must be something simple that I'm missing. After all, hundreds of PC's and iPads are concurrently browsing the internet with any issue whatsoever.

 

[ibpalle]

Can that client actually resolve the smoothwall hostname? Try a ping of just smoothwall on a command line console.

[bald_pig]

Did you set the "exclude simple hostnames" for both the http and https proxy?

[educontractor]

Can that client actually resolve the smoothwall hostname? Try a ping of just smoothwall on a command line console.

 

It can, but possibly only because it's appending the DNS suffix. I'll try adding the simple hostname to the HOSTS file and see if that makes a difference.

 

Apologies for all the ludicrous McCarthyite redactions on the screenshots as well. If it was my site / data I'd be happy to post unedited, but as I'm a contractor I'm being extra cautious.

 

[educontractor]

Did you set the "exclude simple hostnames" for both the http and https proxy?

I did, yes. I checked it to be sure and it's definitely ticked for both options.

[educontractor]

For completeness:

 

Adding the simple hostname to HOSTS allowed it to PING without appending the DNS suffix, but the browsing behaviour is identical.

 

Why doesn't it prepend all URL's with smoothwall:813/? Certain ones like the BBC and Microsoft part-load just without images (when I look at the live reporting for my IP I can see calls to the CDN's have the smoothwall:813 added to the front of them, hence why the images aren't loading)

[ibpalle]

The system should append the search domains to the simple hostnames in order to be able to resolve them, so that is the correct behaviour.

 

Can the client navigate to http://smoothwall/getcert?

[educontractor]

The system should append the search domains to the simple hostnames in order to be able to resolve them, so that is the correct behaviour.

 

Can the client navigate to http://smoothwall/getcert?

Seemingly not. The kerberos bit even gets added onto this, which the proxy then tries to resolve. It's as though the OS is sending it to the proxy whether I like it or not, i.e. ignoring the proxy exceptions and the "ignore simple hostnames" part.

 

[ibpalle]

It does look like that - not certain how to address that

[educontractor]

It does look like that - not certain how to address that

 

Just a thought, and I don't know if this is even possible, but if I were to assign the Macs IP addresses within a specific range, could the Smoothwall be configured to assume authentication to a particular group based on the source IP?

 

Alternatively, is there any way of stopping the kerberos pre-pend being added even when no proxy is set?

[ibpalle]

Create a location object in Guardian - policy objects - locations and setup an additional auth policy for the proxy with that location set in the where field. Set auth to be none and set the unauthenticated requests to be treatred as any group you would like in the third step.

[educontractor]

Hi guys,

 

Just reporting back that we've solved it. The issue wasn't actually with Smoothwall - it was a combination of JAMF and Mac OS.

 

So we knew that we were pushing proxy settings from JAMF, but I'd assumed (wrongly) that the fact we could enter proxy settings manually when logged in as an admin would override these. Put simply - it doesn't!

 

This explained why (a) the authentication precursor was being passed to the proxy (JAMF didn't have the exceptions right) and (b) why changing the proxy settings to different proxies with different authentication types made no difference whatsoever!

 

With the correct exceptions and using a proxy with Core Authentication the issue is entirely resolved.

 

Thank you all for your valuable assistance!