Netsweeper (Lancs) Allowing Unfiltered Bing Image Search

[Strawdog]

Just a heads up for other potentially affected schools - our (Lancs) Netsweeper isn't currently enforcing safe search (Strict) on Bing, it's defaulting to Moderate and students can toggle it to Off allowing unrestricted access to lots of hardcore porn.

 

You can create a rule/configure your network to map Bing to strict.bing.com to enforce safe search (https://support.microsoft.com/en-au/topic/block-adult-content-with-safesearch-946059ed-992b-46a0-944a-28e8fb8f1814) but Netsweeper should be doing this automatically and frankly the easier option is to just block Bing across the board as I remember having exactly the same problem with Lightspeed back in the day.

[Strawdog]

As an added bonus it also appears as though searches performed within Bing Image search aren't being recorded/flagged in the 'search queries' report (searches performed on the main Bing homepage are recorded but not the image search page).

[PaddyNewman]

Are they actually decrypting Bing? Its classified as a search engine (or should be) and would need to be decrypted fully unless you DNS bodge it. We amend it in the core because education, might as well lock the door, but if its set to search engine and safe search+search keywords are denied as categories, then that 'should' do that. Just checked my side and its listed as;

Category Decision: Search Engine, Hypertext Transfer SecureAllowed

 

so unless its changed on your end?

[Strawdog]

Not made any change to its category at this end but a colleague in another local school is seeing different behaviour to so it's likely a setting somewhere in NetSweeper that's allowing it through.

 

We see all search queries for Google (incl. image search) and the Bing homepage, just not seeing Bing image searches recorded.

[PaddyNewman]

Do you see the full string in a report, so https://www.bing.com/12345/abc123? Also is the cert on Bing showing as the Netsweeper/your SSL one or is your browser now bypassing HTTPS.

[Strawdog]

Running the builtin Full Search Query Report(v1) report only shows the search phrase, not the full URL but you can see entries for Google, Google Image search and Bing but I don't see any entries for the test Bing Image searches I did

 

Seeing our (BTLS) SSL certificate on Google/Google Image and Bing/Bing Image.

 

....and just to make me spend hours chasing my own tail I'm now seeing the NetSweeper block page when searching for 'porn' in Bing Images despite it happily serving up all the pics yesterday on multiple machines/OS... I've not made any changes to policies, etc. beyond blocking Bing yesterday and then enabling it again this morning to do more testing.

Edited May 6, 2022 by Strawdog

[PaddyNewman]

Check the certificate on the bing images page? Is it definitely the one you expect?

 

What browser are you using / do you have the option of using Chrome and amending some flags?

[Strawdog]

Using Firefox and Safari, see edited post above re. cert.

 

Test machine is now blocking and enforcing Safe Search correctly on Bing :\

 

There's a machine a student was using yesterday that they had disabled Safe Search and accessed porn on - will jump on a remote session and see if that is still allowing it.

Edited May 6, 2022 by Strawdog

[Strawdog]

After some more testing it appears Bing Safe Search is reliant upon it using the BTLS certificate, Google isn't - clients not using the BTLS certificate to access Bing can toggle Safe Search to Off and it stays Off, clients using the certificate can still toggle it to Off but it reverts back to Strict when you go back in (the Settings menu shows it as Off but it's not, the initial drop down menu just doesn't refresh the entry when it reverts back to Strict).

 

Good to know, now just need to block all clients (e.g. guests) who don't have the BTLS certificate installed from accessing Bing which is easy enough.

 

Still don't know why I was seeing different behaviour yesterday - both the affected clients (student machine and my test machine) should have definitely had the certificate (it's deployed via GPO and both machines are regularly used) but I didn't check it at the time so can't say for definite...

[PaddyNewman]

That sounds a little fishy to be honest! Be good if you can replicate. I've only seen things walk past Netsweeper when either QUIC was used or the decrypt scheme wasn't in place, but sounds like it is as you can get results from other search engines. The fact you see nothing for the Bing images which are from bing.net I think, screams filter bypassing to me...

[Strawdog]

I could replicate it on a test machine though so not a filter bypass site - I was thinking it was probably the certificate missing on the clients as this would explain both being able to turn Safe Search off and the search queries not being logged but I just checked the report again and my test client has some entries for some earlier Google search queries so the certificate must have been present.

 

I suspect that report (Full Search Query Report(v1)) probably just doesn't record Bing Image searches - whether they're still recorded elsewhere or not I'll find out when I get my weekly suspicious search report and it's filled with me searching for 'porn'

[PaddyNewman]

Sorry, my filter bypass is not the netsweeper one, I'd never touch that list, its just bypassing filtering in general. Missing certs should give you errors, if you aren't getting errors then its using the normal cert assigned which usually means its bypassing decryption or not using TCP 443. Do you have a global deny on UDP 80 and 443 on your firewall? I would also block https://dns.google within Netsweeper and deny 8.8.8.8 and 8.8.4.4 on TCP 443 on the firewall, just because its an easy way to walk round Netsweeper.

[Strawdog]

We're on the LEA broadband bundle so I don't have firewall access (beyond individual client firewalls via GPO, etc.)

[craigcamacho]

We have had Bing.com blocked on all our Netsweeper policies since the day it was implemented for this very reason and it is still working for us. Is it possible Students are bringing in portable editions of VPN apps on USB drives? We had the exact same scenario a number of years ago and we ended up blocking the use of external drives for students. Also is it possible the Students have discovered a proxy site that for whatever Netsweeper isn't categorising?

[PaddyNewman]

Oh! I would hope its blocked at that level as QUIC is a very nasty way to bypass filtering! If your WiFi can add rules, it may be worth adding that and potentially on client devices outbound UDP 443 as denied, thats all I feel could be assisting the image and searches to get through!

[Strawdog]

Yep we block student USBs for the same reason but it's not a VPN/proxy (I replicated the issue on a fresh machine with both student and staff profiles yesterday - today it's now working as it should be...), it just appears that if the client isn't using the BTLS SSL certificate for Bing then you can toggle Safe Search Off and without the certificate you also have no visibility of the search query so it allow free reign to access lots of porn (no blocking based on search phrase as it can't decrypt it and no enforced Strict Safe Search).

[PaddyNewman]

This sounds like QUIC, if its not got the certificate that you are signing with and shows the normal MS one, which apparently today is the Microsoft RSA one, then its bypassing filtering and thats a firewall problem which probably needs nipping ASAP as its going to become a huge problem. Can probably check by running wireshark on a machine that gets the ability to access stuff without the certificate and see if its connecting via TCP or UDP.

[craigcamacho]

To help me understand the problem a bit more is there a reason or scenario why a device wouldn't be using the BTLS cert? We have a GPO at the top of our forest which deploys the cert and sets the necessary reg keys and with our schools being from a similar background would have thought yours would be the same. The only devices that don't get the cert are Intune managed iPads which join a different SSID to the main school laptops but we have a separate Netsweeper policy for that and add in the shared lists etc. If it didn't work yesterday but is today is it possible Netsweeper was just having a moment? (wouldn't be the first time)

[Strawdog]

All devices get the cert apart from guests (e.g. NHS nurses, guest speakers, etc.), which are segregated from the main network and have pretty much every potentially dodgy category blocked/shared lists applied, and yep we deploy the cert in the same way.

 

Blocking Google for guests, despite no SSL inspection (due to no cert), is impractical but I could live with it as Safe Search is enforced whether they have the cert or not - I was under the impression Bing worked in the same way but apparently Bing requires the cert to enforce Safe Search and without you can toggle it to Off, that coupled with no keyword blocking (as no cert to see what's being searched for) means guests potentially have access to porn if Bing is not blocked altogether for them.

 

Suspect yesterday was a NetSweeper wobble as I can't replicate the issue today but I swear I have witnesses and it has served to highlight this issue

Edited May 6, 2022 by Strawdog

[Strawdog]

Done some more testing and it appears that clients not only need the certificate but also need the Selective Decryption list applied to their group otherwise Safe Search can be toggled off in Bing.

 

Also found blocking search engines (Bing, Yahoo, etc.) in Policy->Categories doesn't appear to work (with or without the certificate) unless you have the Selective Decryption list applied to the group in question, this might just be a delay with the policy application as Twitter, Facebook, etc. are blocked by Category and are inaccessible but I think it's deeper than that... Blocking search engines in the Group's Local List (URL/Keyword Local List), via their URL, is blocking them correctly so not sure why Category blocks for the search engines aren't working (changes to the Local List also update pretty quick on the device).

 

Also discovered DuckDuckGo happily lets you toggle off Safe Search with or without the certificate and/or wagent which then allows access to a ton of porn, via an image search, unless you have the certificate and Selective Decryption list applied to the group the device sits in (which filters the search based on the keyword list so although it doesn't actually enforce Safe Search it does block the majority of illicit content).

 

Upshot: -

 

Make sure all your groups, whether they have the wagent or not (i.e. iOS/MacOS clients), have the Selective Decryption list applied if they have (or should have) the certificate on them.

If they don't have the certificate on them (e.g. guest devices) block pretty much every single thing you can.

Block Bing and DuckDuckGo for anyone without the certificate and wagent (e.g. guest devices) and consider blocking them for everyone.

Edited May 13, 2022 by Strawdog

[PaddyNewman]

I'd be keen to actually see what is configured as Safe Search should be enforced, as long as its in the denied category list.

 

It sounds like something is misconfigured as the problems you have are simply resolved with Netsweeper, especially category stuff..

[Strawdog]

If you fancy jumping on a Teams call sometime next week I'd be happy to demo, would be nice to have a second pair of eyes on it as it feels a bit creaky to me and I'm tying myself in knots testing a million different variables (OS, VLAN, cert/no cert, wagent/no wagent, browser, user permissions, etc.).

 

The main issues we're having is with non-Windows clients (so no wagent as the LA haven't made one available for anything apart from Windows) of which we have no shortage (iOS, MacOS, Android, Smart TVs, etc.).

Edited May 13, 2022 by Strawdog

[PaddyNewman]

I'll drop you a PM