Why is Generative AI not blocked by default on LGFL ?

[xicor]

does anyone know why LGFL has not blocked Generative AI sites by default ? especially considering the high risk of adult content there ?

[mavhc]

Where is there a high risk of adult content?

[Cat_Jam148]

While there are concerns when it comes to AI content, I don't think there is a need for it to be blocked by default. We allow it in certain situations and environments.

 

If there is a site serving adult AI content, then it should already be blocked under the various other adult filters LGFL has.

[xicor]

While there are concerns when it comes to AI content, I don't think there is a need for it to be blocked by default. We allow it in certain situations and environments.

 

If there is a site serving adult AI content, then it should already be blocked under the various other adult filters LGFL has.

 

in this case LGFL did not block AI sites that can be used to generate adult content, hence wondering why its not blocked by default and leave the specific needed ones to be unblocked manually ?

Edited November 11, 2024 by xicor

[xicor]

Where is there a high risk of adult content?

 

 

not sure what you mean, do you mean which sites or how its risky ?

Edited November 11, 2024 by xicor

[mavhc]

Your question implies that they're all the same, and thus should all be blocked, is that the case, or should the ones that generate adult content be categorised as adult sites?

[xicor]

Your question implies that they're all the same, and thus should all be blocked, is that the case, or should the ones that generate adult content be categorised as adult sites?

 

I think the default is that they should all be blocked and only vetted or business ones to be unblocked by default I don't understand the logic of allowing any ai site without it being vetted

 

For example craiyon was unblocked on LGFL despite it having adult content and it was classified as a technology site until i reported, and this is where I don't understand how it was allowed to be accessible in the first place ? and that's only one example and I tested others and I get the same results, how can this be ?

[synaesthesia]

Wouldn't you also block Google for the same thing? That raises more questions about your filtering than the site itself (I've not been able to get craiyon to generate anything worse than a Google image search provides)

[LGfLMarkB]

Hi xicor

 

Mark from LGfL here. The reason is quite simply that we try not to make sweeping changes that will affect everyone and Gen AI is *relatively* new and we've only had it for about a year.

 

Key things to note:

-It is a BUNDLE, not a category. Regularly updated

-FYI school approaches differ massively regarding appetite for using AI, which takes me back to the first line above

-Important - our recommendation is that schools block Gen AI and then allow the sites they choose to use and which have been risk assessed - again this will be very different per school

-We have pointed this out in training and newsletters to schools etc and also in our templates for reviewing this in the regular checks and reviews you need to do to comply with the DfE standads

-You really need to be cautious if using Gen AI sites, both in terms of age limit in the Ts and Cs - some are 13, some 16 and some even 18 so should not be used. Further, the output can be highly inapporpriate so think about mitigations..

-Filtering is always going to be limited when it comes to AI which is why *monitoring* is key here

-Note also the DfE guidance on AI use in schools

 

Hope that helps.

Best,

Mark

[DrCheese]

in this case LGFL did not block AI sites that can be used to generate adult content, hence wondering why its not blocked by default and leave the specific needed ones to be unblocked manually ?

 

Paint can be used to draw pornographic images. Word can be used to write pornographic stories.

We don't block those because they are simply tools, Ai is the same.

 

I wouldn't want an ISP to slap AI in the adult content category and call it a day.

[mavhc]

I think the default is that they should all be blocked and only vetted or business ones to be unblocked by default I don't understand the logic of allowing any ai site without it being vetted

 

For example craiyon was unblocked on LGFL despite it having adult content and it was classified as a technology site until i reported, and this is where I don't understand how it was allowed to be accessible in the first place ? and that's only one example and I tested others and I get the same results, how can this be ?

 

I don't see any adult content on Craiyon, it says it's designed to not create it

[xicor]

Wouldn't you also block Google for the same thing? That raises more questions about your filtering than the site itself (I've not been able to get craiyon to generate anything worse than a Google image search provides)

 

what questions does this raise about the web filtering, what would you suggest that needs checking ? I did not get any adult content on google image search as its set to safe search by default - it was a different story for craiyon

[TechMonkey]

Craiyon isn't one of these services like YouTube ads, that shows content you have searched for, is it?

Maybe the user that found lots of adult content had been slightly overly vigorous in their checking for adult content or in their background research?

 

I too haven't seen any adult content on there but haven't spent too long on the site.

[xicor]

Paint can be used to draw pornographic images. Word can be used to write pornographic stories.

We don't block those because they are simply tools, Ai is the same.

 

I wouldn't want an ISP to slap AI in the adult content category and call it a day.

 

nobody said bundle it and call it a day, but if an ai site generates adult content with a simple prompt it should be blocked, I don't understand your analogy with word or paint its completely irrelevant and unrelated to web filtering by your logic a pencil can do the same

[xicor]

Hi xicor

 

Mark from LGfL here. The reason is quite simply that we try not to make sweeping changes that will affect everyone and Gen AI is *relatively* new and we've only had it for about a year.

 

Key things to note:

-It is a BUNDLE, not a category. Regularly updated

-FYI school approaches differ massively regarding appetite for using AI, which takes me back to the first line above

-Important - our recommendation is that schools block Gen AI and then allow the sites they choose to use and which have been risk assessed - again this will be very different per school

-We have pointed this out in training and newsletters to schools etc and also in our templates for reviewing this in the regular checks and reviews you need to do to comply with the DfE standads

-You really need to be cautious if using Gen AI sites, both in terms of age limit in the Ts and Cs - some are 13, some 16 and some even 18 so should not be used. Further, the output can be highly inapporpriate so think about mitigations..

-Filtering is always going to be limited when it comes to AI which is why *monitoring* is key here

-Note also the DfE guidance on AI use in schools

 

Hope that helps.

Best,

Mark

 

Hi Mark

 

is the list of urls for that bundle published anywhere ?

[Koldov]

Can we be clear on what the issue is here...?

 

User visits AI site and types in "draw me some p0rn", AI site obliges... p0rn image is created...

 

Is this what is happening?

 

Can this be stopped?

[LGfLMarkB]

Hi Xicor,

I'm sorry it isn't and we can't share it as it's Netsweeper IP (it stems from their categorisation engine and is a rather long list).

Sorry about that but what are you trying to achieve? Maybe we can help you do so in another way? Maybe drop an email to [email protected] if it's specific to your setup though.

TY

Mark

[xicor]

Craiyon isn't one of these services like YouTube ads, that shows content you have searched for, is it?

Maybe the user that found lots of adult content had been slightly overly vigorous in their checking for adult content or in their background research?

 

I too haven't seen any adult content on there but haven't spent too long on the site.

I can only tell you based on what I saw and it did not take alot to get to those images even from simple prompts - openart is another example

[xicor]

Hi Xicor,

I'm sorry it isn't and we can't share it as it's Netsweeper IP (it stems from their categorisation engine and is a rather long list).

Sorry about that but what are you trying to achieve? Maybe we can help you do so in another way? Maybe drop an email to [email protected] if it's specific to your setup though.

TY

Mark

I emailed you just now, would be good if we can have a call if you're free today ?

[DrCheese]

by your logic a pencil can do the same

 

That's my point tho (using your logic), you're saying that Ai should be automatically blocked by default because it can be misused.

[tom_newton]

Early on in the lifecycle of search, we absolutely did used to block all search, and then just allow the ones we "trust" to have decent safesearch! The only thing thats rendered that moot is that running a search engine is nigh on impossible these days, so everything comes back to one of the big 2, and applies their safesearch (Which has its limitations, sure, but it's what we got).

I suspect we will see a similar trajectory in AI, as the same market forces apply. For now, only allowing the tools you've consciously invited into your school seems prudent to me.

[xicor]

Early on in the lifecycle of search, we absolutely did used to block all search, and then just allow the ones we "trust" to have decent safesearch! The only thing thats rendered that moot is that running a search engine is nigh on impossible these days, so everything comes back to one of the big 2, and applies their safesearch (Which has its limitations, sure, but it's what we got).

I suspect we will see a similar trajectory in AI, as the same market forces apply. For now, only allowing the tools you've consciously invited into your school seems prudent to me.

 

to be honest I'm quite shocked of the content I'm seeing and baffled how I can spend an hour and identify dozens of adult focused ai sites all of which have the wrong categories on LGFL and many completely unblocked, surely a mix of smart scripting and a dedicated team can pick this stuff up and many more and deploy counter measures or re-categorise those sites, its baffling tbh

[xicor]

Can we be clear on what the issue is here...?

 

User visits AI site and types in "draw me some p0rn", AI site obliges... p0rn image is created...

 

Is this what is happening?

 

Can this be stopped?

from what i can see that is harder than simply searching the ai site for the images already made by others, I just found way more sites and most/all have the same theme

[TechMonkey]

to be honest I'm quite shocked of the content I'm seeing and baffled how I can spend an hour and identify dozens of adult focused ai sites all of which have the wrong categories on LGFL and many completely unblocked, surely a mix of smart scripting and a dedicated team can pick this stuff up and many more and deploy counter measures or re-categorise those sites, its baffling tbh

 

Can I ask what you are counting as adult focused or adult content though. You keep saying craiyon is an adult site or contains adult content, but others aren't seeing it. So, could it be what you are considering adult content, others of us aren't as concerned about? Can you describe, or post screenshots, suitably censored if needed? Are we talking nudity? Adult themes (gambling, guns, violence)?

 

I've looked at recent and inspirations and not seen anything that would make me double take. Searching for "boobs" on the search page gets a little closer to the bone, but I would hope your filtering catches easy searches like that as it is in the URL.

[sigma]

from what i can see that is harder than simply searching the ai site for the images already made by others, I just found way more sites and most/all have the same theme

 

We also use Netsweeper, and have been through the same loop and keep going through the same loop. I don't think that Netsweeper is particularly good at managing blocking Pornography. We get so many false positives that have to be checked, yet our ISP's (Not LGFL) "out of the box" configuration does allow access to pornography, even though it's category blocked.

 

The latest DfE Policy Paper I can find is:

 

https://www.gov.uk/government/publications/generative-artificial-intelligence-in-education/generative-artificial-intelligence-ai-in-education#opportunities-for-the-education-sector

 

The bit we are mindful of is:

 

Protecting data, pupils and staff

 

Generative AI:

 

stores and learns from the data it is given – any data entered should not be identifiable

can create believable content, including more credible scam emails requesting payment – people interact with generative AI differently and the content may seem more authoritative and believable

 

Schools and colleges should:

 

protect personal and special category data in accordance with data protection legislation

not allow or cause intellectual property, including pupils’ work, to be used to train generative AI models, without appropriate consent or exemption to copyright

review and strengthen their cyber security by referring to the cyber standards – generative AI could increase the sophistication and credibility of attacks

ensure that children and young people are not accessing or creating harmful or inappropriate content online, including through generative AI - keeping children safe in education provides schools and colleges with information on:

what they need to do to protect pupils and students online

how they can limit children’s exposure to risks from the school’s or college’s IT system

refer to the filtering and monitoring standard to make sure they have the appropriate systems in place

 

Which leads back to:

 

KCSIE, and the four areas of risk:

content: being exposed to illegal, inappropriate, or harmful content, for example:

pornography, fake news, racism, misogyny, self-harm, suicide, anti-Semitism,

radicalisation, and extremism.

contact: being subjected to harmful online interaction with other users; for

example: peer to peer pressure, commercial advertising and adults posing as

children or young adults with the intention to groom or exploit them for sexual,

criminal, financial or other purposes.

conduct: online behaviour that increases the likelihood of, or causes, harm; for

example, making, sending and receiving explicit images (e.g. consensual and non-

consensual sharing of nudes and semi-nudes and/or pornography, sharing other

explicit images and online bullying, and

commerce: risks such as online gambling, inappropriate advertising, phishing and

or financial scams.

 

Just because its generative AI (new and shiny), it doesn’t mean the statutory guidance doesn’t apply with regards to both blocking and over-blocking.

 

Netsweeper classified this as porn last week. It had me wondering if its location aware?

 

 

Oddly, the mag was featured on HIGNFY this week.