For me settings the GPO delete cached copies of roaming profiles to not configured and set the Delete user profiles older than a specified number of days on system restart to 1 day seems to have fixed it.
I tried removing the users from security groups, but that didn't make any difference for me!