[Ubuntu] Internet Filter is Sloooooooow...

[Tyiell]

Hi all,

I'm pretty sure this is a Linux problem rather than a filter problem so I've put this here for now...

Basically I have, at length and with the help of serveral guides written by people who know far more about linux that me, built an Intenet Filter using squid and dansguardian running on Ubuntu (desktop - not the server edition). It works fine, we use a little program on the client called smoothwallIDT (no relation to the smoothwall products themselve I'm told) that basically wraps the username and computer details in the IDENT protocal. DansGuardian can then read it and give appropriate group access (banned, moderate filtering, heavy filtering, unbanned).

The trouble has been that the idt program sometimes doesn't load on boot and so users are not getting internet. So this time I built the filter, bound it to the domain using winbind, samba, kerberos, squid and dans, and with much fiddling have fudged it to authenticate directly from Active Directory using NTLM.

All well and good - this does actually work. But it is unusably slow. It never takes less than 5 minutes to load google. Yahoo is timed at 9 minutes. The filter is designed to authenticate using either IDENT or NTLM, so if the idt program is running it uses that intead. In that case, it works fine and loads in seconds.

Now admittedly I'm not running this system on a proper server, just a desktop workstation. But it is a brand new desktop, an HP dc5800, C2D 3GHz, 2 GB ram, etc. And it is just me on my lone computer having my internet filtered to test it, rather than the whole school. So I don't think it has any excuse to be this slow!!

As I understand it, ntlm_auth uses a handshake protocol to authenticate, but nothing I have read implies that it should be this slow. If I swap over and make people use this, they will thing that we just downgraded to dial-up modems

Any thoughts on what I could do to speed things up?

Many thanks!!

[webman]

Anything out of the ordinary in the log files?

/var/log/messages
/var/log/squid/ - cache.log, error.log (or whatever else is there)

Have you tried switching to just NTLM and getting rid of IDENT? Do you have IPv6 enabled? Having IPv6 enabled on IPv4-only networks can sometimes cause networking delays.

EDIT: And how many child processes do you use for the NTLM auth process? I remember a long time ago we made this a large amount as we thought it would be better; but 5 was sufficient.

[Jamo]

Weird! I wouldn't think its the speed, our proxy is an old RM FSeries with about 1GB Ram and a P4 2.4..... its shocking but delivers the web fast enough!!! We use IDENT authentication, its fails sometimes which is annoying but most of the time it is reliable.

Use wireshark to look for collisions or unneccesary traffic from the box.

[Tyiell]

Anything out of the ordinary in the log files?

/var/log/messages
/var/log/squid/ - cache.log, error.log (or whatever else is there)

Have you tried switching to just NTLM and getting rid of IDENT? Do you have IPv6 enabled? Having IPv6 enabled on IPv4-only networks can sometimes cause networking delays.

EDIT: And how many child processes do you use for the NTLM auth process? I remember a long time ago we made this a large amount as we thought it would be better; but 5 was sufficient.

I have 5 child processes running. I haven't intentionally set IPv6 to enabled - is that a setting in squid, or in the networking on the machine?

I'll try it without IDENT next and let you know how it does.

Cheers!

[Tyiell]

Weird! I wouldn't think its the speed, our proxy is an old RM FSeries with about 1GB Ram and a P4 2.4..... its shocking but delivers the web fast enough!!! We use IDENT authentication, its fails sometimes which is annoying but most of the time it is reliable.

Use wireshark to look for collisions or unneccesary traffic from the box.

Good thought - I'll give that a shot!

[webman]

I believe disabling IPv6 varies between versions. Google for "Ubuntu <version name> disable ipv6".

But it will be similar to the procedures outlined in these:

How-To: Disable IPV6 to speed up Internet. [Archive] - Ubuntu Forums
How to Disable IPV6 in Ubuntu|Ubuntu Geek

[Tyiell]

Hmm. It seems to have stopped working entirely now...

Earlier this morning when it was working I didn't have anything helpfull in the logs. Now I have errors in cache log as follows:

utils/ntlm_auth.c:173(get_winbind_domain) could not obtain winbind domain name!
WARNING: ntlmauthenticator #2 (FD 8) exited
WARNING: ntlmauthenticator #5 (FD 11) exited
WARNING: ntlmauthenticator #1 (FD 12) exited
WARNING: ntlmauthenticator #3 (FD 14) exited

So it has stopped authenticating. All I've changed is disabled IPv6 (which we don't use, so that shouldn't be it) and disabled the IDENT. Which I have now turned back on, but it doesn't fix it. How weird!!

Once I've beaten it until it works again, I'll carry on with your suggestions, but if this post goes dead for half a day or so, you know it's being stubborn!

Thanks for your help so far!!

[webman]

Can the box still communicate with the internet and the Windows servers? Do things like wbinfo -u, wbinfo -t etc from the terminal.

[Tyiell]

Can the box still communicate with the internet and the Windows servers? Do things like wbinfo -u, wbinfo -t etc from the terminal.

wbinfo -u/-t both fail. The server can still see the internet itself, and ping the dc. The computer connected tot he internet through it is prompted for a windows dmoain-style login box, but rejects a correct login until I get the a "squid cache Access Denied" page.

Not sure what I did to it, but it's not happy

[webman]

Perhaps it's grumbling about the IPv6 changes - just for curiosity's sake, un-do the disabling you did before to see if that helps it.

Again, there could be something in the logs - likely to be in /var/log/samba.

[tom_newton]

It may be that your Ad is taking too long to respond to queries? I know we cache auth results to keep things nippy, but then I have heard tell of setups like you describe working for smaller numbers of users.

[Tyiell]

Hi all,

Thanks for your help yesterday! It is all fixed now, but as is so often the case when I work in Linux, I'm not entirely sure what I did to fix it!! I just get frustrated until I just start changing things that look in some way suspicious until I either kill it completely or fix it. Guess I got lucky this time!

Seriously though, one of the reasons may well be the removal of Ident from the plugins as webman suggested straight out - I had tried this yesterday and had thought it hadn't worked, but as it transpired the system had already mysteriously died by that point.

In case anyone else stumbles upon this later, it is also worth noting that the permissions on /var/run/samba/winbindd_priviledged are set wrong everytime the service restarts - it needs the group to be "proxy" and the group needs write access. I'll have to work on a script to change that automatically on the service restart, but for now I'm busy being chuffed that my filter works and authenticates from a windows domain!! Beers for me tonight!!

Thanks again for all your suggestions!

[webman]

You're welcome.

It is all fixed now, but as is so often the case when I work in Linux, I'm not entirely sure what I did to fix it!!

Sounds strangely familiar!