Route HTTPS Traffic to upstream proxy

[Cache]

Wondering if someone can help me see if what I'm thinking is right and what the way arround it is.

 

I've been asked to make available guest wi-fi access in the sports hall, which I've agreed to providing I can ensure that the security of the rest of the network stands. So I decided, VLAN, dedicated box with DHCP, DNS, firewall and proxying traffic would be a good way to try and make it as simple as possible for guests coming in to access it. Now I've hit a problem.

 

I can get Squid to obviously transparently route the http traffic and realise I can't do the same with https without doing a MITM, but the traffic somehow needs to get from my routing/proxy box upto the ISP's proxy.

 

Am I trying to do something really stupid/impossible in trying to send the HTTPS traffic straight up to the ISP's proxy, or should I be able to do it but missing the obvious?

 

If not, what's the best way to proxy the internet traffic to ensure it's simple, secure but supports other devices (I'm thinking Android type devices with no proxy support)

[tom_newton]

Without the box where you are doing your transparent proxying supporting transparent proxying of https connections you will not be able to do this - it is simply not possible to "convert" a non-proxied SSL connection into a proxied one without such a transparent proxy.

[Duke5A]

I just went through this with our guest wireless. We also have content filtering that is done by our ISP and is required by law to be used on the network. As you already know, you can't transparently proxy HTTPS without actually doing MITM, and even if you could get it working, I don't know if it would be legal. So I settled for a couple of compromises and set it up the best I can.

 

I setup a Squid 3.2 proxy server running on Ubuntu 10.04 LTS acting as a transparent proxy for HTTP, and let HTTPS route through it without being redirected to Squid. When first connecting using HTTP, Squid will redirect to a captive portal splash screen with the terms of usage for the network with an accept button at the bottom of the page. When accept is clicked it will take you to a page with directions on setting the browser up for a proxy so that HTTPS will work properly. If accept is not clicked, it'll simply keep redirecting you back to the TOS page. Finally, to keep people from being able to get out with HTTPS and bypass the acceptance page, I blocked all outgoing traffic from the guest VLAN on the firewall except for that destined to go to the offsite proxy at our ISP.

 

I did set the ISP's content filter as a cache peer in Squid, so even if the guest doesn't set proxy settings HTTP will still go out and up to the content filter. This at least gets some degree of web surfing to mobile devices. Bind was also setup on the Squid box so browsers that support WPAD would work with a single check box.

 

I hope this helps and I can provide examples from my configuration files if you want.

Edited March 1, 2012 by Duke5A

[Cache]

You know when you spend a couple of days on something and then your convinced it should do it? The more I thought about it last night the more I realised that trying to forward to the upstream proxy is no different from me trying to proxy it locally. Daft question after all.

 

If you could provide samples @Duke5A that would be great. I've read up how to create a captive portal on Squid, just not sure on your redirect and the redirecting to the instructions page.

 

Thanks

[Duke5A]

You know when you spend a couple of days on something and then your convinced it should do it? The more I thought about it last night the more I realised that trying to forward to the upstream proxy is no different from me trying to proxy it locally. Daft question after all.

 

If you could provide samples @Duke5A that would be great. I've read up how to create a captive portal on Squid, just not sure on your redirect and the redirecting to the instructions page.

 

Thanks

 

Sure, no problem! Getting Squid setup as a transparent proxy and tweaking iptables to foward all HTTP traffic to Squid on port 3128 was relatively easy with the help of a couple guides. What took me three days to figure out how to do was setup the session helper in Squid to keep track of both user sessions, and who accepted the terms and conditions. Here's a snippet of my Squid configuration file detailing how to do this.

 

external_acl_type session concurrency=100 ttl=3 %SRC /usr/lib/squid3/ext_session_acl -a -T 10800 -b /etc/squid3/session/

acl session_login external session LOGIN

external_acl_type session_active_def concurrency=100 ttl=3 %SRC /usr/lib/squid3/ext_session_acl -a -T 10800 -b /etc/squid3/session/

acl session_is_active external session_active_def
acl clicked_login_url urlpath_regex -i SplashAccepted.html

acl Bypass_Cache_Peer dstdomain .guestwifi.local
acl splash dstdomain .guestwifi.local

deny_info http://service.guestwifi.local/SplashRules.html session_is_active

http_access allow clicked_login_url session_login
http_access allow splash
http_access deny !session_is_active

cache_peer upstreamproxyaddressgoeshere parent 8080 0 no-query proxy-only
always_direct allow Bypass_Cache_Peer
never_direct allow all

 

The gist of this configuration is Squid uses an external helper ext_session_acl to keep track of currently connected sessions. When a client initially connects to Squid a session is created and entered into a DB using the IP address to identify it. Squid checks to see if the session has been marked active and will only allow HTTP traffic to pass once it has been. If not, you’re simply redirected to the acceptance page SplashRules.html over and over again. The way to get your session marked active is to click the acceptance link at the bottom of the page linking to SplashAccepted.html. Once Squid sees you’ve browsed to a URL containing that string it instructs the helper to mark the session active.

 

The last three lines of configuration forward all HTTP traffic up to the content filter at our ISP. For HTTPS, there are directions on the SplashAccepted.html page detailing how to add the proxy settings to popular browsers. I also setup WPAD for auto proxy configuration. Even if users don’t go through the added steps, they’ll still get filtered HTTP. HTTPS seems to be working for iOS and Android if the users follow the directions.

 

What tripped me up for three days was the helper wasn’t behaving like it should as detailed in the documentation. It turns out that there is a bug that prevents it from accurately keeping track of sessions when used in active mode. From what little I found on the issue, you need version 1.2 of the helper to get around this. The fixed version is included in Squid 3.2, but the latest version available in the Ubuntu repositories is 3.0. It was recommended that I start with a clean install and compile the latest Squid from scratch. I wasn’t totally enthused to set Bind, DHCPD, Squid, Apache, and a host of other services back up, so I compiled Squid 3.2 on a new clean install and simply copied the new version of the helper over. It worked without any hitches.

 

I hope this helps. If you need anything else, or snippets from my iptables rules or such, don’t hesitate to ask.

[Cache]

Brilliant, thanks for the help!

 

Will have another stab at it when I get chance next week.

[Duke5A]

I received a request for a copy of a compiled version of the session helper so I'm attaching it to this thread. This is version 1.2 and was compiled on an x86 Ubuntu install.

 

ext_session_acl.zip

 

Be sure to set execute permissions on this file when you place it.

[Cache]

Thanks @Duke5A - I did actually manage to compile my own on a virtual machine and got it up and running.

 

I encountered some really weird errors with it though - sometimes the session would start, sometimes the session started on one computer but wouldn't on 3 others I was testing.

 

I do have now though a fully working captive portal with access only on a single vlan!

 

Couldn't have done it without your advice though.

[Duke5A]

Hey @Cache - No problem! Glad I could help.