VLAN gotchas

[gshaw]

Just looking for any lessons you've learnt from putting VLANs onto a previously flat network?

 

We're looking to do it in summer and so far I've thought of...

 

• Turn on IP routing on core switch
  
• DHCP helper on switch to point clients to DHCP server (can I put multiple helpers if I have split scope DHCP servers?)
  
• set default gateway on core switch to point to firewall LAN address
  
• new scopes on DHCP server for each VLAN
  
• set default gateway option on DHCP to point to gateway of the VLAN for each scope
  
• reconfigure firewall with a static route so it can still port forward to servers in a separate VLAN for web services etc
  
• reconfigure printers into new VLAN and map via GP Preferences
  
• ensure uplink ports get tagged for all VLANs going to the edge switches and matching ports on core switch (I'm talking ProCurve speak btw)
  
• ensure documentation is up to date showing which ports are going where on our edge switches when untagging ports to a VLAN i.e. so machines go in the right VLAN!
  
• check switch ACLs are working as expected
  

 

Did you encounter any unexpected issues when doing your VLAN projects and what lessons did you learn?

Edited March 9, 2011 by gshaw

[maark]

make sure everything is running ok after vlans setup for a while before putting acls on. Also procurve manager is useful if you can afford it for HP - probably worth making sure switches are running latest firmware as well.

[nicholab]

You can just have 1 DHCP server to rule them all! You would actually specify that other vlan to use the other helper address.

 

The best way I have seen for documenting port allocation is a excel in a visual box format with version control on the 1st sheet.

Edited March 9, 2011 by nicholab

[gshaw]

You can just have 1 DHCP server to rule them all! You would actually specify that other vlan to use the other helper address.

 

Could have one but I prefer to split it in case Windows has a wobbly. OK recovery is easier once it's virtualised but I'm paranoid

 

What do you mean about "that other VLAN"? As I understand it in each VLAN I define I put the IP helper address e.g.

 

vlan 40
name "Curriculum"
tagged A10-A15
ip helper-address 192.168.11.250 
ip address 192.168.40.254 255.255.255.0 

 

So the DHCP server is in VLAN 11 (for this example) and the PCs looking for DHCP are in VLAN 40. Would look the same for say VLAN 50 just changing the IP address for the VLAN in the config

[teejay]

Could have one but I prefer to split it in case Windows has a wobbly. OK recovery is easier once it's virtualised but I'm paranoid

 

What do you mean about "that other VLAN"? As I understand it in each VLAN I define I put the IP helper address e.g.

 

vlan 40
name "Curriculum"
tagged A10-A15
ip helper-address 192.168.11.250 
ip address 192.168.40.254 255.255.255.0 

 

So the DHCP server is in VLAN 11 (for this example) and the PCs looking for DHCP are in VLAN 40. Would look the same for say VLAN 50 just changing the IP address for the VLAN in the config

 

Yes you can have multiple ip helper addresses.

You only want to specify the ip helper addresses and the ip address on the switch that will be the default route for the vlan.

[gshaw]

Yes you can have multiple ip helper addresses.

You only want to specify the ip helper addresses and the ip address on the switch that will be the default route for the vlan.

 

Yup that makes sense, if I've got the design right the config above (and the rest of the VLANs) will be going on the core 5406zl in the server room. The edge switches are 2610-24s connected back by fiber and they'll have something like this on them...

 

vlan 4
name "Curriculum"
untagged 11-20
tagged 24

 

Untagged ports for the client machines and tagged port for the fiber uplink

[teejay]

That looks spot on.

Few other things:

Make sure routing is disabled on all the switches that don't require it and the default gateway on all switches except your core switch is updated to your core switch that's doing the routing.

When you've set up the switches, on each one download the configuration file and also go into the console cli interface and do a write mem.

Some of your edge switches may be defaulted to 8 vlans max, change this if required.

For added security you may want to look at moving switch management off the default VLAN.

Oh and make sure you've got a stress ball to hand ;-)

[gshaw]

I've got a collection of stress balls on top of my PC and the latest addition is one that lights up when I throw it around... been carrying it around the corridors recently thanks to upcoming server virtualisation project :-P

 

On the edge switches would the default gateway be the main management IP address of the core switch (what would usually be in VLAN 1 but as you say might be safer moved to something separate). In what circumstances would the edge switches use this as in theory all the clients should be using the gateway of the VLAN they're in?

 

E.g. a client in the 192.168.40.0 VLAN would have its gateway set to 192.168.40.254 by DHCP scope 003 Router option.

 

Just reading around some of the HP configs on their forums some people have IPs for the switch in every VLAN (on their edge config). Didn't think this was necessary?

Edited March 9, 2011 by gshaw

[teejay]

The default gateway is used by the inter switch communication traffic and if you use stuff life snmp on the switches. If you don't have it set to the core switch and you move say a monitoring server which uses snmp to a vlan then it won't communicate.

You only need an ip address on the core switch, the only other time you'll need an ip address is if you move switch management onto it's own VLAN and then you'll need an ip address for each switch in that VLAN.

[CyberNerd]

It might be worth telling your active directory about the new IP scopes !

[CyberNerd]

On the edge switches would the default gateway be the main management IP address of the core switch (what would usually be in VLAN 1 but as you say might be safer moved to something separate). In what circumstances would the edge switches use this as in theory all the clients should be using the gateway of the VLAN they're in?

 

Give all the switches IP's from the management VLAN - ie VLAN1 and set the core as the default route. It will only complicate things moving the management vlan to something else, I can't see why it would be more secure? All clients should be using the core switch as the default gateway. I think you may be confusing things - VLANs work at Layer 2 but the routing is working at layer 3.

[gshaw]

It might be worth telling your active directory about the new IP scopes !

 

Was wondering about this, some people recommend putting each one in Sites and Services and others don't? I'm thinking of erring on the side of caution and putting them all in there.

 

Same goes for lookup zones in DNS I'd presume, best to have everything in there.

 

So if I leave the switches on the default management VLAN I'd end up with...

 

• one IP for the core switch e.g. 192.168.1.1
  
• an IP for each edge switch e.g. 192.168.1.11 (for say the 10th switch)
  
• a default gateway command on each edge switch pointing to 192.168.1.1
  
• clients DHCP pointing to the IP address of the VLAN they're in e.g. 192.168.7.254
  

[teejay]

It will only complicate things moving the management vlan to something else, I can't see why it would be more secure? All clients should be using the core switch as the default gateway. I think you may be confusing things - VLANs work at Layer 2 but the routing is working at layer 3.

It's recommended by HP to move the management off the default VLAN, http://www.hp.com/rnd/pdfs/Hardening_ProCurve_Switches_White_Paper.pdf

[gshaw]

All clients should be using the core switch as the default gateway. I think you may be confusing things - VLANs work at Layer 2 but the routing is working at layer 3.

 

Seems to go against all the docs I've been reading so far, if I use the default switch IP as gateway on clients I'd up with something like...

 

On client

 

IP address: 192.168.40.100

Subnet: 255.255.255.0

Gateway: 192.168.1.1

 

Doesn't seem right? Pretty much all the HP guides I've seen so far say that the IP address of the VLAN (set on core switch) should be the default gateway for clients?

[teejay]

Seems to go against all the docs I've been reading so far, if I use the default switch IP as gateway on clients I'd up with something like...

 

On client

 

IP address: 192.168.40.100

Subnet: 255.255.255.0

Gateway: 192.168.1.1

 

Doesn't seem right? Pretty much all the HP guides I've seen so far say that the IP address of the VLAN (set on core switch) should be the default gateway for clients?

 

No, it's not right, on the core switch you assign an ip address for each vlan as per your configuration in op and use this as your default gateway address in that vlan. In your example you would set the default gateway to 192.168.40.254 if the code example given earlier was used on your core switch.

Edited March 9, 2011 by teejay

[gshaw]

Phew, glad I'm not going mad

 

Looks like to shift the management onto a separate VLAN I need to run

 

ProCurve Switch(config)# management-vlan

 

After configuring another VLAN on the core and putting the switch static IP address in the edge config (along with the extra VLAN declaration and any untagged ports required for a management PC)

 

According to the HP docs if a VLAN is defined for management only devices in that VLAN can manage the switch, that VLAN also becomes non-routable. I guess it depends on the level of paranoia and whether someone would try to reconfigure a switch but probably worth doing. Basically means you end up needing a set PC to manage the switches set up on its own untagged port in the management VLAN.

 

Does make things a bit more complicated than using VLAN 1 but I guess that's the price you pay for security...

 

Just those pesky ACLs to double check now!

Edited March 9, 2011 by gshaw

[SkreeM1980]

Gotcha's we found:

 

All the usual switching stuff which you've covered, although you also need a DHCP helper pointing at your PXE server if you ever want network boot to work. But the oddest problems were with some client software. Things that talk to network licensing servers/dongles can be funny and may need special configuration. We had one app that needed the licensing server IP entering in a config file, and another that needed the name entering as an environment variable. These programs usually use broadcast to locate the server, of course once you have VLANNED that doesn't work. Also if you use WOL there is a command about directed-broadcast you will need to enter in the core switch.

 

ISsues since then have been when adding additionalo VLANS making sure that the path back to the core through multiple uplinks all have the tagged traffic enabled. takes a while to diagnose if it's not right.

[CyberNerd]

Seems to go against all the docs I've been reading so far, if I use the default switch IP as gateway on clients I'd up with something like...

 

On client

 

IP address: 192.168.40.100

Subnet: 255.255.255.0

Gateway: 192.168.1.1

 

Doesn't seem right? Pretty much all the HP guides I've seen so far say that the IP address of the VLAN (set on core switch) should be the default gateway for clients?

 

You quite right. I'm sorry just re-read my own post and not even sure how I came to that conclusion myself. has been a long day.

The DHCP should set the default route to the core switch as per teejays response. The IP's of the switches should be in the management VLAN - and the edge switches default route should be that of the core switches management VLAN IP address. is sort of what I meant to say.

[gshaw]

No worries, made me think and double check my understanding so works out well in the end

 

WOL is an interesting one as I might play with it for patching with SCCM in time and I've read a few things about making it work over VLANs. The main thing is NetSupport School but seeing as that's done by the Smartboard PC in the classroom should be OK as it will always be in the same VLAN as the machines it's waking up.

[SkreeM1980]

I wouldn't use the management-vlan option, just have all the switches in a VLAN, better off not using vlan 1, if you have vlan 1 as a non routed vlan with DHCP you can use it as a cleanup/catchall vlan to find clients that are connected to ports not properly VLAN assigned.

 

The WOL stuff is useful; we have some power management software, that is used to switch the it suites on for the start of the day so uses WOL, and we can fire up machines directly when wanting to config them or troubleshoot. obviously this is most useful on a large site as it can save a lot of running around.

 

Skr

[tumbleweed]

It might be worth telling your active directory about the new IP scopes !

 

Definitely - add new subnets into AD S&S, I couldn't browse mapped drives until this happened.

[SkreeM1980]

I wouldn't use the management-vlan option, just have all the switches in a VLAN, better off not using vlan 1, if you have vlan 1 as a non routed vlan with DHCP you can use it as a cleanup/catchall vlan to find clients that are connected to ports not properly VLAN assigned.

 

I think i may have changed my mind on this following some other conversations i was having with Other college IT managers yesterday. I have also been advised that I should move away from telnet where possible for switch management.

 

Skr

[gshaw]

Just been having a think about a migration plan to get onto the new infrastructure...

 

- set up new VLAN on the core switch, enable IP routing etc

- VMWare farm is then reconfigured for VLANs, initially I'll put all the VMs in one VLAN, same IP addressing as the current flat network

- then I was thinking of going to the edge switches and setting all ports (apart from the trunk fiber) to one VLAN e.g. VLAN10, same IP addressing as current flat network

- at this point the network should be pretty much the same as it was before

- then I create a new VLAN for classroom machines and move a few test machines into it so now I've got two VLANs

- make appropriate DHCP, DNS and AD config changes for new subnet etc

- check connectivity

- if this works move all classroom PCs across... done

- do the same for a test printer

- if it works OK move the rest across and update GPOs accordingly (gives us a chance to tidy these up at last!)

- then move servers into their own VLAN as required

- last part, test ACLs on a new VLAN and if working OK apply to classroom PC VLAN as appropriate

 

Hopefully this method should let users continue working while we're doing the reconfiguration and seems the easiest way to do it, any thoughts?

[chazzy2501]

Can I Hijack this thred a bit... Can I (oversimplistically) e.g. have 3 switches 2 edge 1 core

 

can I set an untagged vlan (4) for all the edge devices on edge switch 1 do the same for edge switch 2 but with vlan(5) and leave the uplinks as tagged for the core switch. Then on the core switch make the servers port a (un)tagged member of both?

 

i.e. Do I need the core switch to route ip traffic between vlans? I don't care if 1 edge device on 1 switch can see another edge device on another switch. In this setup I could avoid subnetting as well?

[localzuk]

Untagged ports can only be untagged for a single VLAN.

 

Your solution would work if your server had 2 network ports, one tagged 4, and one tagged 5. How you would handle DHCP via that method though, I don't know.