[ubuntu] virtual interfaces & vlans

[MarsRed]

Hi there,

 

I have an Ubuntu Server 12.04 machine with one physical NIC connected to a trunk port on a Cisco switch. I've installed the vlan package and configured /etc/network/interfaces to use 5 virtual interfaces with static IPs with vlan tagging, each on a different vlan.

 

The problem I've run into is that only one of the virtual interfaces seems to be "active" at a time, meaning only one of the IP addresses can be pinged. Sometimes, with a restart, which interface this works on will change. I had read that I should only configure the gateway on one of the interfaces, so I did, but that's not even the interface that always works!

 

Any ideas?

 

Thanks.

[morganw]

Have you got the 8021q kernel module loaded?

lsmod

[MarsRed]

Have you got the 8021q kernel module loaded?

lsmod

 

Yes, I do.

[morganw]

Can you post the interfaces file and the NIC model?

Also, are you pinging from a device that you are putting onto the relevant VLAN first?

[MarsRed]

Can you post the interfaces file and the NIC model?

Also, are you pinging from a device that you are putting onto the relevant VLAN first?

 

Here's the interfaces file:

 

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface

auto eth0.1
iface eth0.1 inet static
address 10.1.9.1
netmask 255.255.0.0
network 10.1.0.0
broadcast 10.1.255.255
gateway 10.1.0.1
vlan_raw_device eth0

auto eth0.2
iface eth0.2 inet static
address 10.2.9.1
netmask 255.255.0.0
gateway 10.2.0.1
vlan_raw_device eth0

auto eth0.3
iface eth0.3 inet static
address 10.3.9.1
netmask 255.255.0.0
gateway 10.3.0.1
vlan_raw_device eth0

auto eth0.4
iface eth0.4 inet static
address 10.4.9.1
netmask 255.255.0.0
gateway 10.4.0.1
vlan_raw_device eth0

 

NIC is Broadcom Corporation NetXtreme BCM5755 Gigabit Ethernet PCI Express.

 

I think your question about which vlan I'm pinging from might lead to an answer! I was only trying to ping from one vlan which can get to all other vlans. But now I realize (and just confirmed) that, when I'm on a given vlan, I am able to ping the interface of the machine that is on that vlan. i.e. from vlan1 I can ping the interface on vlan1, on vlan2 on I can ping the interface on vlan2, etc. Why might this be the case?

[morganw]

Have you got IP forwarding enabled?

cat /proc/sys/net/ipv4/ip_forward

[MarsRed]

Doesn't look like it. Should I?

[morganw]

You'll need that turned on to route between interfaces, otherwise the behaviour you've described is normal. Edit /etc/sysctl.conf and change net.ipv4.ip_forward from 0 to 1, reboot and re-test. Depending on what you are doing you might want to use a firewall on the server to restrict what is forwarded.

[MarsRed]

Okay, I just did this and can no longer contact the machine. I'll have to wait until I'm back in the office to look at the machine in person.

 

Now I don't really want to use this machine as a router, so does enabling IP forwarding have any implications?

[morganw]

Enabling it turns it into a router, otherwise what you have is multi-homed server that can offer services to each VLAN individually. As long as you are aware that without a firewall all traffic can be be forwarded onto another VLAN, that should be the only implication - how good or bad this is depends on your use case.

[MarsRed]

I'm using the machine to route mDNS between vlans, but it doesn't use traditional IP routing. The Avahi software just "reflects" mDNS traffic among the different interfaces/vlans without any routes required. Now that I understand that the behavior I was experiencing is normal without routing enabled, I'm okay with it.

 

But I wonder if this has anything to do with another issue I was having? I am also using Avahi to advertise cups print queues via mDNS (emulating Apple's AirPrint) so that iPads can print to non-AirPrint printers. The queues function normally when I print a test page from the cups web interface and the queues are successfully advertised by Avahi to the iPads, but printing is unsuccessful from the iPads. The iPads and printers are on separate vlans. Would this have anything to do with my interfaces issue?

[morganw]

I guess the advertisement of printers is making it to the iPad as Avahi is working on all interfaces, but when the iPad tries to print, the print server is running on a different VLAN and so there is no route to it.

[MarsRed]

That's what I was thinking. But the print server is on the same vlan since it's that same machine that's on all of the vlans via the virtual interfaces. Maybe that's part of the problem?

 

Thanks for all of your help.

[morganw]

You would probably have to check if your print server is listening on all interfaces, or just specific interfaces you've told it about. You could run tcpdump on the VLAN interface while trying to print to see what the iPad is doing.

[morganw]

Also it would depend on the gateway that is set on yout iPad.

[MarsRed]

I've configured cups to listen on all interfaces, but I'll try out tcpdump too to see what's going on. The iPad's gateway is the gateway for that wireless vlan (on which one of the machine's interfaces exists). Though I'm starting to think that my issue may have to do with how the printer is configured in cups. I'll try some things out and report back... Thanks!

[MarsRed]

Everything is working perfectly now! Not sure what was going on before, but thanks for your input.

[caffrey]

Any ideas what fixed this?, I'm getting similar behaviour with airplay and avahi over vlan, the broadcast name keeps dropping in and out too

[MarsRed]

Any ideas what fixed this?, I'm getting similar behaviour with airplay and avahi over vlan, the broadcast name keeps dropping in and out too

 

Re: Avahi not broadcasting continuously, I haven't found a clear explanation for it, but did find a solution based on my realization that restarting Avahi fixed the non-broadcasting issue. I have a cron job that restarts avahi-daemon every minute (simply "restart avahi-daemon"). This seems to keep things going.

 

Was that all you were having trouble with? Or was there another part to it?

[caffrey]

It's hard to explain but avahi reflects with a different name.

e.g. the broadcast name is somepc and on the Ipad it'll say ipad(11) or another random number and you can see the blue airplay bar fading in and out on the Ipad too.

 

Might try a different nic today altho I also redid everything on a raspberry pi and that failed too.

 

I had it working via a appletv for about 10 mins, no idea what fixed that

 

Edited March 1, 2013 by caffrey

[morganw]

Do you have any logs that you could look at?

I've just happened to notice on my server that I was getting service name collisions between Avahi and Netatalk.

 

Mar 2 15:55:16 server afpd[3245]: AFP/TCP started, advertising 192.168.1.10:548 (2.2.2)

Mar 2 15:55:17 server afpd[3245]: Failed to add service: Local name collision

[caffrey]

I'm starting to suspect the smoothwalls (I run two) blocking traffic from each subnet, not sure which logs to look at.

[caffrey]

Ok so now I think I've got to the bottom of it, just not sure how to fix it.

 

If I ping a machine on vlan1 from vlan1 using .local (eg testmachine.local) its fine, I can ping it no problem.

 

However if I ping a machine from vlan1 to vlan2 .local (eg ipad1.local) it doesn't work.

 

A traceroute reveals the traffic going to the smoothwall on vlan1 and leaving the network going outside.

(It is resolving the proper IP though).

 

Any clues?

Edited March 4, 2013 by caffrey