[debian] Bizarre IP behaviour on a multi-homed Linux box

[will_]

Hi,

 

I'm trying to set up a FOG server on Debian 8. It has two network interfaces, both with static IP addresses.

 

eth0: 10.0.248.68 / 255.255.252.0 <-- This is to connect to the outside world.

eth1: 10.0.252.1 / 255.255.255.0 <-- This is for clients to connect to

 

Client machines awaiting a rebuild are connected to eth1, from where they are offered an IP address in the range [10.0.252.10-254] and PXE boot into FOG. This is where I'm stuck.

 

As far as I can tell, DHCP is somehow overwriting eth1's static IP address - it keeps changing from 10.0.252.1 to 10.0.252.10. Weird.

 

This doesn't seem to make much sense to me. I'll nip down to the server now and try and get you some more concrete information.

[localzuk]

How are you defining your static IP?

 

Can you post your /etc/network/interfaces file?

[pete]

Post the contents of /etc/network/interfaces

 

Putty on your desktop and OpenSSH on the server (it's probably already installed) will also make your life easier.

 

edit: beaten.

[will_]

/etc/network/interfaces

 

# This file describes the network interfaces available on your system

# and how to activates them. For more information, see interfaces(5).

 

source /etc/network/interfaces.d/*

 

# The loopback network interface

auto lo

iface lo inet loopback

 

# Interface 1: the outside world

allow-hotplug eth0

auto eth0

iface eth0 inet static

address 10.0.248.67

netmask 255.255.252.0

network 10.0.248.0

broadcast 10.0.251.255

gateway 10.0.250.16

 

# Interface 2: local network for imaging

allow-hotplug eth1

auto eth1

iface eth1 inet static

address 10.0.252.1

netmask 255.255.255.0

network 10.0.252.0

broadcast 10.0.252.255

 

# Old setup using DHCP

# allow-hotplug eth1

# iface eth0 inet dhcp

 

/etc/network/interfaces.d/ is empty.

Edited May 19, 2015 by will_

[will_]

A client connecting to the server showed this on its PXE boot screen:

 

CLIENT MAC ADDR: blah blah blah GUID: blah blah blah

CLIENT IP: 10.0.252.11 MASK: 255.255.255.0 DHCP IP: 10.0.252.1

GATEWAY IP: 10.0.252.1

 

PXE-E11: ARP timeout

PXE-E11: ARP timeout

PXE-E30: TFTP cannot open connection

PCE-M0F: Exiting PXE ROM.

[localzuk]

Delete auto eth1 and auto eth0 - auto means dhcp.

[will_]

Delete auto eth1 and auto eth0 - auto means dhcp.

Really? I'll give it a try but the man page for interfaces says,

Lines beginning with the word "auto" are used to identify the physical interfaces to be brought up when if is run with the -a option.

[Muz]

Yep, DHCP is something like this:

 

iface eth0 inet dhcp

[will_]

I tried removing the auto lines and running /etc/init.d/networking restart but it left with with:

 

eth0: no IP address

eth1: 10.0.252.10 <-- the troublesome DHCP lease, not the static IP in interfaces

[localzuk]

Oh derp. That'll be flustered brain! :/

 

What is this line?

 

allow-hotplug ethic

 

Should that not be eth1.

 

Also, unless the 2 ethernet cards are removable (ie. not always plugged in) you don't really need the allow-hotplug lines - they are more aimed at things like wifi dongles and wifi cards that can be turned off.

Edited May 19, 2015 by localzuk

[Geoff]

@Muz is right. 'auto ethX' tells the network scripts to setup the connection on boot.

 

I'd suggest you remove those hotplug lines. Hotplug in this case also covers removing the cable (it's to handle things like wifi really).

 

Also, do you have network manager running?

 

Also bear in mind a PXE-E11: ARP timeout error actually means the machine can't talk to the TFTP server listed in the DHCP options. So check that's right and check the TFTP server is running and bound to the correct interface/ip.

[will_]

allow-hotplug ethic

Oops! Over-enthusiastic spell-checker. Yes, it should have been eth1. I've edited my original post and put the brakes on the spell-checker's auto-correct!

 

Also, unless the 2 ethernet cards are removable (ie. not always plugged in) you don't really need the allow-hotplug lines - they are more aimed at things like wifi dongles and wifi cards that can be turned off.

I'd left them in to catch the cable being plugged in and unplugged (which is happening quite a bit while I'm trouble-shooting). I'll remove them if it doesn't affect that.

 

EDIT: Come to think of it, maybe it's a hotplug event that's kicking DHCP into action. I'll try disabling it.

Edited May 19, 2015 by will_

[localzuk]

Realistically though, those configs should work fine. If the hotplug lines don't make any difference, I think @Geoff might be on the right lines - do you have network-manager running? (I hate network-manager so disable it on any system it appears on!)

[will_]

allow-hotplugs removed.

 

# /etc/init.d/networking restart

[ok]

# ifconfig eth1

10.0.252.1

 

Cable unplugged and re-plugged.

 

# ifconfig eth1

10.0.252.10

 

It's definitely something to do with an event firing off. Just to be doubly sure, I'm going to try restarting the whole server. (It's not yet doing anything except frustrating me!)

[will_]

do you have network-manager running?

I don't think so.

 

# ps aux | grep net

[netns]

grep net

 

...Unless its process has a more obscure name which I'm unaware of.

Edited May 19, 2015 by will_

[Muz]

Have DHCP options 66 and 67 been set on your DHCP server?

[pete]

Are you seeing anything relevant in:

 

/var/log/dmesg

/var/log/syslog

/var/log/messages

 

when unplugging and re-inserting the cable for eth1?

 

What happens if you (as a test) change eth1 to:

 

auto eth1
iface eth1 inet static
       address 172.27.0.2
       netmask 255.255.255.0

 

Does it change after hotplugging or not?

[Geoff]

I always set my network cards up like this:

 

# The primary network interface
auto eth0
iface eth0 inet static
       address 192.168.1.16
       netmask 255.255.255.0
       gateway 192.168.1.254
       dns-nameservers 192.168.1.10
       dns-search dev.work.co.uk

[will_]

Have DHCP options 66 and 67 been set on your DHCP server?

FOG installed (or at least configured) the DHCP server, so hopefully. I don't think the problem is with the DHCP being served to the client, though; it is that the server's IP address is changing so the client can't find the PXE server specified in the DHCP offer.

 

Are you seeing anything relevant in:

/var/log/dmesg

/var/log/syslog

/var/log/messages

when unplugging and re-inserting the cable for eth1?

That's a good point. I'll have a look today.

 

What happens if you (as a test) change eth1 to:

auto eth1
iface eth1 inet static
       address 172.27.0.2
       netmask 255.255.255.0

Does it change after hotplugging or not?

That's another great suggestion. Thanks. I'll try it.