This scenario often causes two somewhat annoying behaviours:
1) the return traffic often is sourced from the “primary” IP address of the host/server, most often the one that is on the subnet associated with the default gateway
2) a surprising number of alleged
“network administrators” seem to think having multiple gateways (one for each IP address of course
) is a good idea. Well, over the years I have come across this situation and, in every case, this has obviously
NEVER WORKED.
Situation #2 can only be fixed by education and occasionally the dismissal of the “IT” person for someone more qualified. As for situation #1, RedHat/CentOS supports via
iproute2 the ability to make traffic rules, ensuring that IP traffic is sourced by a particular IP in cases you can define. Great! Multiple NIC or logical interface routing on Linux is possible! (and yes, it involves having multiple gateways, but not stupidly and blindly adding them in the routing table….)
It is very simple to implement and involves the steps below. As an example, lets assume we created a management VLAN (VLAN4) and want to add an logical interface on a server in that VLAN to access it internally. We will be using 10.0.10.0/28 as an inside network.
Step 1: Create a VLAN interface
This creates the necessary interface on VLAN4 from primary physical interface eth0:
vi /etc/sysconfig/network-scripts/ifcfg-eth0.4
DEVICE=eth0.4
BOOTPROTO=static
ONBOOT=yes
TYPE=Ethernet
IPV6INIT=no
IPADDR=10.0.10.2
NETMASK=255.255.255.240
NETWORK=10.0.10.0
VLAN=yes
Step 2: Create a iproute2 table for that management network
Edit /etc/iproute2/rt_tables to add a new entry and give it a arbitrary (unused
) name:
vi /etc/iproute2/rt_tables
#
# reserved values
#
255 local
254 main
253 default
0 unspec
#
# local
#
#1 inr.ruhep
200 MGMT
Note that between 200 and MGMT is a tab character.
Step 3: Create a default route for that network
vi /etc/sysconfig/network-scripts/route-eth0.4
default table MGMT via 10.0.10.1
this creates a default route for the MGMT/10.0.10.0/28 network to 10.0.10.1 which is your inside routing intelligence.
Step 4: Create routing rule for 10.0.10.2
To ensure that traffic received on 10.0.10.2 is utilizing the MGMT network only as a source address, a rule must be defined to enable this:
vi /etc/sysconfig/network-scripts/rule-eth0.4
from 10.0.10.2 table MGMT
and thats it! restart your network:
/etc/rc.d/init.d/network restart
Using
iproute2 commands, we can check that what we did works (as well as using wireshark
)
[root@server network-scripts]# ip rule show
0: from all lookup local
32765: from 10.0.10.2 lookup MGMT
32766: from all lookup main
32767: from all lookup default
[root@server network-scripts]# ip route show
10.0.10.0/28 dev eth0.4 proto kernel scope link src 10.0.10.2
66.1.1.0/25 dev eth0 proto kernel scope link src 66.1.1.12
169.254.0.0/16 dev eth0 scope link metric 1002
169.254.0.0/16 dev eth0.4 scope link metric 1006
default via 66.1.1.125 dev eth0
Note: this also would work with a second physical interface, for instance to utilize a second NIC card instead of a VLAN logical interface, substitute all use of eth0.4 for eth1.
