Assigning IP's to second interface

[wheel]

I've got a block of IP's I want to assign to a second ethernet port on my webserver. The IP's are all from different C-classes, have different gateway's etc. In addition, the IP's assigned to my my primary ethernet port go through my host's firewall, this second block on the second port do not go through the firewall.

I've set them up using in the same fashion as I did my first set of IP's. However I can't access them externally.

I can ping them from the machine itself. I can also ping them from another machine on the same network as the first port (the one going through the firewall). I can't however ping them externally.

That to me says my ISP has something misconfigured since I can only get at the IP's within his network. He's claiming that I've got a misconfiguration with my default gateway (he was vague, I think he's saying traffic on the second port isn't coming and going in the right fashion).

Any thoughts on what I might be doing wrong? Is there a trick to this if there are different default gateways for different IP's on different ports?

[mcavic]

It can be tricky when you have two different interfaces that are both connecting to the Internet via different routers. When you ping the second interface from the Internet, the machine is probably responding to the ping via the first interface, but with a different IP address, the response would be invalid. If I'm right, that would explain why local pings work.

The key is the routing table, but you first need to decide exactly which IP's/subnets the second interface will communicate with.

[Sharper]

What's happening is similar to what the above comment is saying, the packet comes in the right interface, then is sent out the wrong interface.

The problem is going to be that a stateful firewall on the "wrong" interface will see the packet coming back, but didn't see the first packet going in, so it's going to block the response packet as not being in it's connection table.

The basic problem is that the response packets are leaving from the wrong interface. Without more details about at least what operating system you are running (uname -a would be helpful), and preferably what services you plan to run on the new IPs (http via apache?), we aren't going to be able to offer specific advice about the best way fix the issue.

[wheel]

I believe the above is the problem, but don't know where to fix it :). Running Mandrake linux 10.2. 2.6.11-6mdksmp kernel, apache2.

(thanks for the help!)

[mcavic]

What's the purpose of the second interface?

[wheel]

Got assigned a 2nd block of IP's that don't go through the ISP's firewall. THe first block on the other port do go through the ISP's firewall. Two seperate physical routes, thus I need two nic's.

And unfortunately I can't get all the IP's outside the firewall or inside. I've got to take the half and half.

[danny]

I have a machine that is multihomed with two different routers for each interface. I set up separate tables for routes from each IP address, to make sure packets go back out the interface they came in on.

# table 1 has rules for X.Y.151.5 (eth0)
ip route add X.Y.151.0/24 dev eth0 src X.Y.151.5 table 1
ip route add default via X.Y.151.254 table 1
ip route add 127.0.0.0/8 dev lo table 1
ip rule add from X.Y.151.5 table 1

# table 2 has rules for X.Y.131.5 (eth1)
ip route add X.Y.131.0/24 dev eth1 src X.Y.131.5 table 2
ip route add default via X.Y.131.254 table 2
ip route add 127.0.0.0/8 dev lo table 2
ip rule add from X.Y.131.5 table 2

And for default routes - where the connection is initiated from my server - I do load balancing across the two interfaces.

# for default routes, do load balancing
ip route replace default scope global nexthop via X.Y.151.254 dev eth0 weight 1 nexthop via X.Y.131.254 dev eth1 weight 1

[Sharper]

Yeah, Danny's got it.

For more details, see [lartc.org...]

The key is the "src" argument, which specifies to only use that route for packets from that source.