Connecting Web Server to new ISP

Need Help with making Linux respond to NEW and OLD IP

[tmhcorp]

We are going to use a new ISP and need to make our web server respond to requests from both sets of IPs. This way, we can make the DNS change and give it time to propagate before killing off the old ISP.

Our server has one NIC. The old IP address is assigned to ifcfg-eth0. Three other old IPs are assigned to virtual interfaces ifcfg-eth0:0, 0:1, 0:2. This setup has worked fine. These address are on a 255.255.255.248 netmask.

I'm trying to add 4 more virtual interfaces as ifcfg-eth0:3, 0:4, 0:5, 0:6 for the new IP addresses from the new ISP. These have a 255.255.255.0 netmask.

This seems to work locally. I can ping the new IPs from another machine on the same internal network (IP from same block) running W2K. The problem is, when I try to ping the Linux box from the internet, I get no response. I first thought it was an external routing problem, but if I ping the W2k box from the internet it DOES work. So the requests are getting to the local network, but the Linux box is somehow not responding properly when the request is external.

I suspect that a virtual interface cannot be on a different netmask as the real interface, but I cannot find documentation to specify if it should or should not work.

Can any networking gurus help me out here?

Thanks!

[VectorJ]

If I'm understanding your setup correctly, I think there may be two seperate problems. One is that the current ISP won't be responsible for routing the IPs from the new ISP, so any attempt to ping that IP would go to the new ISP and be dropped. (And if the W2K box is using an old IP, that would still be routable, which is why you can still ping it). Also you can't attach IPs that don't fall into the same netmask to the same interface.

FWIW, in my experience, when moving from one ISP to another it's a good idea to have a copy of your website running from both the new and old locations until the new IP propagates. If possible, get your old ISP to set the time-to-live (TTL) on the old DNS record to something like one hour so when the change goes through it will be picked up more quickly on non-authoritative DNS servers.

[tmhcorp]

Sorry, let me clarify a little bit:

We run our own webserver, so we will not be using a different box. What we are doing is moving from a DSL connection to a T1 connection. We have both connections live right now.

The W2K box is using one of the new IPs (for testing only).

>Also you can't attach IPs that don't fall into the same netmask to the same interface.

This is what I was afraid of. If this is true, then that is my problem. Is the only solution to add a second NIC to the server?

Thanks for the TTL suggestion. I hadn't thought of that.

[VectorJ]

My experience is with BSD, but the principle should be the same for Linux: the netmask for virtual interfaces has to be within the netmask for the primary interface. You should be able to add a second NIC without much trouble though.

[tmhcorp]

>the principle should be the same for Linux: the netmask for virtual interfaces has to be within the netmask for the primary interface.

So right now, the primary interface netmask is 255.255.255.248 and the virtuals I'm trying to add are 255.255.255.0.

If I make one of the new IPs the primary and the old IPs as virtual would that work?
I.e. is the 255.255.255.248 considered within 255.255.255.0?

(This would make it easier to delete the old IPs later anyway).

[VectorJ]

Sorry, I made a mistake: the virtual IP (not virtual netmask. I think the virtual netmask always has to be 255.255.255.255) has to be within the same netmask as the primary netmask. So in a /29 (a 255.255.255.248 netmask) you have six usable IPs. As long as the new and old IPs are part of that /29 and the netmask for the primary is 255.255.255.248 and the netmask for the virtual IP is 255.255.255.255, then it should work. Though frankly, I don't know what would happen with two different lines in this case....

But yeah, 255.255.255.248 is within 255.255.255.0. And I don't know if it would work to make the primary IP the new one and the old one the virtual one. It's worth a try!

[tmhcorp]

Thanks VectorJ.

Can anyone confirm that putting a /24 address as the main IP and moving the /29 as virtuals is the proper thing to do? (Before I try it on a live server?) :-)

[SeanW]

A virtual IP does not have to have any correlation to the primary address. I'm doing it right now:

eth1 Link encap:Ethernet HWaddr 00:10:5A:E1:8F:F9
inet addr:192.168.1.1 Bcast:192.168.1.255 Mask:255.255.255.0

eth1:1 Link encap:Ethernet HWaddr 00:10:5A:E1:8F:F9
inet addr:10.50.0.253 Bcast:10.255.255.255 Mask:255.0.0.0

Your problem is most likely routing, since if a packet was directed to the old address, it should return through the old connection. You need to route based on source address, which is called policy routing.

DNS is probably your best bet -- lower the TTLs, make the cutover, and wait.

Sean

[Sharper]

If it's that important to keep this service up that you can't just lower the DNS TTL to 5 minutes and then switch after that's propogated, then really, just spring the $10 for a 2nd NIC card and you can keep both connections going without any major issues.

Of course, like the above poster stated, a virtual can have an IP and subnet totally unrelated to the main address with no problem. The only thing a 2nd nic will buy you is a seperate physical connection so that you can connect it to your other network physically.

You should do some traceroutes to confirm, but it sounds like you have incoming packets from two different networks, but probably have a default route for outgoing packets for only one network. This is only a major problem if your outgoing network provider doesn't route packets for you that don't appear to have originated from their network. This is done as an anti-spoofing measure.

[tmhcorp]

OK. I eventually went with the lowering TTL solution. Lowered TTL to 5 minutes, waited for the time period of the old TTL (it was a few hours), then made the switch. Everything went well.

Someday, I'll revisit the issue of connecting both networks to the server. I've always wondered how a redundant ISP connection to a web server would work. (It's not as easy as I first thought...)

Thanks to everyone for your help.

[SeanW]

I've always wondered how a redundant ISP connection to a web server would work

In a nutshell, your web server only gets one IP, your router has multiple connections and uses a protocol called BGP to advertise your network out both connections. Definately best left to an ISP or large company, especially since a decent router capable of holding 2 copies of the Internet routing table is in the $30K range.

Sean

[Sharper]

Not quite that expensive for the router nowadays and you can actually run BGP on the web server itself (FreeBSD/Linux make it pretty easy), but you'd still have to purchase an ASIN, convince your ISPs to allow you to connect to their BGP routers and have a decent network guy configure it all for you, so it's still not going to be cheap overall.