Inspired by working onsite at a contract in Los Angeles this week.
I’ve worked on a lot of other peoples networks. And while it can be entertaining, it’s rarely fun. Usually you are expected to jump in and fix the random problem that they are having within minutes, which is never a reality (except in the case of very obvious things like the power cord being unplugged).
Thankfully I’m not dealing with that this week, though I am having some fun with trying to figure out why certain things have been done, and more importantly, why things haven’t been done.
For example, the company that I’m onsite at have 2 large internet connections, one each to different carriers. What you normally would do with that – and I’m sure that was the intention here – is apply for an Autonomous System (AS) number, apply for a bank of IP addresses from ARIN, then setup those two connections to be redundant to each other, both inbound and outbound, by advertising your AS number and IP address range over the Internet.
Simple, right?
Well, actually it’s not all that simple, but it’s a pretty straightforward thing to do. In fact it’s pretty much the default configuration for large networks connected to two or more ISP’s.
Well, things aren’t quite setup like that here. They have the big Internet connections. They have an AS number. And they qualified for and got a class C bank of IP addresses from ARIN. But then things sort of fell apart. They attempted to setup the dual homing, and their internet went down. So they rolled back off of that to having two separate internet connections with all of their inbound traffic coming in one, and all of their outbound traffic going out the other. There is some failover, but only for web browsing – if the connection goes down that their external servers are on, then they will have no ability for their customers to reach them over their secondary line.
So now I’ve mentioned this and been asked to fix it.
Which is where the fun really starts.
It’s really not that bad to make a change like this, just time consuming. You have to make a lot of phone calls, make some changes to the way that your routers are configured, add some translations into your firealls and then wait around until all that propagates across the Internet. Once your new IP addresses are available across the Internet, you now have to update the DNS entries for your servers. Then when that information propagates, you can go back and fix the translations on your firewall. And then stop using the ISP supplied IP addresses that you have been using up to this point.
That is a bit simplified, but it covers the basics. Like I said, it’s not complicated, just time consuming. But if something does go wrong it can be even more time consuming to fix.
I’m in the process of writing up the detailed plan of how I’m going to make the changes to get this running correctly. Hopefully all goes smoothly.
And hopefully when their normal network admin gets back from vacation he’s not too confused by what I’ve done……