Posted on 2008-09-04 16:00:00
Tags: Networking, Peering
There are several kind of internet providers. One of them are the edge providers, who have their own IP space, AS number, provide application hosting and network services for their customers and have one or two uplinks to the internet which get charged per gigabyte of data. And a port on the local internet exchange of course! The other kind are the big ones, who have multiple links with other big ISPs and sell transit to the little ones. Oh, and also provide application hosting and network services for their own customers, and sometimes they are even on the local internet exchange because of that!
Local internet exchanges are places where multiple local ISP come together (you pay for the speed and for the port on the internet exchange, not for the traffic) and agree to route traffic between destined for the other ISPs on the internet exchange directly via the internet exchange instead of via the uplinks to the big ISPs. To push this behaviour, the address ranges advertised to the local internet exchanges are often /24s while the address ranges advertised on the internet uplinks are often /21s and bigger.
So euhm... What is the issue? Nothing yet, but it is coming :-)
If you are an edge provider, what should you do? Take an uplink which is also on the local internet exchange, advertise your /24s to the local internet exchange and your big /21 to the uplink provider. Why? Because the uplink provider will advertise your /21 to the rest of the internet, while it will internally route it via the /24s to the local internet exchange. Free inbound traffic! And if your port on the local internet exchange is 100Mbps or 1000Mbps and the link towards your uplink provider is less than 100Mbps, you will have a nice extra speed increase with it too. Of course this is only for the inbound traffic, the outbound traffic to the internet still goes via the slow uplink, but downloading goes fast. And since you are an edge provider, most of the traffic will be inbound.
The thing you have to take care about is that you have to monitor, specially in the first months, that the bill of your uplink provider does reflect the real traffic going over their uplink. Some providers run their accounting systems based on all the IP traffic going through their edge routers and will bill you for the traffic even if it doesn't go over the physical wire. Check your terms and conditions to see what you can do about this Layer 8 behaviour.
So what can the big ISPs do about this? Design their network properly. Consider the three different services which are being provided: Users, Services, Transit. Users traffic and Services traffic can go via the local internet exchange, but Transit traffic shouldn't. The network should be designed to have three routing clouds (call them Autonomous Systems if you want), and the exchange of routes between the three clouds and the outside world should be regulated carefully to make sure that the Systems and Users clouds are only providing the big ISPs IP space to the transit cloud. That way the /21 is in the Transit cloud and the /24s are in the Users and Services clouds.
Is this possible? I think so. Even if the Service networks and User networks are scattered around the world, with IP-over-IP tunnels between them it will make it look like one contiguous network. Doing proper routing traffic exchange between the three routing clouds internally and between the individual routing clouds and the external networks and the network behaves the way it was meant to work.
| Share on Facebook | Share on Twitter| From: | Mathieu Arnold |
| URL: | http://w.mat.cc/ |
| Posted on: | 2008-09-04 19:00:22 |
| Comment | Unless some pretty specific cases, people selling you internet transit won't peer with you, because, well, they'd be loosing money by doing so :-) |
| Reply | I understand your comment, but want to point out that at the local internet exchange in Sydney (http://www.pipenetworks.com/Peering/who.shtml#NewSouthWales) at least Uecomm and PacNet (Asia Netcom / Pacific Internet) have joined, and they both are selling transit bandwidth to a couple of ISPs on the internet exchange I know. And both don't have split their network in the three clouds. |
| From: | craig@askings.com.au |
| URL: | |
| Posted on: | 2008-09-04 21:07:11 |
| Comment | If you are a content provider (webhost etc) you can pull the opposite trick with your outbound traffic, which is harder to filter against. Just place a static default route (with a better local pref than your transit link) across the peering fabric pointing towards your transit providers router.
Filter out the routes you receive on your transit link and away you go. As the transit provider filtering at the BGP level isn't enough. You effectively have to blackhole their traffic by using an ACL on your peering router as a means to force them back to using their transit link |
| Reply | Heh, great trick but kind of kills the automatic rerouting when the destination IP address of the static route suddenly doesn't answer anymore. It might not happen often, but it will happen at the moment you can use it least. I think it can be overcome, by the uplink provider, with a "simple" incoming ACL filter on the port connected to the local internet exchange which says "If the destination address of the incoming packets doesn't match the address space I advertise, then drop it." |
| From: | craig@askings.com.au |
| URL: | |
| Posted on: | 2008-09-04 21:19:18 |
| Comment | You are correct that on PIPE's peering networks there are many transit providers and their customers (and their customer's customers). However there are some simple filtering techniques to use to prevent you learning customer routes of the IX.
1) AS Path filters that exclude your customer's AS Numbers. 2) Have your customers register their IP ranges with you and build ip prefix-list to deny those routes from being learnt. e.g. ip prefix-list PIPE-IN seq 10 deny 10.0.0.0/8 le 32 etc You would have these filters in place anyway to catch normal BGP mishaps like accepting a default route advertisement, RFC1918 space or worse yet someone leaking a full table onto the IX. Extending it to control what you get from your customers isn't that hard. |
| Reply | - |