Overloading BGP for VPN Can Be Harmful
Other than TCP/IP itself, Border Gateway Protocol (BGP) is perhaps the most important protocol for the Internet. The importance of BGP to the Internet cannot be overstated; it's the ingredient that glues together all the different autonomous systems (ASs) spanning the entire globe and under various administrative authorities. Each AS is a collection of routers and network links that uses local policies for selecting and advertising routes.
From a protocol point of view, BGP is amazingly simple; there are only four message types, and its finite state machine has only six states. However, BGP has proven to be a complex protocol to implement. This is the direct result of the large number of attributes that BGP supports. The criteria for selecting routes can vary considerably, based on the policies applied. For example, the need to prevent black holes makes the injection of routes into the forwarding table and the condition for advertising routes into BGP necessarily complex. The presence of policies and absence of route-refreshing schemes forces the BGP speaker to keep separate copies for both the incoming routes and outgoing routes for each peer.
BGP has proven to be an even more complex protocol to operate. This is again the consequence of the complexity of the inter-domain routing system, which directly corresponds to the combined complexity of the network topology, the attributes of the routes, and the policies governing the route selections.
The Art of BGP
Consider the following facts about running BGP on the Internet:
BGP protocol is not guaranteed to converge.
An unfiltered Internet routing table has about 120,000 entries.
There are about 11,000 active ASs on the Internet, and their connectivity graph can be arbitrary based on public and private peer arrangements.
There are possibly several tens of BGP sessions, both internal and external, to be managed for a single router.
Because of the complexity of BGP and the dynamic nature of the Internet, the operation of BGP on an ISP backbone has long been more of an art than a science. Only the most senior backbone routing engineers are permitted to change BGP configurations for the core routers, and configurations usually are not changed until things are broken. Data from the daily CIDR report suggests that simple route aggregation could drastically reduce the size of the Internet routing table. For example, a reduction of 35% of announced routes from the top 30 ASs can be achieved.
Interestingly, although clear advantages were pointed out on the merit of simple route aggregation, little has been done to achieve it. Perhaps only the network engineering departments that manage those autonomous systems can answer this question precisely. The reasons may be twofold. First, the network environment is highly dynamic, and any gain achieved may only be transient, thus not warranting the effort to implementing it. Second, there is a lack of expertise on the BGP policy management within the organization, and thus those necessary changes cannot be implemented.