BGP Network Design – BGP Confederation

BGP confederations allow network engineers to manage the complexity and scale of large autonomous systems. BGP confederations can be seen as an alternative to BGP route reflectors, or possibly used together. The focus of this article will be solely on BGP confederation: the basic overview, usage and configuration.

Why would you want to implement BGP Confederation?

Based on the idea that an autonomous system (AS) can be split into multiple, more manageable sub-autonomous systems (sub-ASs from now on), each of which has its own AS number, BGP confederation enables you to apply a scalable approach when it comes to BGP deployment and controlling your large autonomous system. Implementing BGP confederation significantly reduces the total number of BGP connections within an autonomous system, that is, the number of iBGP peering sessions per router within that autonomous system. Large number of iBGP sessions might consume a significant amount of bandwidth, as well as cause high CPU utilization, and can therefore negatively affect the performance of the overall network.

When dealing with the iBGP scalability issue, you may either focus on BGP confederations or BGP route reflectors, or maybe even both. The idea behind route reflectors is explained in this article, if you wish to have a look. Nevertheless, it’s necessary to understand why choose one over the other:

• Implementing BGP route reflectors does not require major changes to existing configuration and topology, as opposed to confederation (please do not give up already!). It implies choosing routers that will act as a focal point for iBGP sessions within a single AS, running a single IGP.
• Implementing BGP confederations involves quite a change to BGP configurations and the architecture itself, adding more complexity to achieve stable and scalable BGP network.

The main difference between the two lies in the fact that a confederation may contain different IGPs, adding more flexibility to scaling your network. Therefore, choosing a confederation over route reflectors would be more appropriate in case your IGP is exceeding its scalability limit and becomes unmanageable, and you would like to manage many independent ASs, each of which may run a different IGP.

BGP Confederation

The next picture represents the idea of splitting one autonomous system into several sub-ASs. Each sub-AS has different AS number, applies its own iBGP rules, and is sometimes running different IGP from other sub-ASs. Also, all BGP speakers inside the sub-AS are fully meshed in order to learn external routes from external sources and exchange external routing information. In other words, since every sub-AS is identified by its unique AS number (usually private: 64512 – 65535), and functions as a fully meshed domain (a network of interconnected iBGP peers), the connection between them is always eBGP peering, also referred to as intraconfederation eBGP. eBGP routes that are exchanged between the sub-ASs are also known as confederation external routes, which are preferred over iBGP routes when it comes to best path selection. If BGP has to choose between two paths to the same destination, one path leading inside the sub-AS, and another outside the sub-AS but within confederation, it will choose the external path – towards the neighboring sub-AS. If it has to choose between confederation eBGP route and eBGP route that leads outside the confederation, BGP will choose the second one.

An eBGP connection between sub-ASs also serves as kind of a loop-avoidance mechanism. Namely, AS path list that is exchanged between eBGP peers within confederation is used to detect if routing update leaving one sub-AS returns to that same sub-AS. If it sees its own sub-AS number in AS path list, sub-AS will not accept that routing update.

As you may guess, even when divided into several sub-ASs, your network confederation is identified by AS Confederation Identifier and appears as a single AS to the outside world, that is, to BGP peers external to the confederation.

The role of AS_PATH attribute in Confederation

Let’s talk about the BGP AS_PATH attribute. As you may already know, it is an attribute that contains a list of autonomous system numbers a route has traversed to reach its destination, including the number of AS that originated that route. When routing update passes an AS, the AS number is prepended to the list. In other words, AS_PATH attribute within routing update is modified only when traversing eBGP.

In standard BGP, AS_PATH attribute contains two parameters:

• AS_SET: unordered set of ASs that a route has traversed
• AS_SEQUENCE: ordered set of ASs that a route has traversed

In case of a confederation, two additional parameters are introduced:

• AS_CONFED_SET: unordered set of sub-ASs that a route has traversed in confederation
• AS_CONFED_SEQUENCE: ordered set of sub-ASs that a route has traversed in confederation

Confederation set (AS_CONFED_SET) is prepended to the existing AS_SET only when the route traverses the confederation, and is removed and replaced by the AS Confederation Identifier when the route leaves the confederation.

The AS number is prepended only by routers that are advertising routes to their eBGP neighbors. This means that whenever a router has to send a routing update towards another sub-AS, it will do it via intraconfederation eBGP session and add its sub-AS number to the list. Again, sub-AS number is only preserved within the confederation, which prevents routing information loops inside it. If a router has to send an update towards its iBGP neighbour within a sub-AS, it will not change the AS_PATH attribute.

There is one more thing that needs to be kept in mind when it comes to eBGP peering within confederation. In case all the sub-ASs run the same IGP, attributes such as next hop, MED, and local preference do not change when routing update is traversing the confederation eBGP connection, as opposed to standard eBGP. Essentially, the sub-ASs exchange routing information as if they are using iBGP, and the only attribute that changes is AS_PATH. In other words, eBGP behaves like iBGP when implemented inside a confederation. In case different IGPs are running within sub-ASs, the next hop value is changed in routing updates between eBGP peers.

Configuring BGP Confederation (on Cisco routers)

Based on the below picture, here is presented the basic configuration of the AS 100 as a confederation, identified by confederation number 100, and consisting of two sub-ASs 65001 and 65002.

When BGP peer A has to send an update outside the confederation (for example towards AS 200), it advertises BGP confederation identifier 100 in the AS_PATH attribute, but before, it removes all the sub-AS numbers from the AS_PATH attribute, i.e. AS_CONFED_SET and AS_CONFED_SEQUENCE. In other words, Confederation Identifier is used to advertise routes outside the confederation, meaning that sub-AS numbers are not visible externally.

On the other hand, when it has to send a routing update towards another sub-AS within the same confederation (to sub-AS 65002), router A prepends its own sub-AS number (65001) as the last element of the sequence in AS_CONFED_SEQUENCE of the AS_PATH attribute. When it advertises a route to its iBGP peer (within the same sub-AS), it does not modify the AS_PATH attribute associated with that route. For more detailed explanation on that matter, you can refer to the RFC 5065.

BGP Confederation and possible issues

As mentioned before, in standard BGP the AS path is defined by the sequence of ASs that a route in routing update has traversed to reach a destination. Subdividing an AS may increase the overall AS path length, thereby affecting optimal routing inside the AS. Additionally, subdividing an AS without any logic and connecting sub-ASs randomly may cause routing information to be duplicated, result with delays, route flaps, or waste of system resources.

It is recommended to introduce one central sub-AS, through which all other sub-AS will exchange routing information between themselves. In that way, every sub-AS will exchange routing information with one sub-AS, resulting with uniform routing within the confederation. Since sub-ASs do not have to be connected to each other but only to this central (transit) sub-AS, the number of intraconfederation eBGP sessions is reduced. Such a scenario results with predictable AS_PATH within confederation, and therefore simplifies design changes, for example in case of introducing new BGP speakers to the confederation, as well as eventual troubleshooting.

Conclusion

As a conclusion, the following are the things worth to remember about BGP confederation:

• it is a means to deal with complex full-meshed topology of iBGP sessions
• implies dividing an autonomous system into sub-autonomous systems, each running full mesh iBGP
• new type of eBGP – intraconfederation eBGP behaving slightly different from the standard one
• AS_PATH attribute contains AS_CONFED_SET parameter which is modified inside the confederation only
• in case the confederation runs one IGP, NEXT_HOP, MED, LOCAL_PREF do not change when routing update traverses intraconfederation eBGP

Even though the idea of segmenting one large system into small parts seems straightforward, you should take the use of BGP confederation with caution. Migrating to confederation requires quite a change to your BGP network, as opposed to introducing route reflectors, which integrate easily to an existing network. Bear in mind that the logical topology of your network will change, which implies new approach to documenting, controlling and troubleshooting it.

You can combine route reflectors with confederation as well by implementing a route reflector inside each sub-AS. As with route reflectors, it all depends on the size of your network/AS, and your administrative demands and preferences. If you feel confident enough to implement a confederation, go ahead and give BGP that power. Migrating from non-confederation to a confederation is not an easy quest and it takes time, so arm yourself with patience and caution.

Are you among those engineers who were brave enough to implement a confederation, or at least thinking about it? Feel free to share your story or comments regarding this matter.

Keep calm and love BGP, till next time!