AWS Cloud WAN as an
Brillat, Global Domain Leader for Advanced Wireless, 5G, & Edge Computing,
December 2, 2021
Today, Amazon Web Services, Inc. (AWS) announced a significant new feature set in their networking portfolio, AWS Cloud WAN. AWS Cloud WAN brings new and powerful methods of connectivity among AWS resources, especially at global scale. As a launch partner for AWS Cloud WAN, we at Kyndryl have been examining common use cases where AWS Cloud WAN is able to help improve our customer's real-world networks. In this article, I will examine one common network design where the performance can be increased, the complexity reduced, and the reliability improved - all through the use of AWS Cloud WAN.
This is a simple scenario but very common in enterprise networks. I have two server instances in AWS which need to communicate. However, they are each in different VPCs, located in different regions, and hosted in different AWS accounts. How can I connect them?
The problem: how to connect Server01 with Server02?
The solution looks simple just create the black line between Server01 and Server02. However, given the constraints of the problem most specifically, different AWS accounts, the solution is not straightforward. While several possibilities exist, we will explore a single common design used to create this connectivity.
To establish IP reachability between the two instances, one possible solution is to take advantage of a 3rd point with connectivity which already exists: the Direct Connect or VPN link to the corporate datacenter. In this example, we assume that our company has established redundant VPN links between their corporate HQ datacenter and both AWS accounts. The resulting configuration is as follows:
VPN tunnels link the two accounts with the corporate datacenter, and therefore with each other.
This solution is simple and effective. It provides a path between subnet A1-1 and subnet B1-1 by way of the corporate datacenter's VPN router, using redundant VPN tunnels over the Internet. To create this connectivity, a number of elements were added to the VPC, including critically a Virtual Private Gateway, with the route table adjusted to accept BGP route exchange from that gateway. The VPN router on-site at the corporate location terminates a minimum of 4 VPN tunnels, 2 to each VPC.
In the AWS Console we can see the status of our VPN tunnels:
Both VPN tunnels to Account A are active.
The VPN solution is active and provides the necessary connectivity. However, it is limited in a few meaningful ways. First, it is inefficient. Traffic between two AWS locations is routed via the corporate HQ, increasing latency and adding bandwidth charges. It also consumes bandwidth from the Internet links of the HQ site, decreasing the available bandwidth for the users at that site for other traffic. Second, the resiliency of this environment is impaired, as we have created a point of failure at the corporate datacenter. If the HQ site experiences an outage, then this traffic fails as well even though neither Server01 nor Server02 are located at the HQ. Finally, the performance is limited by the speed of AWS Site to Site VPNs. Each Site to Site VPN link can only provide 1.25Gbps of throughput. To overcome this, some designs may implement multiple VPN links with load balancing, further increasing the solution complexity.
To demonstrate the performance limitation, we have built the network exactly as drawn in our lab environment. In our network, Server01 and Server02 are each c6i.xlarge EC2 hosts, each capable of 12.5 Gbps of throughput on a single network interface. They run iperf3 for network performance testing. We will leave iperf3 at the defaults and use TCP tests only for this article.
A simple iperf3 test, delivered 1.21 GW over the S2S VPN
This result is only 1/10 the performance of our c6i.xlarge host, and potentially much lower than we might desire for our application. Can this be improved?
Using AWS Cloud WAN, we can provide direct connectivity between the two AWS accounts. To create this connection, we utilize the new construct of a Core Network, and share that network across accounts using AWS Resource Access Manager. The resulting network configuration is as follows:
AWS Cloud WAN provides the link between various AWS Accounts and VPCs
The AWS Cloud WAN provides several new elements which together provide the connectivity between Server01 and Server02. The aptly named Global Network sits centrally between all of the connected accounts and VPCs, establishing the intra-region and inter-region connectivity between VPCs. The Global Network contains a Core Network, and access to a core network is gatewayed in each region by a Core Network Edge (CNE), a router element based upon Transit Gateway. Segments provide discrete, segregated routing tables to enable separation of traffic across the AWS Cloud WAN. Each VPC is connected to a designated segment via Attachments, and the selected VPC Route Table is pointed at the Core Network.
In our lab, we have created this environment and will review the key elements necessary to build this configuration.
The Global Network is the first unit of an AWS Cloud WAN design. It acts as a container for all other elements. The global topology view of the Global WAN is presented via the Network Manager console:
A map view of the Global Network is presented at the dashboard
A Core Network hosts all of the segments and routing connectivity of the AWS Cloud WAN. The Core Network is created in the Network Manager interface under Global Networks. To enable inter-account connectivity, the Core Network must be shared among all of the connecting accounts using the Resource Access Manager.
In the Resource Access Manager, create a new share, and choose the newly-available Core Networks option as resource type. Choose your selected Core Network from the list:
On the choose principals step, the 12-digit AWS account ID of all other participating AWS accounts must be entered:
This will propagate the Core Network to other accounts, where they will be prompted to accept the share and can then view it in Resource Access Manager as a resource under shared with me. The AWS Cloud WAN is now allowed to provide connectivity among the shared accounts.
To provide the necessary inter-VPC connectivity, we must connect each of our target VPCs to the Core Network. This is accessed again under new Network Manager interface as Attachments, where we can choose Create Attachment to search and connect our VPCs to the desired AWS Cloud WAN Core Network and segment. Once connected, they will appear in the list:
Once connected, the Route Table of each connected VPC must be updated to point at the Core Network ARN, for the IP prefixes of the WAN destinations:
The Network Manager console includes both a Topology Graph and Topology Tree, visually displaying the connectivity provided by the AWS Cloud WAN across the regions. As is shown in the drawing, one Core Network Edge is created per region. The VPCs and segments are also visible.
The connectivity we desired, between Server01 and Server02, has now been completed. The remaining question is, how have we improved the connectivity? At first glance, it may not look less complex. However, all of the elements held by the Global Network are scalable elements which must only be created once. For the entire enterprise, that core setup is now completed. In our lab, we only have two VPCs now connected, but we could scale to support many.
AWS Cloud WAN has also addressed the resiliency of the connection. The Core Network Edge devices, built on transit gateway, are connected with many redundant links in a full mesh among regions, fully managed by AWS. The physical failure of the corporate HQ site has been eliminated from the path.
Finally, the performance of this network has been greatly improved, as AWS Cloud WAN can provide up to 50 Gbps of throughput per VPC attachment to a Core Network Edge.
To demonstrate the improved performance, we have reconfigured our lab environment with the same c6i.xlarge hosts. Server01 and Server02 sit in separate VPCs, separate accounts, as shown in the diagram above. With a 50 Gbps throughput limit, we will not be able to reach that with these two hosts, but our iperf3 data does demonstrate the significant improvement. Each host has a maximum possible throughput of 12.5 Gbps.
We started with a single TCP stream, but as is often the case with high bandwidth circuits and iperf3, this did not saturate to our known limit. We therefore moved to 4 parallel TCP streams, and were able to saturate the NIC:
Iperf3 from Server01-Server02 over AWS Cloud WAN 12 Gbps
The delivered 11.8 Gbps is exactly as expected for a known 12.5 Gbps network interface. Our previous 1.2 Gbps result is nearly 10x lower the throughput delivered on the same server! We have maxed out the capabilities of this particular lab setup, but we expect performance to continue upwards as an AWS Cloud WAN network has 50 Gbps data-sheet performance.
AWS Cloud WAN offers a powerful new way to link complex, multi-account, multi-region AWS environments. As a launch partner of AWS Cloud WAN, Kyndryl's team of network Architects and Specialists is ready to help our customers put this new technology in to practice. The work will start with assessing current AWS environments for areas where AWS Cloud WAN might be used to improve the efficiency, performance, or reliability of the network. Once such opportunities are identified, new architectures utilizing the full power and feature set of AWS Cloud WAN can be designed, tested, and implemented. The new network designs arising from AWS Cloud WAN will enable continued growth and scalability of applications in the cloud.
To learn more about the Kyndryl-AWS alliance and how we can help your business with AWS Cloud WAN, visit: http://www.kyndryl.com/alliances/aws
To learn more about our services for enterprise networking, wireless, and Edge computing, visit Kyndryl's Advisory and Implementation Services at: https://www.kyndryl.com/us/en/services/advisory-implementation-services
Benjamin Brillat is a Senior Technical Staff Member (STSM) and the Global Domain Leader for 5G/Wireless and Edge Computing in Kyndryl's Advisory & Implementation Services. He is responsible for developing the technical solutions and reference architectures to provide wireless network services to corporate campuses, hospitals, mines, factories, stadiums, retailers, and more. Ben has spent over 20 years with IBM/Kyndryl, holds a degree in Computer Science from Rensselaer in Troy, NY, and is a Cisco Certified Internetwork Expert (CCIE) Emeritus.