Using CNI Cilium VPC Native Routing
Overview
CNI (Container Network Interface) Cilium VPC Native Routing is a mechanism that helps Kubernetes manage networks without using overlay networks. Instead of using virtual network layers, CNI Cilium VPC Native Routing leverages the direct routing capabilities of cloud service providers' VPCs (Virtual Private Clouds) to optimize data transfer between nodes and pods in the Kubernetes cluster.
Model
On VKS, CNI (Container Network Interface) Cilium VPC Native Routing operates according to the following model:
In there:
Each Node has a private IP address range for pods (Pod CIDR). Pods in each node use addresses from this CIDR and communicate over the virtual network.
Cilium and eBPF perform network management for all pods on each node, including handling traffic going from pod to pod, or from node to node. When necessary, eBPF performs masquerading to hide the internal IP address of the pod when communicating with the external network.
Cilium ensures that pods can communicate with each other both within the same node and between different nodes.
Necessary conditions
To be able to initialize a Cluster and Deploy a Workload , you need:
There is at least 1 VPC and 1 Subnet in ACTIVE state . If you do not have any VPC, Subnet, please initialize VPC, Subnet according to the instructions below:
Step 1: Access the vServer homepage at the link https://hcm-3.console.vngcloud.vn/vserver
Step 2: Select the VPCs menu in the left menu of the screen.
Step 3: Here, if you don't have any VPC yet, please select Create VPC by entering the VPC name and defining the desired CIDR/16 range.
Step 4: After having at least 1 VPC, to create a subnet, you need to select View Detail to expand the control panel at the bottom, including the Subnet section.
Step 5: In the Subnet section, select Add Subnet. Now, you need to enter:
Subnet name: the subnet's mnemonic name
Primary CIDR : :This is the primary IP address range of the subnet. All internal IP addresses of virtual machines (VMs) in this subnet will be taken from this address range. For example, if you set Primary CIDR to 10.1.0.0/16, the IP addresses of the VMs will be in the range of 10.1.0.1 to 10.1.255.254.
Secondary CIDR : This is a secondary IP address range, used to provide additional IP addresses for specific purposes such as alias IP ranges or to separate different services within the same subnet. Suppose you can add a Secondary CIDR of 10.2.0.0/20 to use for different services or applications without creating a new subnet.
Attention:
The IP address ranges of Primary CIDR and Secondary CIDR cannot overlap. This means that the address range of Secondary CIDR must be outside the range of Primary CIDR and vice versa. For example, if Primary CIDR is 10.1.0.0/16, then Secondary CIDR cannot be 10.1.0.0/20 because it is within the range of Primary CIDR. Instead, you can use a different address range like 10.2.0.0/20.
There is at least 1 SSH key in ACTIVE state . If you do not have any SSH key, please initialize SSH key following the instructions here .
kubectl installed and configured on your device. please refer here if you are not sure how to install and use kuberctl. In addition, you should not use an outdated version of kubectl, we recommend that you use a kubectl version that is no more than one version different from the cluster version.
Create a Cluster using CNI Cilium VPC Native Routing
To initialize a Cluster, follow the steps below:
Step 1: Access https://vks.console.vngcloud.vn/overview
Step 2: On the Overview screen , select Activate.
Step 3: Wait until we successfully initialize your VKS account. After successfully Activating, select Create a Cluster.
Step 4: At the Cluster initialization screen, we have set up the information for the Cluster and a Default Node Group for you. To use CNI Cilium VPC Native Routing for your Cluster , please select:
Network type : Cilium VPC Native Routing and other parameters as follows:
Calculating the number of IPs for pods and nodes:
Suppose, when initializing the cluster, I choose:
VPC : 10.111.0.0/16
Subnet:
Primary IP Range: 10.111.0.0/24
Secondary IP Range: 10.111.160.0/20
Node CIDR mask size: Selectable values range from /20 to /26 .
Therefore:
The larger the node CIDR mask size (eg /20 , /21 ), the fewer nodes you will have but each node will have more IPs for pods.
Smaller node CIDR mask size (eg /24 , /25 ), you can create more nodes but each node will have less IP addresses for pods.
Attention:
Only one networktype: In a cluster, you can use only one of three networktypes: Calico Overlay, Cilium Overlay, or Cilium VPC Native Routing
Multiple subnets for a cluster: VKS supports the use of multiple subnets for a cluster. This allows you to configure each node group in the cluster to be located on different subnets within the same VPC, helping to optimize resource allocation and network management.
Cilium VPC Native Routing and Secondary IP Range : When using Cilium VPC Native Routing for a cluster, you can use multiple Secondary IP Ranges. However, each Secondary IP Range can only be used by a single cluster. This helps avoid IP address conflicts and ensures consistency in network management.
When there are not enough IP addresses in the Node CIDR range or Secondary IP range to create more nodes, specifically:
If you cannot use the new Node because of running out of IP addresses in the Secondary IP range . At this time, new nodes will still be created and joined to the cluster but you cannot use them. The pods that are required to launch on this new node will be stuck in the " ContainerCreating " state because they cannot find a suitable node to deploy. Solution : You need to expand the Secondary IP range (if your network infrastructure allows) or adjust the cluster configuration (increase the number of CIDR nodes).
Not enough IP addresses for Pods : If a node is initialized but does not have enough IP addresses to allocate to new pods, pods will not be able to launch on that node. VKS will not allocate more pods to that node, and the pod will enter the " ContainerCreating " state until resources or IPs are available. Solution : You can:
Increase the number of nodes to reduce load and increase the amount of IP for pods.
Optimize Node CIDR mask size so that each node has more IPs.
Step 5: Select Create Kubernetes cluster. Please wait a few minutes for us to initialize your Cluster, the status of the Cluster is now Creating .
Step 6: When the Cluster status is Active , you can view Cluster information and Node Group information by selecting Cluster Name in the Name column .
Deploy a Workload
Below are instructions for deploying an nginx deployment and testing IP assignment for the pods deployed in your cluster.
Step 1: Access https://vks.console.vngcloud.vn/k8s-cluster
Step 2: The Cluster list is displayed, select the icon Download and select Download Config File to download the kubeconfig file. This file will give you full access to your Cluster.
Step 3 : Rename this file to config and save it to the ~/.kube/config folder
Step 4: Perform Cluster check via command:
Run the following command to check the node
If the result is as below, it means your Cluster is successfully initialized with 3 nodes:
Continue by running the following command to check the pods deployed on your kube-system namespace:
If the result is as below, it means that the pods supporting Cilium VPC Native have been successfully run:
Step 2: Deploy nginx on the newly created cluster:
Initialize the nginx-deployment.yaml file with the following content:
Perform this deployment via command:
Step 3: Check the deployed nginx pods and the IP address assigned to each pod
Perform a check of the pods via the command:
You can observe below, the nginx pods are assigned IPs 10.111.16x.x which satisfy the Secondary IP range and Node CIDR mask size conditions that we specified above:
You can also perform a detailed description of each pod to check this pod information via the command:
Step 4: There are a few steps you can take to thoroughly test the performance of Cilium. Specifically:
First, you need to install Cilium CLI following the instructions here .
After installing Cilium CLS, check the status of Cilium in your cluster via the command:
If the result is displayed as below, it means that Cilium is working properly and fully :
Step 5: You can perform a healthy check to check Cilium in your cluster
Run the following command to perform a healthy check
Reference results
Additionally, you can also perform additional End-to-End connectivity tests or Network performance tests following the instructions at End-To-End Connectivity Testing or Network Performance Test .
Step 6: Check the connection between Pods
Perform a connectivity test between pods, ensuring that the pods can communicate via the VPC IP address without going through overlay networks . For example, below I perform a ping from the pod nginx-app-7c79c4bf97-6v88s with IP address: 10.111.161.53 to a server in the same VPC with IP address: 10.111.0.10:
If the result is as follows, the connection is successful:
Last updated
