Introduction to AKS Private Clusters
AKS is the Kubernetes service offering from Azure. It allows you to manage the lifecycle of Kubernetes clusters at scale. By default, an AKS cluster uses a public IP address for the control plane. However, using a public IP address will expose the control traffic in your cluster to the Internet.
If you are concerned about the security threats of exposing control traffic to the Internet, you can create an AKS cluster with a private control plane IP address. However, the absence of a public IP address prevents you from directly connecting to a private AKS cluster from your computer.
Creating an AKS Private Cluster
Let’s create an AKS private cluster via Azure portal.
Step-1
Log in to the Azure portal and click on ‘Create a resource’.
Step-2
Type ‘kubernetes’ in the search bar and click on ‘Kubernetes Service’
Step-3
Click on the ‘Create’ button.
Step-4
Azure portal guides you through a set of steps for creating the cluster. The mandatory parameters ‘Subscription’ and ‘Resource Group’ are used for billing purposes. If you do not have already created them, please create them by clicking on ‘Create new’.
AKS offers several preset configurations for Kubernetes clusters. Click on ‘View all preset configurations’ and select your preferred configuration. If you want to try out AKS, ‘Dev/ Test’ would be quite enough.
The other mandatory parameter at this step is the ‘Kubernetes cluster name’. Leave the defaults for the rest of the parameters and click the ‘Next’ button at the bottom of the screen.
Step-5
In this screen, you can add ‘Node pools’. A node pool is a set Azure VMs with the same configurations. You can add multiple node pools to an AKS cluster depending on the types of workloads you want to run.
Let’s create this cluster with the default node pool that comes with just 3 nodes.
Step-6
Leave the default parameters in the ‘Authentication’ screen and click on ‘Next’
Step-7
In the ‘Networking’ screen select ‘Azure CNI’ as the networking configuration. Azure will be creating a new virtual network and a subnet to accommodate this AKS cluster. Note the name of the virtual network, as we would need it later when launching a VM.
Also, check the ‘Enable private cluster’ check box. This is the key parameter for a private AKS cluster.
Step-8
Leave the default parameters in the ‘Integrations’ and ‘Tags’ screen each and click on ‘Next’.
Step-9
In the ‘Review and Create’ screen, you will see a ‘Validation passed’ checkbox if all your values are correct. If not you may need to go back and correct the problematic parameters. After everything is fixed, click on the ‘Create’ button at the bottom of the screen.
Azure will take several minutes to create the Kubernetes cluster.
Once, the Kubernetes cluster is created, you will get the message ‘Your deployment is complete’.
Connecting to an AKS Private Cluster
Since an AKS private cluster does not use a public IP for the Kubernetes control plane, we cannot directly use ‘kubectl’ from our computer to manage the cluster. Instead, we must either use the Azure Cloud Shell or the Azure CLI installed in an Azure VM to connect to this AKS cluster.
Using the Azure Cloud Shell
The Azure CLI is the command-line interface tool provided by Azure for managing cloud resources. You can install it locally on your PC or can use the Azure Cloud Shell which is the browser-based implementation of the Azure CLI.
Launch the Azure Cloud Shell by clicking the shell icon in the right side of the top toolbar in the Azure portal.
If this is the first time you are using the Cloud Shell, Azure would ask you to create a storage, so go ahead and create it.
First, we must configure the Azure subscription and the Resource Groups in the Cloud Shell. In the Azure portal home, type ‘subscriptions’ in the main search box and select ‘Subscriptions’ from the drop-down menu. In the ‘Subscriptions’ page, copy the
‘Subscription ID’ of the subscription that we used for creating the AKS cluster. Similarly, get the name of the Resource Group as well.
Set the subscription in the CLI.
$ az account set --subscription f764c3e1-a358-4e79-98b5-3f96ac2bc656
Set the credentials for ‘kubectl’ to access the cluster.
$ az aks get-credentials --resource-group cloud-qubes --name private-aks
Azure CLI implements a special command named ‘command invoke’ which can pass ‘kubectl’ commands to a private AKS cluster.
$ az aks command invoke --resource-group cloud-qubes --name private- aks --command "kubectl get namespaces"$ az aks command invoke --resource-group cloud-qubes --name private- aks --command "kubectl get pods -n kube-system"
Using an Azure VM
Alternatively, we can use an Azure VM to access an AKS private cluster. This VM must reside in the same virtual network that is used by the AKS cluster. So, let’s create a VM.
Step-1
From the Azure portal, click on ‘Create resource’ and type ‘Virtual machine’ in the search bar. Then, click on ‘Virtual machine.
Step-2
Click on the ‘Create’ button.
Step-3
In the ‘Basics’ tab select the appropriate ‘Subscription’ and the ‘Resource groups’. Azure sets the username field to ‘azureuser’ by default. This would be required to access the VM by SSH. Provide a name for the VM and select the Ubuntu 20.04 LTS image. Then, click on ‘Next’.
Step-4
Leave the default parameters in the ‘Disks’ screen and click ‘Next’.
Step-5
In the ‘Networking’ screen, select the virtual network and the subnet that we have just created during the AKS cluster deployment.
Step-6
Leave the default parameters in the ‘Management’, ‘Advanced’, and ‘Tags’ screens and click ‘Next’ at each.
Step-7
After getting the ‘Validation passed’ message click on ‘Create’.
Step-8
Download and store the private key that would be required to set up SSH access to the VM.
Step-9
After the deployment is completed, obtain the public IP address assigned to the VM. Login to the VM via SSH using the private key that we downloaded in the earlier step.
Step-10
Install the Azure CLI in the VM.
$ sudo apt-get update$ sudo apt-get install ca-certificates curl apt-transport-https lsb- release gnupg$ curl -sL https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/microsoft.gpg> /dev/null$ AZ_REPO=$(lsb_release -cs)$ echo "deb [arch=amd64] https://packages.microsoft.com/repos/azure- cli/ $AZ_REPO main" | sudo tee /etc/apt/sources.list.d/azure- cli.list$ sudo apt-get update$ sudo apt-get install azure-cli
Step-11
In the VM, set the credentials for ‘kubectl’ to access the cluster.
$ az aks get-credentials --resource-group cloud-qubes --name private-aks
Now, we can access our AKS cluster directly from this host.
$ kubectl get nodes
Integrate Azure Container Registry with the AKS Private Cluster
A container registry provides a location to store container images that are to be deployed
on to Kubernetes. So, let’s create a container registry on Azure.
Step-1
In the Azure portal, go to ‘Create resource’ and type ‘Container registry’ in the search box. Then, click on ‘Container Registry’ that appear in the drop-down menu.
Provide appropriate values for the ‘Subscription’ and the ‘Resource Group’ parameters as you have done previously.
Input a unique name for the ‘Registry Name’. Select ‘Premium’ SKU so that the registry supports private networking. Click on the ‘Next’ button at the bottom of the screen.
Step-2
In the ‘Networking’ screen, select ‘Private access and click on ‘Create private endpoint connection’.
Provide a name for the endpoint and select the virtual network and the subnet that we created during the AKS cluster deployment. Click the ‘OK’ button and click on ‘Next’.
Step-3
Leave the default parameters in the ‘Encryption’ and the ‘Tags’ screens and click on ‘Next’ in each.
Step-4
After you get the ‘Validation passed’ notification, click on the ‘Create’ button. Wait till the deployment is completed.
Step-5
We must integrate this container registry with our AKS cluster so that we can use the registry to store the images to be deployed. We will use the Azure CLI for this purpose.
$ az aks update -n private-aks -g cloud-qubes --attach-acr cloudqubescontainers
Step-6
Then, pull the latest Nginx image to the container registry.
$ az acr import --name cloudqubescontainers --source docker.io/ library/nginx:latest --image nginx:v1
You can verify the availability of the image in the container registry.
$ az acr repository list --name cloudqubescontainers ["nginx"]
In the Azure portal go to the container registry and click on ‘Repositories’. The Nginx image that we just downloaded would be listed there. Click on ‘v1’ tag and note the value of the
artifact reference. We shall be using that value to refer the image while creating a deployment with Helm charts.
Creating a Deployment with Helm charts
Step-1
Install Helm in the VM that we created earlier.
$ curl https://baltocdn.com/helm/signing.asc | sudo apt-key add -$ sudo apt-get install apt-transport-https —yes$ echo "deb https://baltocdn.com/helm/stable/debian/ all main"| sudo tee /etc/apt/sources.list.d/helm-stable-debian.list$ sudo apt-get update$ sudo apt-get install helm
Step-2
Initialize a new Helm repository.
$ helm create my-nginx-app
This will create a new helm chart in the folder ‘my-nginx-app’.
Step-3
In the ‘my-nginx-app/values.yaml’ file, under the image directive, update the parameters as below.
The ‘repository’ and the ‘tag’ parameters are set according to the ‘Artifact reference’ that we noted earlier.
image:repository: cloudqubescontainers.azurecr.io/nginxpullPolicy: Always
# Overrides the image tag whose default is the chart appVersiontag: "v1"
Step-4
Create a new deployment with our helm chart.
$ helm install my-nginx-app ./my-nginx-app/ --set service.type=NodePort
Step-5
List the pods in our deployment.
$ kubectl get pods
Use case of AKS private clusters
An AKS public cluster uses a public IP address for the Kubernetes API server so that it is directly accessible from your computer via the Internet. Therefore, it is also easier to administrate.
However, a public AKS cluster poses two main security risks. Since it is accessible via the Internet, a malicious user can attempt to gain access to the cluster. Also, the control traffic from nodes to API server traverses the Internet opening up the possibilities for man-in-the-middle attacks.
Therefore, certain organizations have enforced security policies that require all Kubernetes administrative traffic to be separated from the Internet. AKS private clusters can satisfy this requirement and it is the main use case of AKS private clusters.
Conclusion
In this article, we have created a private cluster on AKS and demonstrated two ways of accessing the cluster. Following these methods, you can keep all your Kubernetes control traffic within a private network thus improving the security of the overall architecture.