Secure your AKS deployments - microservice user authentication with Azure AD and OAuth 2 proxy (2023)

Following mineprevious blog postSince we cover SSL termination and NGINX, in this post we'll extend our implementation to now include user authentication of the new web application.

As with every other article in this series, this one is driven by user cases. In this case, the customer wanted a development web application in the public domain, but limited to allow access to only authenticated users from Azure AD.

Before diving deeper into the use case and implementation, it is important to understand the various components in case they are unfamiliar. I'll cover the benefits of specific technologies along the way, but it's worth taking a quick look at these links as a set of levels if needed:

• NGINX-Ingress-Controller –https://docs.nginx.com/nginx-ingress-controller/intro/overview/
• Azure Kubernetes Service –https://azure.microsoft.com/en-us/products/kubernetes-service
• Azure Key Vault –https://azure.microsoft.com/en-gb/products/key-vault
• OAuth2 -https://auth0.com/intro-to-iam/what-is-oauth-2
• OAuth2-Proxy –https://oauth2-proxy.github.io/oauth2-proxy/
• Azure Active Directory –https://azure.microsoft.com/en-us/products/active-directory#overview

use case

As previously mentioned, a customer wanted to add authentication to their development applications. There are many other reasons you might want to add user authentication to your application, such as any application that wants to provide users with different content or functionality based on a mapped property. Learn more about how scope and permissions can be found in the Microsoft Identity PlatformHere.

In this case, the customer wanted their production codebase and development codebase to be as similar as possible. Could not use the Microsoft authentication library.

This got me thinking about the developer effort involved in implementing application-level authentication. I started talking to a few developers I know and found a common theme summed up eloquently by one developer who said, "I just want to spend my time developing value-added features, why do I need to worry about authentication." ?"

I agree that authentication is an infrastructure and security issue in most cases.

Moving your authentication outside of your application and into middleware has two distinct advantages:

1. Developer effort –Developers can spend less time implementing authentication and more time working on features that add value to your business and customers.
2. Application workload –Offloading authentication to dedicated middleware reduces the processing load of the computing power hosting your application.

However, this is not appropriate for every application or every business. It may still be the case that developers remain responsible for authentication after they leave the application. In this case, it can actually increase the developer's workload.

ACCORDING TO

It's worth noting that OAuth2 Proxy was not the first authentication middleware I worked with while creating this demo and blog. At first I usedDAPR's Oauth2 middleware component.This component is promising and usefultrailer architecturesecure your authentication component. This is great because it allows you to configure authentication for each microservice separately. However, unfortunately, the DAPR authentication component is still in alpha and exhibits unexpected behavior. I raised a relevant questionHere, hopefully this component will be something that will be revisited in the future.

implementation

I will be using Azure Key Vault to store some sensitive data used in this blog. Below I will briefly explain how to configure and set it up for AKS. This is not strictly necessary for setting up Oauth2 in AKS and adds unnecessary complexity if built as a POC/demo. Feel free to skip this section if it is not relevant to you.

I will also build on the architecture created in mineoriginal blog post. It doesn't necessarily have to be completed before this implementation, but it gives context.

I changed the application being deployed from my boring API to a rather exciting web application that he developedMark Harrison, a senior specialist here at Microsoft UK.

We will deploy the web application with an Oauth2 reverse proxy to ensure that only authenticated users can access the web application. We will also provide a web application API that does not use an OAuth proxy and is only accessible from within the cluster. For more clarity, see the floors in the architecture at the top of the page.

requirements

• Create an Azure Kubernetes service cluster.
• Create Azure Key Vault
• Install OpenSSL

Optional

• Custom domain
• Copying the architecture from the first post

The full implementation files for this post can be found at:https://github.com/owainow/microservice-authentication-oauth2-aks

Azure Active Directory - Register the application

Since we are authenticating users accessing our application through Azure Active Directory, we need to create itapplication registrationin our directory.

These registrations can be thought of as an application definition and are objects that describe the application to Azure AD. To create our application registration, we will go through the portal in Azure Active Directory.

When registering an application, we need to share some information, all of which may change after the application registration is created. It is important to emphasize the redirect URI here because it redirects users after authentication is complete. We will pass the callback URL to be used later when configuring our OAuth2 proxy.

Update the redirect URI according to your domain and protocol.

https:///oauth2/callback

Once your application has been created, note the Application ID (Client ID) on the summary page. We'll need it later.

As part of registering our app, we also need to create a client secret that will be used to identify our app. We could also use a certificate for more security, but in this example the secret is sufficient.

Make a note of your client secret when it is created, as it can only be seen once (you can create a new secret if you lose it).

First of all, these are all the settings needed to register the app. However, some additional features are worth highlighting. This app registration allows you to create custom branding for your login to provide an integrated experience with the rest of your app.

API permissions should also be considered. By default,Microsoft Graphis added for this application to allow basic user data to be retrieved at login. Additional permissions can be added if needed for the application. However, the user must consent to the application's use of this data when logging in for the first time.

Azure Key Vault

In this scenario, we use Azure Key Vault to protect our secrets when used by our AKS applications.

In this demo we will usesecure cookiesand as a result we need to create a secret cookie. The following command allows us to create a cookie secret using OpenSSL.

export cookie_secret="$(openssl rand -hex 16)" # Create a local variable echo $cookie_secret # Check the output bbf240d482fc7236cd0bf01cec54422d

Now we need to set the secrets in the key vault. To do this, we will use the Azure CLI to save time. This can also be done through the portal. First, make sure you are logged in and have the appropriate subscription. Then run:

az keyvault Secret Set --vault-name "aks-zero-trust-kv" --name "oauth2-proxy-client-id" --value ""az Keyvault Secret Set --vault- name „aks-zero-trust-kv“ --name „oauth2-proxy-client-secret“ –vrijednost „“az keyvault Secret Set --vault-name „aks-zero-trust-kv“ – -name "oauth2-cookie-secret" --value $cookie_secret

If we then check our key vault, we should see our secrets:

Azure Kubernetes service

Now we need to deploy our applications and components to our Kubernetes cluster.

Azure Key VaultIntegration

First we need to enable thisAzure Secret Store CSI Driveron our cluster if we didn't enable it when we created it. We can do this with the following command:

az aks enable-addons --addons azure-keyvault-secrets-provider --name myAKSCluster --resource-group myResourceGroup

We can then verify the installation by running:

kubectl get pods -n kube-system -l 'app in (secrets-store-csi-driver,secrets-store-provider-azure)

Now that our driver is running on our cluster, we need to decide how to allow our AKS cluster to access our key vault. We have several options for this:

• Service manager
• AAzure Active Directory - Workload Identity
• A managed identity assigned to a user or system

In this implementation, we will use user-assigned managed identities, although the workload identity will be GA very soon. I encourage you to take a moment to look at the difference between a user-assigned managed identity, which we'll be using today, and a workload identity.

In the future I will edit these deployment files and blog to also show how Workload Identity can be leveraged.

To create a managed identity for our cluster, we can use the following command:

az aks update -g-n--enable-managed-identity

We can then request the Identity Client ID that was created for us.

az aks show -g-n--query addonProfiles.azureKeyvaultSecretsProvider.identity.clientId -o tsv

Next, we need to add this client ID to our key vault with the appropriate permissions:

# Set the key access policy in your key vault keyvault set-policy -n--key-permissions dobiti --spn# Set policies for accessing secrets in your key vault az keyvault set- Policy -n--secret-permits to gain --spn# Set rules for accessing certificates in your key vault. keyset rule -n--certificate-permissions dobiti --spn

You can use the portal to check whether permissions are set on your key vault at any time.

Now we need to create a SecretProviderClass. The Secret Provider class accesses our KeyVault using the managed identity we just created. Since the OAuth2 proxy also requires our secrets to be passed as environment variables, we'll include some secret objects in this file. There are two important things to keep in mind here:

1. Secrets of Kubernetes– Basically, these key vault secrets are still passed as Kubernetes secrets in this case because we pass them as environment variables. For some environments this may not be secure enough. Alternatively, we could provide those secrets in a module and direct them to the mount point. However, it still carries risks. We still benefit from being able to rotate, update and deactivate secrets from Azure Key Vault.
2. Secret Sync –The great thing about the secret provider class is that you can keep the secrets in sync with the version in your key vault regardless of how you access the secrets. The secret is created when the capsule is placed. The latest version from the key vault is currently being used. However, the Secret Provider class does not restart application pods that are already running.

The secret provider class requires a managed identity client ID and the tenant ID of your key vault (Github-link)

apiVersion: v1 Type: Namespace # We split our application and API into multiple namespaces for later use. Metadata: Tags: app.kubernetes.io/name: Colors Name: Colors-web---apiVersion: Secrets-Store.csi.x -k8s.io/v1kind: SecretProviderClassmetadata: Name: azure-aks-zero-trust-user -msi # must be unique per namespace Namespace: Colors-Webspec: Provider: Azure SecretObjects: # SecretObjects defines the desired state of synchronized K8s secret objects - SecretName: client-id type: opaque data: - Object Name: oauth2-proxy- client -id key: oauth2_proxy_client_id - SecretName: client secret type: opaque data: - object name: oauth2-proxy-client-secret key: oauth2_proxy_client_secret - SecretName: cookie secret type: opaque data: - object name: oauth2-proxy-cookie - secret Key: oauth2_proxy_cookie_secret Parameters: usePodIdentity: "false" useVMManagedIdentity: "true" userAssignedIdentityID:keyvaultName: aks-zero-trust -kv # Set cloudName to your key vault name: "" # [OPTIONAL for Azure] If not specified, the Azure environment uses AzurePublicCloud objects by default: | string: - | Object Name: oauth2-proxy-client-id Object Type: Secret # Object Types: Secret, Key, or Certificate Object Version: "" # [OPTIONAL] Object versions, defaults to latest version if empty - | Object Name: oauth2-proxy-client-secret Object Type: Secret # Object Types: Secret, Key, or Certificate Object Version: "" # [OPTIONAL] Object versions, defaults to latest version if empty - | Object Name: oauth2-proxy-cookie-secret Object Type: Secret # Object Types: Secret, Key, or Certificate Object Version: "" # [OPTIONAL] Object versions, defaults to latest version if empty. Tenant ID:# Tenant ID from your key safe

Then we need to apply the secret provider class:

kubectl apply -f Secretproviderclass.yaml

It is currently worth inspecting the secrets and noticing that the defined objects are not currently being created. This is because, as mentioned earlier, they are created when the application needs them.

Implementation of the application

Now we need to deploy our application. In this example, we will implement two applications. Web application and API. First, let's implement the API with the following manifest (Github-link)

apiVersion:v1kind:namespacemetadata:labels:app.kubernetes.io/name:colors name:colors-api---apiVersion:apps/v1kind:deploymentmetadata:name:colors-api-depl namespace:colors-apispec:replicates:1 selector : matchLabels: Application: Colors API Service Template: Metadata: Labels: Application: Colors API Service Specification: Container: - Name: Colors API Image Image: ghcr.io/markharrison/colorsapi:latest imagePullPolicy: Always Resources: Requirements: Memory : " 128Mi" CPU: "500m" Limits: Memory: "256Mi" CPU: "1000m" ---apiVersion: v1kind: Servicemetadata: Name: Colors-api-srv Namespace: Colors-apispec: Type: ClusterIP Selector: App: Colors-API-Service-Ports: - Name: Colors-API-Service-HTTP-Protocol: TCP-Port: 80 DestinationPort: 80

Now we apply this manifest:

kubectl apply -f farbenwebapi.yaml

Then we need to deploy the web application with the following manifest (Github-link):

apiVersion: v1 vrsta: namespacemetadata: labels: app.kubernetes.io/name: colors name: colors-web---apiVersion: apps/v1kind: deploymentmetadata: name: colors-web-depl namespace: colors-webspec: replike: 1 Selektor: matchLabels: Aplikacija: Colors-Web-Service-Template: Metapodaci: Oznake: App: Colors-Web-Service-Specification: Container: - Naziv: Colors-Web-Image Slika: ghcr.io/markharrison/colorsweb:latest imagePullPolicy : Uvijek Resursi: Zahtjevi: Memorija: "128Mi" CPU: "500m" Ograničenja: Memorija: "256Mi" CPU: "1000m" ------apiVersion: v1kind: Servicemetadata: Naziv: Colors-web-clusterip-srv Namespace :colors -webspec:type:clusterip selector:app:colors-web-service-ports:-name:colors-web-service-http-protocol:tcpport:80 destinationport:80

Note that oauth2 is not mentioned yet. We do not share any information with the app and the app has no built-in user authentication.

Now we need to create two components of our Oauth2 container to manage authentication and our input resource. First, we will implement an OAuth2 application.

The implementation looks like this (Github-link)

apiVersion: apps/v1kind: Deploymentmetadata: Labels: Application: Colors-Service-Oauth2-Proxy-Name: Colors-Service-Oauth2-Proxy-Deployment-Namespace: Colors-Webspec: Replica: 1 Selector: MatchLabels: Application: Colors-Service -Oauth2 -Proxy-Template: Metadata: Tags: Application: Colors-Service-Oauth2-Proxy-Specification: Containers: - Arguments: - --provider=oidc - --azure-tenant=# Azure AD OAuth2 proxy app tenant ID - --pass-access-token=true - --cookie-name=_proxycookie - --upstream=- --cookie-csrf-per-request=true - --cookie-csrf-expire=5m # Avoid unauthorized CSRF cookie failures. - --email-domain=* # Email domains allowed to use proxy - --http-address=0.0.0.0:4180 - --oidc-issuer-url=https://login.microsoftonline.com //v2.0 - --user-id-claim=oid name: Colors-service-oauth2-proxy image: quay.io/oauth2-proxy/oauth2-proxy:v7.4.0 imagePullPolicy: Always volumeMounts: - name: Secrets - store01-inline mountPath: "/mnt/secrets-store" readOnly: true env: - Name: OAUTH2_PROXY_CLIENT_ID # Keep this name - it must be defined as such by the OAuth2 proxy. valueFrom: SecretKeyRef: Name: Client-ID Key: oauth2_proxy_client_id - Name: OAUTH2_PROXY_CLIENT_SECRET # Keep this name - must be defined by OAuth2 proxy Must be defined by OAuth2 proxy. valueFrom: SecretKeyRef: Name: Cookie Secret Key: oauth2_proxy_cookie_secret Ports: - ContainerPort: 4180 Protocol: TCP Resources: Limits: CPU: 100M Memory: 128Mi Requirements: CPU: 100M Memory: 128Mi Volumes: - Name: Secrets-Store01-Inline CSI: Driver: Secrets-Store.csi.k8s.io ReadOnly: True VolumeAttributes: SecretProviderClass: "Azure-AKS-Zero-Trust-User-Msi"---APIVersion: V1Kind: Servicemetadata: Labels: Application: Colors-Service-Oauth2- Proxy Name: Colors-Service-Oauth2-Proxy-SVC Namespace: Colors-Webspec: Ports: - Name: http-Port: 4180 Protocol: TCP TargetPort: 4180 Selector: Application: Colors-Service -oauth2-proxy---apiVersion: networking.k8s.io/v1kind: Ingressmetadata: Notes: nginx.ingress.kubernetes.io/ssl-redirect: "true" nginx.ingress.kubernetes.io/proxy-body-size: "2000m" nginx.ingress.kubernetes. io/proxy-buffer-size: "32k" Name: Colors-service-oauth2-proxy-ingress Namespace: Colors-webspec: ingressClassName: Nginx Rules: -http:paths: -path: /oauth2 PathType: Prefix Backend: Service: Name: Colors-Service-oauth2-proxy-svc Port: Number: 4180

In this implementation, we refer to the Secret objects contained in the Secrets provider, which are created after we apply this manifest. We can also see that we are using Secrets notebooks. The volumes determine the CSI and class of the secret provider, and the volume carrier then distributes the secrets to the pod. This allows the implementation to create secrets.

After you add IDs for your specific implementation, we can deploy that implementation.

kubectl implements oauth2proxy.yaml

If we seek our secrets, we should now see that secrets have been created.

REDMOND+owaino@DESKTOP-9V6KSRB MINGW64 ~/Documents/Azure-Demo-Projects/AKS-Zero-Trust (main$ kubectl get SecretsNAME TYPE DATA AGEclient-id opaque 1 2sclient-secret opaque 1 2scookie-secret opaque 1 2stls-secret kubernetes .io/tls 2 6d17h)

Now we will set up Ingress. If you follow the previous post, we will replace the existing Ingress. Only ifInstall the NGINX Ingress Controlleron your cluster.

The input manifest is as follows (Github-link)

apiVersion: networking.k8s.io/v1kind: Ingressmetadata: Naziv: ingress-srv Namespace: Colors-Web Anmerkungen: kuberentes.io/ingress.class: nginx nginx.ingress.kubernetes.io/use-regex: 'true' nginx. ingress.kubernetes.io/affinity: „kolačić“ nginx.ingress.kubernetes.io/session-cookie-name: „route“ nginx.ingress.kubernetes.io/session-cookie-hash: „sha1“ nginx.ingress.kubernetes .io/session-cookie-expires: „172800“ nginx.ingress.kubernetes.io/session-cookie-max-age: „172800“ nginx.ingress.kubernetes.io/proxy-connect-timeout: „360“ nginx. ingress.kubernetes.io/proxy-send-timeout: „360“ nginx.ingress.kubernetes.io/proxy-read-timeout: „360“ nginx.ingress.kubernetes.io/proxy-body-size: „2000m“ nginx .ingress.kubernetes.io/proxy-buffer-size: „32k“ nginx.ingress.kubernetes.io/ssl-redirect: „true“ nginx.ingress.kubernetes.io/auth-url: „https://www. owain.online/oauth2/auth“ nginx.ingress.kubernetes.io/auth-signin: „https://www.owain.online/oauth2/start?rd=https://www.owain.online/oauth2/callback " nginx.ingress.kubernetes.io/auth-response-headers: „Authorization, X-Auth-Request-Email, X-Auth-Request-User, X-Forwarded-Access-Token“ Oznake: Aplikacija: Colors-Web- Specifikacija usluge: ingressClassName: nginx tls: - Hostovi: - owain.online SecretName: Test-TLS-Regeln: - Host: owain.online - http: Pfade: - Pfad: / pathType: Präfix Backend: Dienst: Naziv: Colors-Web- Clusterip -srv-Port: Broj: 80

There are many comments about Nginx ingress. Three that should be highlighted are:

nginx.ingress.kubernetes.io/auth-url: „https://www.owain.online/oauth2/auth“nginx.ingress.kubernetes.io/auth-signin: „https://www.owain.online/ oauth2/start?rd=https://www.owain.online/oauth2/callback"nginx.ingress.kubernetes.io/auth-response-headers: "Autorisierung, X-Auth-Request-E-mail, X-Auth-Request -Benutzer, X-Forwarded-Access-Token“

The authentication URL specifies the URL that your requests are also forwarded to when they arrive in this inbox. Here we use the /oauth2 endpoint, which our Oauth proxy supports. Auth-Sign-In points to the start URL of the authentication flow and passes our callback URL. Finally, the authentication response headers allow us to specify which values ​​from our authorization we want to pass for use by the application.

These are the basic notes required for NGINX external authentication, but I encourage you to do itTake a look at the wider spectrum, some of which are very powerful.

Now that we understand the notes related to authentication, we can apply our input:

kubectl apply -f ingress-srv.yaml

Now we can verify that our units are deployed and working as expected:

REDMOND+owaino@DESKTOP-9V6KSRB MINGW64 ~/Documents/Azure-Demo-Projects/AKS-Zero-Trust (main)$ kubectl get podsNAME READY STATUS RESTARTS AGEcolors-service-oauth2-proxy-deployment-78ffb756f5-sd4v5 1/1 Will pokrenuto 0 16mcolors-web-depl-554b54449c-ntflf 1/1 pokrenuto 2 (prije 5d18h) 6d17hREDMOND+owaino@DESKTOP-9V6KSRB MINGW64 ~/Documents/Azure-Demo-Projects/AKS-Zero-Trust (main)$ kubectl get pods - n boje-apiNAME READY STATUS RESTARTD AGEcolors-api-depl-79c887f867-vhgg9 1/1 trčanje 0 13 m

Assuming you don't see any errors, you should now be able to navigate to the domain or IP address you used to configure this deployment and be greeted by the Azure AD login screen at the route you specified.

As soon as we authenticate as a user in your Azure AD directory, we will be greeted by Mark's excellent color application.

We have now managed to differentiate our web application with Azure AD authentication without the need for any code changes and with Azure Key Vault integration!

Finally it's time to configure our app to see what it does and also point out that we can now access our internal API without exposing it to the public.

To configure the application, we need the FQDN of the API service that we provided earlier. Since Kubernetes doesn't restrict traffic between pods or namespaces by default, we can specify the service name of our API for internal calls. Since our API service is not connected to the Internet, we need to uncheck the box so that our calls are made through the modules that run our application, not through the client.

We also need to include the route to the API, which in this case is /colors/random, but you can look at another routeOptions available here.

To get the FQDN of our service we can run the following commands:

kubectl exec -it -n colors-apideployment/colors-api-depl -- apt-get ažuriranje -ykubectl exec -it -n colors-apideployment/colors-api-depl -- apt-get instalacija dnsutils -ykubectl exec -it - n colors-api-deployment/colors-api-depl --nslookup colors-api-srv

The output includes our FQDN:

REDMOND+owaino@DESKTOP-9V6KSRB MINGW64 ~/Documents/Azure-Demo-Projects/AKS-Zero-Trust (main$ kubectl exec -it -n colors-apideployment/colors-api-depl -- nslookup colors-api-srvServer : 10.0.0.10Adresa: 10.0.0.10#53Naziv: farben-api-srv.colors-api.svc.cluster.localAddress: 10.0.254.92)

Now let's configure the application:

We need to add an API route and uncheck the direct calls box:

http://colors-api-srv.colors-api.svc.cluster.local/colors/random

Diploma

If you're following the first blog, it's worth taking another look at the architecture to see again what's been implemented, including SSL termination!

Using an Oauth2 reverse proxy, we were able to authenticate at the ingress layer. Although this application has nothing to do with the headers passed to it, you can now easily set feature tags or unique user content based on the authentication information passed in the headers.

In my next blog post, I'll look at network policies and Open Service Mesh to explore how we can leverage various features to throttle network traffic, enable mTLS, and manage visibility.

As mentioned earlier, the yaml manifests included in this post are available at:https://github.com/owainow/microservice-authentication-oauth2-aks