Kyma runtime stories. mTLS routes made easy (-ier) with APIRule v2

Eventually, following the official announcement, the lastest istio-based APIRule in the stable version v2 is out.

Kyma Runtime – API Gateway module: Update to version 3.0.2

This version of the API Gateway module introduces APIRule CR in the stable version v2.

The new CRD allows you to expose your workloads using one of the three supported access strategies: jwt, noAuth, and extAuth. The noAuth access strategy provides a simple configuration for exposing workloads over the specified HTTP methods. The jwt access strategy allows you to secure your workload by defining Istio JWT configuration and the extAuth access strategy allows for providing custom authentication and authorization logic. For more information, see APIRule Custom Resource

Additionally, version 3.0.2 contains the following fixes:We’ve fixed the behaviour of in-cluster connectivity when using JWT handler in APIRule. Now, in-cluster connectivity is blocked.Now, when you create an APIRule with the noAuth access strategy, it is validated if the target workload has an Istio sidecar proxy injected.We’ve added a deprecation message, which is displayed when you use the k8s API to interact with APIRule v1beta1 CRs. Now, the following deprecation message is presented during each call: Warning: Version v1beta1 of APIRule is deprecated and will be removed in future releases. Use version v2 instead.

Until now, customers and developers were invited to test the istio-based APIRule in the version v2alpha1.

Good to know:

The promotion of APIRule manifests from version v2alpha1 to version v2 merely consists of manually changing apiVersion from v2alpha1 to the stable version v2 in APIRule manifests.As depicted above, for convenience, Kyma dashboard will display the APIRules CRs grouped accordingly to their apiVersion.

APIRule v2 and mTLS routes

The APIRule v2 is istio-based. What that means is that both authentication and authorisation (permissions) are handled natively through istio.

Let me try to demonstrate how and whether it can help to ease the implementation effort of the mTLS routes.

Good to know:

Albeit the mTLS routes are mainly used for system to system communications, they can also be used for the principal propagation sake with short-lived client certificates.For reference, the basics of mTLS communications with kyma runtime environment are covered in the following blogpost, namely: Developing enterprise-grade applications with mutual TLS easy with SAP BTP, Kyma runtime and BTP destinations

When implementing mTLS routes one needs to forward the client certificate to the server side. This is achieved by requesting the istio SSL headers that will be forwarded to the server side, for instance:

request:
headers:
X-CLIENT-SSL-CN: ‘%DOWNSTREAM_PEER_SUBJECT%’
X-CLIENT-SSL-ISSUER: ‘%DOWNSTREAM_PEER_ISSUER%’
X-CLIENT-SSL-SAN: ‘%DOWNSTREAM_PEER_URI_SAN%’
test: ‘true’

Let’s consider the following APIRule v2 manifest:

apiVersion: gateway.kyma-project.io/v2
kind: APIRule
metadata:
labels:
app.kubernetes.io/name: httpbin-mtls-${namespace}

Good to know:

Any custom headers can be included there and they will be forwarded to the server side as well.This is how it looks like in GUI of APIRule in the kyma dashboard, as depicted below:

The above APIRule definition is translated into auto-generated istio VirtualService and the AuthorizationPolicy resource definitions, namely:

VirtualService

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
labels:
apirule.gateway.kyma-project.io/v1beta1: httpbin-mtls-${namespace}.montypython

spec:
gateways:
– ${namespace}/quovadis-${namespace}-gateway-mtls
hosts:
– httpbin-mtls-${namespace}.mtls-quovadis-<id>.quovadis.kyma.dev.sap
http:
– headers:
request:
set:
X-CLIENT-SSL-CN: ‘%DOWNSTREAM_PEER_SUBJECT%’
X-CLIENT-SSL-ISSUER: ‘%DOWNSTREAM_PEER_ISSUER%’
X-CLIENT-SSL-SAN: ‘%DOWNSTREAM_PEER_URI_SAN%’
test: ‘true’
x-forwarded-host: >-
httpbin-mtls-${namespace}.mtls-quovadis-<id>.quovadis.kyma.dev.sap
response:
remove:
– Access-Control-Allow-Origin
– Access-Control-Expose-Headers
– Access-Control-Allow-Headers
– Access-Control-Allow-Credentials
– Access-Control-Allow-Methods
– Access-Control-Max-Age
match:
– method:
regex: ^(GET)$
uri:
prefix: /
route:
– destination:
host: httpbin.montypython.svc.cluster.local
port:
number: 8000
weight: 100
timeout: 300s

AuthorizationPolicy

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
labels:
apirule.gateway.kyma-project.io/v1beta1: httpbin-mtls-${namespace}.montypython
gateway.kyma-project.io/hash: montypython.47fnidc80q3pr.65q2a0rkmeuic
gateway.kyma-project.io/index: ‘0’

Good to know:

The current design of the APIRule implies having multiple AuthorizationPolicy objects, one per path.The above AuthorizationPolicy guarantees a given workload can only be accessed via an istio ingress gateway, with the advertised HTTP method(s) and from the advertised paths.

Smoke-testing of the mTLS route

Good to know:

The smoke test using curl command is described here

Assuming we have a CA-signed client certificate at hand, we can test the XFF headers being forwarded to the httpbin workload running on a kyma cluster, as follows:

curl https://httpbin-mtls-${namespace}.mtls-quovadis-<id>.quovadis.kyma.dev.sap/headers –cert poster.x509 –key poster.key | jq .{
“headers”: {
“Accept”: “*/*”,
“Host”: “httpbin-mtls-${namespace}.mtls-quovadis-<id>.quovadis.kyma.dev.sap”,
“Test”: “true”,
“User-Agent”: “curl/8.7.1”,
“X-Client-Ssl-Cn”: “CN=poster-quovadis (P000000),L=poster.***.demo.sap,OU=37d98dec-719a-4137-9ed7-***,OU=SAP Cloud Platform Clients,O=SAP SE,C=DE”,
“X-Client-Ssl-Issuer”: “CN=SAP Cloud Platform Client CA,OU=SAP Cloud Platform Clients,O=SAP SE,L=EU10,C=DE”,
“X-Envoy-Attempt-Count”: “1”,
“X-Envoy-External-Address”: “**.214.184.**”,
“X-Forwarded-Client-Cert”: “Hash=***;Cert=”—–BEGIN%20CERTIFICATE—–%0AMIIGxTCCBK2gAwIBAgIRAPPDecfD1O2rVgSbz5qRTw0wDQYJKoZIhvcNAQELBQAw%0AeTELMAkG
***************** truncated ********************
upVq0l1ImDE%0A—–END%20CERTIFICATE—–%0A”;Subject=”CN=poster-quovadis (P000000),L=poster.***.demo.sap,OU=37d98dec-719a-4137-9ed7-***,OU=SAP Cloud Platform Clients,O=SAP SE,C=DE”;URI=,By=spiffe://cluster.local/ns/montypython/sa/httpbin;Hash=9fbcc6e4a30e6c24673dd6d822ddea353346d939d907b8cfd3cd3d558d91524b;Subject=””;URI=spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account”,
“X-Forwarded-Host”: “httpbin-mtls-${namespace}.mtls-quovadis-<id>.quovadis.kyma.dev.sap”
}
}

So far so good. But is that all?

Well, not necessarily, as it would imply having to validate the X-Forwarded-Client-Cert header in each workload that receives it.

When using istio, one would simply have all these checks done by a dedicated AuthorizationPolicy, for instance:

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:

That’s it. By simply adding the above manifest if the client certificate sent does not have the policy matching X-Client-Ssl-Cn header the workload access will be denied with the following error message:

RBAC: access denied

~quovadis

Appendix

Here goes a terraform provisioner script snippet to help deploy the httpbin workload into a kyma cluster.resource “terraform_data” “httpbin” {

provisioner “local-exec” {
interpreter = [“/bin/bash”, “-c”]
on_failure = continue
command = <<EOF
(
KUBECONFIG=kubeconfig-headless.yaml
NAMESPACE=montypython

set -e -o pipefail
HTTPBIN=$(./kubectl –kubeconfig $KUBECONFIG -n $NAMESPACE get deployment httpbin –ignore-not-found)
if [ “$HTTPBIN” = “” ]
then
./kubectl create ns $NAMESPACE –kubeconfig $KUBECONFIG –dry-run=client -o yaml | ./kubectl apply –kubeconfig $KUBECONFIG -f –
./kubectl label namespace $NAMESPACE istio-injection=enabled –kubeconfig $KUBECONFIG
./kubectl -n $NAMESPACE create -f https://raw.githubusercontent.com/quovadis-btp/istio/refs/heads/master/samples/httpbin/httpbin.yaml –kubeconfig $KUBECONFIG

while [ “$(./kubectl –kubeconfig $KUBECONFIG -n $NAMESPACE get deployment httpbin –ignore-not-found)” = “” ]
do
echo “no deployment httpbin”
sleep 1
done
fi

HTTPBIN=$(./kubectl –kubeconfig $KUBECONFIG -n $NAMESPACE rollout status deployment httpbin –timeout 5m)
echo $HTTPBIN

)
EOF
}
}

For reference, the basics of mTLS communications with kyma runtime environment are covered in the following blogpost, namely: Developing enterprise-grade applications with mutual TLS easy with SAP BTP, Kyma runtime and BTP destinations

Kyma runtime stories. mTLS routes made easy (-ier) with APIRule v2Eventually, following the official announcement, the lastest istio-based APIRule in the stable version v2 is out.Kyma Runtime – API Gateway module: Update to version 3.0.2This version of the API Gateway module introduces APIRule CR in the stable version v2.The new CRD allows you to expose your workloads using one of the three supported access strategies: jwt, noAuth, and extAuth. The noAuth access strategy provides a simple configuration for exposing workloads over the specified HTTP methods. The jwt access strategy allows you to secure your workload by defining Istio JWT configuration and the extAuth access strategy allows for providing custom authentication and authorization logic. For more information, see APIRule Custom Resource Additionally, version 3.0.2 contains the following fixes:We’ve fixed the behaviour of in-cluster connectivity when using JWT handler in APIRule. Now, in-cluster connectivity is blocked.Now, when you create an APIRule with the noAuth access strategy, it is validated if the target workload has an Istio sidecar proxy injected.We’ve added a deprecation message, which is displayed when you use the k8s API to interact with APIRule v1beta1 CRs. Now, the following deprecation message is presented during each call: Warning: Version v1beta1 of APIRule is deprecated and will be removed in future releases. Use version v2 instead.Until now, customers and developers were invited to test the istio-based APIRule in the version v2alpha1.Good to know:The promotion of APIRule manifests from version v2alpha1 to version v2 merely consists of manually changing apiVersion from v2alpha1 to the stable version v2 in APIRule manifests.As depicted above, for convenience, Kyma dashboard will display the APIRules CRs grouped accordingly to their apiVersion.APIRule v2 and mTLS routesThe APIRule v2 is istio-based. What that means is that both authentication and authorisation (permissions) are handled natively through istio.Let me try to demonstrate how and whether it can help to ease the implementation effort of the mTLS routes.Good to know:Albeit the mTLS routes are mainly used for system to system communications, they can also be used for the principal propagation sake with short-lived client certificates.For reference, the basics of mTLS communications with kyma runtime environment are covered in the following blogpost, namely: Developing enterprise-grade applications with mutual TLS easy with SAP BTP, Kyma runtime and BTP destinationsWhen implementing mTLS routes one needs to forward the client certificate to the server side. This is achieved by requesting the istio SSL headers that will be forwarded to the server side, for instance: request:
headers:
X-CLIENT-SSL-CN: ‘%DOWNSTREAM_PEER_SUBJECT%’
X-CLIENT-SSL-ISSUER: ‘%DOWNSTREAM_PEER_ISSUER%’
X-CLIENT-SSL-SAN: ‘%DOWNSTREAM_PEER_URI_SAN%’
test: ‘true’Let’s consider the following APIRule v2 manifest:apiVersion: gateway.kyma-project.io/v2
kind: APIRule
metadata:
labels:
app.kubernetes.io/name: httpbin-mtls-${namespace}

name: httpbin-mtls-${namespace}
namespace: montypython
spec:
gateway: ${namespace}/quovadis-${namespace}-gateway-mtls
hosts:
– httpbin-mtls-${namespace}.${mtls}.${base_domain}
rules:
– methods:
– GET
noAuth: true
path: /*
timeout: 300
request:
headers:
X-CLIENT-SSL-CN: ‘%DOWNSTREAM_PEER_SUBJECT%’
X-CLIENT-SSL-ISSUER: ‘%DOWNSTREAM_PEER_ISSUER%’
X-CLIENT-SSL-SAN: ‘%DOWNSTREAM_PEER_URI_SAN%’
test: ‘true’
service:
name: httpbin
port: 8000Good to know:Any custom headers can be included there and they will be forwarded to the server side as well.This is how it looks like in GUI of APIRule in the kyma dashboard, as depicted below:The above APIRule definition is translated into auto-generated istio VirtualService and the AuthorizationPolicy resource definitions, namely:VirtualServiceapiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
labels:
apirule.gateway.kyma-project.io/v1beta1: httpbin-mtls-${namespace}.montypython

name: httpbin-mtls-${namespace}-g6kzx
namespace: montypython
spec:
rules:
– from:
– source:
principals:
– >-
cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account
to:
– operation:
methods:
– GET
paths:
– /{**}
selector:
matchLabels:
app: httpbinGood to know:The current design of the APIRule implies having multiple AuthorizationPolicy objects, one per path.The above AuthorizationPolicy guarantees a given workload can only be accessed via an istio ingress gateway, with the advertised HTTP method(s) and from the advertised paths.Smoke-testing of the mTLS routeGood to know:The smoke test using curl command is described hereAssuming we have a CA-signed client certificate at hand, we can test the XFF headers being forwarded to the httpbin workload running on a kyma cluster, as follows:curl https://httpbin-mtls-${namespace}.mtls-quovadis-<id>.quovadis.kyma.dev.sap/headers –cert poster.x509 –key poster.key | jq .{
“headers”: {
“Accept”: “*/*”,
“Host”: “httpbin-mtls-${namespace}.mtls-quovadis-<id>.quovadis.kyma.dev.sap”,
“Test”: “true”,
“User-Agent”: “curl/8.7.1”,
“X-Client-Ssl-Cn”: “CN=poster-quovadis (P000000),L=poster.***.demo.sap,OU=37d98dec-719a-4137-9ed7-***,OU=SAP Cloud Platform Clients,O=SAP SE,C=DE”,
“X-Client-Ssl-Issuer”: “CN=SAP Cloud Platform Client CA,OU=SAP Cloud Platform Clients,O=SAP SE,L=EU10,C=DE”,
“X-Envoy-Attempt-Count”: “1”,
“X-Envoy-External-Address”: “**.214.184.**”,
“X-Forwarded-Client-Cert”: “Hash=***;Cert=”—–BEGIN%20CERTIFICATE—–%0AMIIGxTCCBK2gAwIBAgIRAPPDecfD1O2rVgSbz5qRTw0wDQYJKoZIhvcNAQELBQAw%0AeTELMAkG
***************** truncated ********************
upVq0l1ImDE%0A—–END%20CERTIFICATE—–%0A”;Subject=”CN=poster-quovadis (P000000),L=poster.***.demo.sap,OU=37d98dec-719a-4137-9ed7-***,OU=SAP Cloud Platform Clients,O=SAP SE,C=DE”;URI=,By=spiffe://cluster.local/ns/montypython/sa/httpbin;Hash=9fbcc6e4a30e6c24673dd6d822ddea353346d939d907b8cfd3cd3d558d91524b;Subject=””;URI=spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account”,
“X-Forwarded-Host”: “httpbin-mtls-${namespace}.mtls-quovadis-<id>.quovadis.kyma.dev.sap”
}
}So far so good. But is that all?Well, not necessarily, as it would imply having to validate the X-Forwarded-Client-Cert header in each workload that receives it.When using istio, one would simply have all these checks done by a dedicated AuthorizationPolicy, for instance:apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:

spec:
action: ALLOW
rules:
– to:
– operation:
hosts:
– >-
httpbin-mtls-${namespace}.mtls-quovadis-<id>.quovadis.kyma.dev.sap
when:
– key: request.headers[X-Client-Ssl-Cn]
values:
– >-
CN=poster-quovadis (P000000),L=poster.***.demo.sap,OU=37d98dec-719a-4137-9ed7-***,OU=SAP Cloud Platform Clients,O=SAP SE,C=DE
targetRef:
group: gateway.networking.k8s.io
kind: Gateway
name: quovadis-${namespace}-gateway-mtlsThat’s it. By simply adding the above manifest if the client certificate sent does not have the policy matching X-Client-Ssl-Cn header the workload access will be denied with the following error message:RBAC: access denied ~quovadisAppendixHere goes a terraform provisioner script snippet to help deploy the httpbin workload into a kyma cluster.resource “terraform_data” “httpbin” {

provisioner “local-exec” {
interpreter = [“/bin/bash”, “-c”]
on_failure = continue
command = <<EOF
(
KUBECONFIG=kubeconfig-headless.yaml
NAMESPACE=montypython

while [ “$(./kubectl –kubeconfig $KUBECONFIG -n $NAMESPACE get deployment httpbin –ignore-not-found)” = “” ]
do
echo “no deployment httpbin”
sleep 1
done
fi

HTTPBIN=$(./kubectl –kubeconfig $KUBECONFIG -n $NAMESPACE rollout status deployment httpbin –timeout 5m)
echo $HTTPBIN

)
EOF
}
} For reference, the basics of mTLS communications with kyma runtime environment are covered in the following blogpost, namely: Developing enterprise-grade applications with mutual TLS easy with SAP BTP, Kyma runtime and BTP destinations Read More Technology Blog Posts by SAP articles

#SAP

#SAPTechnologyblog