I'd like to diff a Kubernetes YAML template against the actual deployed ressources. This should be possible using kubectl diff. However, on my Kubernetes cluster in Azure, I get the following error:
Error from server (InternalError): Internal error occurred: admission webhook "aks-webhook-admission-controller.azmk8s.io" does not support dry run
Is there something I can enable on AKS to let this work or is there some other way of achieving the diff?
As a workaround you can use standard GNU/Linux diff
command in the following way:
diff -uN <(kubectl get pods nginx-pod -o yaml) example_pod.yaml
I know this is not a solution but just workaround but I think it still can be considered as full-fledged replacement tool.
Thanks, but that doesn't work for me, because it's not just one pod I'm interested in, it's a whole Helm release with deployment, services, jobs, etc. – dploeger
But anyway you won't compare everything at once, will you ?
You can use it for any resource you like, not only for Pods
. Just substitute Pod
by any other resource you like.
Anyway, under the hood kubectl diff
uses diff command
In kubectl diff --help
you can read:
KUBECTL_EXTERNAL_DIFF environment variable can be used to select your own diff command. By default, the "diff" command available in your path will be run with "-u" (unified diff) and "-N" (treat absent files as empty) options.
The real problem in your case is that you cannot use for some reason --dry-run
on your AKS Cluster, which is question to AKS users/experts. Maybe it can be enabled somehow but unfortunately I have no idea how.
Basically kubectl diff
compares already deployed resource, which we can get by:
kubectl get resource-type resource-name -o yaml
with the result of:
kubectl apply -f nginx.yaml --dry-run --output yaml
and not with actual content of your yaml file (simple cat nginx.yaml
would be ok for that purpose).
You can additionally use:
kubectl get all -l "app.kubernetes.io/instance=<helm_release_name>" -o yaml
to get yamls
of all resources belonging to specific helm release.
As you can read in man diff
it has following options:
--from-file=FILE1
compare FILE1 to all operands; FILE1 can be a directory
--to-file=FILE2
compare all operands to FILE2; FILE2 can be a directory
so we are not limited to comparing single files but also files located in specific directory. Only we can't use these two options together.
So the full diff
command for comparing all resources belonging to specific helm release currently deployed on our kubernetes cluster with yaml
files from a specific directory may look like this:
diff -uN <(kubectl get all -l "app.kubernetes.io/instance=<helm_release_name>" -o yaml) --to-file=directory_containing_yamls/
Thanks, but that doesn't work for me, because it's not just one pod I'm interested in, it's a whole Helm release with deployment, services, jobs, etc.
If I understood it correctly,
kubectl diff
will diff the contents of the given YAML file with the resources already deployed. My YAML file contains many different resource types, so testing every resource manually is not feasible. I know, that I can script my way around this by analyzing the YAML file and then fetching the resources and diff them but I wanted to know about other possibilities. But thanks so far!I understand that comparing resources one by one would be very tedious task and you may rather want to avoid it. You can additionally use
kubectl get all -l "app.kubernetes.io/instance=<helm_release_name>" -o yaml
to getyamls
of all resources belonging to specific helm release.That’s a great idea. Could you work that into your answer? I'd like to accept it then.
Sure, I just did it.