Getting Started (Alpha)
The Kubernetes Provider (Alpha) is an optional package which an be installed to enable writing policies which interact with Kubernetes related resources.
Install Kubernetes Plugin
First, ensure you have installed the base Cloud Custodian application. Cloud Custodian is a Python application and must run on an actively supported version.
Once the base install is complete, you are now ready to install the Kubernetes provider package using one of the following options:
Option 1: Install released packages to local Python Environment
pip install c7n
pip install c7n-kube
Option 2: Install latest from the repository
git clone https://github.com/cloud-custodian/cloud-custodian.git
pip install -e ./cloud-custodian
pip install -e ./cloud-custodian/tools/c7n_kube
Connecting to your Cluster
The Custodian Kubernetes provider automatically uses your kubectl configuration or the config
file set by the environment variable KUBECONFIG. See the Kubernetes Docs
for more information.
Write Your First Policy
A policy is the primary way that Custodian is configured to manage cloud resources. It is a YAML file that follows a predetermined schema to describe what you want Custodian to do.
There are three main components to a policy:
Resource: the type of resource to run the policy against
Filters: criteria to produce a specific subset of resources
Actions: directives to take on the filtered set of resources
In the example below, we will write a policy that filters for pods with a label “custodian” and deletes it:
First, lets create a pod resource that we want to target with the policy:
❯ kubectl run nginx --image=nginx --labels=name=custodian
❯ kubectl get pod -o wide --show-labels
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES LABELS
nginx 1/1 Running 0 24s 10.0.1.224 worker <none> <none> name=custodian
Now in the example below, we will write a policy that filters for pods with a label “custodian” and deletes it:
Filename: custodian.yml
policies:
- name: my-first-policy
description: |
Deletes pods with label name:custodian
resource: k8s.pod
filters:
- type: value
key: metadata.labels.name
value: custodian
actions:
- type: delete
Run Your Policy
First, ensure you have configured connectivity to your cluster.
Next, run the following command to execute the policy with Custodian:
custodian run --output-dir=output custodian.yml --cache-period 0 -v
If successful, you should see output similar to the following on the command line:
2022-09-14 12:28:38,735: custodian.cache:DEBUG Disabling cache
2022-09-14 12:28:38,735: custodian.commands:DEBUG Loaded file pod.yaml. Contains 1 policies
2022-09-14 12:28:38,736: custodian.output:DEBUG Storing output with <LogFile file://output/my-first-policy/custodian-run.log>
2022-09-14 12:28:38,737: custodian.policy:DEBUG Running policy:pod resource:k8s.pod region:default c7n:0.9.18
2022-09-14 12:28:38,754: custodian.k8s.client:DEBUG connecting to https://127.0.0.1:61427
2022-09-14 12:28:38,819: custodian.resources.pod:DEBUG Filtered from 17 to 1 pod
2022-09-14 12:28:38,820: custodian.policy:INFO policy:pod resource:k8s.pod region: count:1 time:0.08
2022-09-14 12:28:38,837: custodian.k8s.client:DEBUG connecting to https://127.0.0.1:61427
2022-09-14 12:28:38,863: custodian.policy:INFO policy:pod action:deleteresource resources:1 execution_time:0.04
2022-09-14 12:28:38,864: custodian.output:DEBUG metric:ResourceCount Count:1 policy:pod restype:k8s.pod scope:policy
You should also find a new output/my-first-policy directory with a log and other
files (subsequent runs will append to the log by default, rather than
overwriting it).
See Generic Filters for more information on the features of the Value filter used in this sample.
You can also use custodian schema to get more information on the filters available to you.
❯ custodian schema k8s
resources:
- k8s.config-map
- k8s.custom-cluster-resource
- k8s.custom-namespaced-resource
- k8s.daemon-set
- k8s.deployment
- k8s.namespace
- k8s.node
- k8s.pod
- k8s.replica-set
- k8s.replication-controller
- k8s.secret
- k8s.service
- k8s.service-account
- k8s.stateful-set
- k8s.volume
- k8s.volume-claim
To understand which values are available for a resource you can use kubectl, so for example to understand what attributes a persistent volume has on it you can run:
❯ kubectl explain persistentvolume --recursive
KIND: PersistentVolume
VERSION: v1
DESCRIPTION:
PersistentVolume (PV) is a storage resource provisioned by an
administrator. It is analogous to a node. More info:
https://kubernetes.io/docs/concepts/storage/persistent-volumes
FIELDS:
apiVersion <string>
kind <string>
metadata <Object>
annotations <map[string]string>
creationTimestamp <string>
deletionGracePeriodSeconds <integer>
deletionTimestamp <string>
finalizers <[]string>
generateName <string>
generation <integer>
labels <map[string]string>
....
Or if you have a resource already deployed in your cluster and you want to figure out how to taret it you can output it to json and review the available attributes that way:
❯ kubectl get pv node-pv-volume -o json
{
"apiVersion": "v1",
"kind": "PersistentVolume",
"metadata": {
"annotations": {
"kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"v1\",\"kind\":\"PersistentVolume\",\"metadata\":{\"annotations\":{},\"labels\":{\"type\":\"local\"},\"name\":\"node-pv-volume\"},\"spec\":{\"accessModes\":[\"ReadWriteOnce\"],\"capacity\":{\"storage\":\"1Gi\"},\"hostPath\":{\"path\":\"/tmp/k8s\"},\"storageClassName\":\"manual\"}}\n"
},
"creationTimestamp": "2022-10-14T19:34:45Z",
"finalizers": [
"kubernetes.io/pv-protection"
],
"labels": {
"type": "local"
},
"name": "node-pv-volume",
"resourceVersion": "394700",
"uid": "ad414486-9fd9-48ac-8cc5-7d6b9c24b524"
},
"spec": {
"accessModes": [
"ReadWriteOnce"
],
"capacity": {
"storage": "1Gi"
},
"hostPath": {
"path": "/tmp/k8s",
"type": ""
},
"persistentVolumeReclaimPolicy": "Retain",
"storageClassName": "manual",
"volumeMode": "Filesystem"
},
"status": {
"phase": "Available"
}
}