# K8S plugin

The `k8s` plugin gathers measurements about Kubernetes pods.

## Requirements

You need:
1. A Kubernetes cluster
2. A ServiceAccount token (see the configuration section)

We do not require a minimum version, because our use of the API is very minimal.

To test the plugin locally, you can use [minikube](https://minikube.sigs.k8s.io).

## Metrics

Here are the metrics collected by the plugin's sources.

|Name|Type|Unit|Description|Resource|ResourceConsumer|Attributes|
|----|----|----|-----------|--------|----------------|----------|
|`cpu_time_delta`|Delta|nanoseconds|time spent by the pod executing on the CPU|`LocalMachine`|`Cgroup`|see below|
|`cpu_percent`|Gauge|Percent (0 to 100)|`cpu_time_delta / delta_t / n_cores` (all cores used fully = 100%)|`LocalMachine`|`Cgroup`|see below|
|`memory_usage`|Gauge|Bytes|total pod's memory usage|`LocalMachine`|`Cgroup`|see below|
|`cgroup_memory_anonymous`|Gauge|Bytes|anonymous memory usage|`LocalMachine`|`Cgroup`|see below|
|`cgroup_memory_file`|Gauge|Bytes|memory used to cache filesystem data|`LocalMachine`|`Cgroup`|see below|
|`cgroup_memory_kernel_stack`|Gauge|Bytes|memory allocated to kernel stacks|`LocalMachine`|`Cgroup`|see below|
|`cgroup_memory_pagetables`|Gauge|Bytes|memory reserved for the page tables|`LocalMachine`|`Cgroup`|see below|

### Attributes

The measurements produced by the `k8s` plugin have the following attributes:
- `uid`: the pod's UUID
- `name`: the pod's name
- `namespace`: the pod's namespace
- `node`: the name of the node (see the configuration)

The **cpu** measurements have an additional attribute `kind`, which can be one of:
- `total`: time spent in kernel and user mode
- `system`: time spent in kernel mode only
- `user`: time spent in user mode only

## Annotation of the Measurements Provided by Other Plugins

Other plugins, such as the [`process-to-cgroup-bridge`](../../process-to-cgroup-bridge/README.md), can produce measurements related to the cgroups of k8s pods.
However, they cannot add job-specific information (such as the job id) to the measurements.

To do that, use the annotation feature of the `k8s` plugin by enabling the following configuration option.

```toml
annotate_foreign_measurements = true
```

Be sure to enable the `k8s` plugin **after** the plugins that produce the measurements that you want to annotate.
For instance, the `k8s` configuration section should be after the `process-to-cgroup-bridge` section.

```toml
[plugins.process-to-cgroup-bridge]
…

[plugins.k8s]
…
```

Sometimes, others plugins also retrieve cgroups associated to a container. So in order to annotate them, you need to use the `annotate_containers`
and set it to **true**. If you do this, you will allow plugin to check for pod UID of a containers.
For example if `annotate_containers` is:
- **true** - "/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podUIDX.slice/crio-UIDY.scope" will resolve as pod UID = UIDX
- **false** - "/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podUIDX.slice/crio-UIDY.scope" will not find the pod UID

## Configuration

Here are some examples of how to configure this plugin.

### Example Configuration for Minikube
<!-- markdownlint-disable MD029 -->

Context: you have started Minikube on your local machine and want to run Alumet alongside of it (not in a pod).

Prerequisites:
1. create a namespace and service account:

```sh
kubectl create ns alumet
kubectl create serviceaccount alumet-reader -n alumet
```

The service account's token will be created and retrieved by the `k8s` Alumet plugin itself.

2. Make the K8S API available locally:

```sh
kubectl proxy --port=8080
```

Then, you can use the following configuration:

```toml
[plugins.k8s]
# Node name 
k8s_node = "minikube"
# URL used to contact Kubernetes API
k8s_api_url = "http://127.0.0.1:8080"
# Way to retrieve token which grant access to Kubernetes API
token_retrieval = "auto"
# Interval between each measurement.
poll_interval = "5s"
# If `true`, adds attributes like `uid`, `name`, `namespace`, `node` to the cgroup measurements produced by other plugins.
annotate_foreign_measurements = false
# Decides whether the cgroups at container level should be annotated or not. A `false` value will only annotate pod cgroups. Note that `annotate_foreign_measurements` needs to be true.
annotate_containers = true
```

### Example Configuration for a full K8S Cluster

Context: you have a K8S cluster and are deploying Alumet in a pod.

Prerequisites:
1. Inject the name of the node in the `NODE_NAME` environment variable of the pod that runs the Alumet agent. See [K8S Docs − Expose Pod Information to Containers Through Environment Variables](https://kubernetes.io/docs/tasks/inject-data-application/environment-variable-expose-pod-information/).
2. Create a ServiceAccount and mount its token in the pod that runs the Alumet agent.

Then, configure the `k8s` plugin.
A typical configuration would look like the following:

```toml
[plugins.k8s]
# Node name 
k8s_node = "${NODE_NAME}"
# URL used to contact Kubernetes API
k8s_api_url = "https://kubernetes.default.svc:443"
# Way to retrieve token which grant access to Kubernetes API
token_retrieval = "file"
# Interval between each measurement.
poll_interval = "5s"
# If `true`, adds attributes like `uid`, `name`, `namespace`, `node` to the cgroup measurements produced by other plugins.
annotate_foreign_measurements = true
# Decides whether the cgroups at container level should be annotated or not. A `false` value will only annotate pod cgroups. Note that `annotate_foreign_measurements` needs to be true.
annotate_containers = true
```

### Possible Token Retrieval Strategies

```toml
# try "file" and fall back to "kubectl"
token_retrieval = "auto"

# run 'kubectl create token'
token_retrieval = "kubectl"

# read /var/run/secrets/kubernetes.io/serviceaccount/token
token_retrieval = "file"

# custom file
token_retrieval.file = "/path/to/token"

# custom kubectl
token_retrieval.kubectl = {
    service_account = "alumet-reader"
    namespace = "alumet"
}
```
