Manage nodes and pods with `kwok` #

This document walks you through how to manage nodes and pods with kwok.

Node Management #

For all nodes #

With the --manage-all-nodes=true argument, kwok will be in charge of all nodes in the cluster and maintain their heartbeats to the API Server. In this way, all nodes will behave like real nodes and stay in the Ready state.

For specific nodes with annotation #

With the --manage-nodes-with-annotation-selector=kwok.x-k8s.io/node=fake argument, kwok just manages the nodes with the annotation kwok.x-k8s.io/node=fake. If the annotation is not present, kwok will not manage the node.

For specific nodes with label #

With the --manage-nodes-with-label-selector=kwok.x-k8s.io/node=fake argument, kwok just manages the nodes with the label kwok.x-k8s.io/node=fake. If the label is not present, kwok will not manage the node.

For single node #

With the --manage-single-node=fake-node argument, kwok only manages the node named fake-node.

Create a Node #

With kwok, you can join arbitrary Node(s) simply by creating v1.Node object(s):

The status can be any value.

kubectl apply -f - <<EOF
apiVersion: v1
kind: Node
metadata:
  annotations:
    node.alpha.kubernetes.io/ttl: "0"
    kwok.x-k8s.io/node: fake
  labels:
    beta.kubernetes.io/arch: amd64
    beta.kubernetes.io/os: linux
    kubernetes.io/arch: amd64
    kubernetes.io/hostname: kwok-node-0
    kubernetes.io/os: linux
    kubernetes.io/role: agent
    node-role.kubernetes.io/agent: ""
    type: kwok
  name: kwok-node-0
spec:
  taints: # Avoid scheduling actual running pods to fake Node
  - effect: NoSchedule
    key: kwok.x-k8s.io/node
    value: fake
status:
  allocatable:
    cpu: 32
    memory: 256Gi
    pods: 110
  capacity:
    cpu: 32
    memory: 256Gi
    pods: 110
  nodeInfo:
    architecture: amd64
    bootID: ""
    containerRuntimeVersion: ""
    kernelVersion: ""
    kubeProxyVersion: fake
    kubeletVersion: fake
    machineID: ""
    operatingSystem: linux
    osImage: ""
    systemUUID: ""
  phase: Running
EOF

After the Node is created, kwok will maintain its heartbeat and keep it in the ready state.

$ kubectl get node -o wide
NAME          STATUS   ROLES   AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE    KERNEL-VERSION   CONTAINER-RUNTIME
kwok-node-0   Ready    agent   5s    fake      196.168.0.1   <none>        <unknown>   <unknown>        <unknown>

Create a Pod #

Now we create some Pods to verify if they can land on the previously created Nodes:

kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: fake-pod
  namespace: default
spec:
  replicas: 10
  selector:
    matchLabels:
      app: fake-pod
  template:
    metadata:
      labels:
        app: fake-pod
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: type
                operator: In
                values:
                - kwok
      # A taints was added to an automatically created Node.
      # You can remove taints of Node or add this tolerations.
      tolerations:
      - key: "kwok.x-k8s.io/node"
        operator: "Exists"
        effect: "NoSchedule"
      containers:
      - name: fake-container
        image: fake-image
EOF

After the Pod is created, we can see that all the Pods are landed on that Node(s) and are in the Running state.

$ kubectl get pod -o wide
NAME                        READY   STATUS    RESTARTS   AGE   IP          NODE          NOMINATED NODE   READINESS GATES
fake-pod-59bb47845f-4vl9f   1/1     Running   0          5s    10.0.0.5    kwok-node-0   <none>           <none>
fake-pod-59bb47845f-bc49m   1/1     Running   0          5s    10.0.0.4    kwok-node-0   <none>           <none>
fake-pod-59bb47845f-cnjsv   1/1     Running   0          5s    10.0.0.7    kwok-node-0   <none>           <none>
fake-pod-59bb47845f-g29wz   1/1     Running   0          5s    10.0.0.2    kwok-node-0   <none>           <none>
fake-pod-59bb47845f-gxq88   1/1     Running   0          5s    10.0.0.10   kwok-node-0   <none>           <none>
fake-pod-59bb47845f-pnzmn   1/1     Running   0          5s    10.0.0.9    kwok-node-0   <none>           <none>
fake-pod-59bb47845f-sfkk4   1/1     Running   0          5s    10.0.0.3    kwok-node-0   <none>           <none>
fake-pod-59bb47845f-vl2z5   1/1     Running   0          5s    10.0.0.8    kwok-node-0   <none>           <none>
fake-pod-59bb47845f-vpfhv   1/1     Running   0          5s    10.0.0.6    kwok-node-0   <none>           <none>
fake-pod-59bb47845f-wxn4b   1/1     Running   0          5s    10.0.0.1    kwok-node-0   <none>           <none>

Update spec of nodes or pods #

In a kwok context, Nodes and Pods are nothing but pure API objects so feel free to mutate their API specs to do whatever simulation or testing you want.

Manage nodes and pods with kwok #