https://kwok.sigs.k8s.io/docs/Recent content in Docs on KWOKHugoenhttps://kwok.sigs.k8s.io/docs/generated/apis/Mon, 01 Jan 0001 00:00:00 +0000https://kwok.sigs.k8s.io/docs/generated/apis/API reference Packages: action.kwok.x-k8s.io/v1alpha1 config.kwok.x-k8s.io/v1alpha1 kwok.x-k8s.io/v1alpha1 action.kwok.x-k8s.io/v1alpha1 # Package v1alpha1 implements the v1alpha1 apiVersion of kwokctl’s action Resource Types: ResourcePatch ResourcePatch # ResourcePatch provides resource definition for kwokctl. this is a action of resource patch. Field Description apiVersion string action.kwok.x-k8s.io/v1alpha1 kind string ResourcePatch resource GroupVersionResource Resource represents the resource to be patched. target Target Target represents the target of the ResourcePatch. durationNanosecond time.Duration DurationNanosecond represents the duration of the patch in nanoseconds.https://kwok.sigs.k8s.io/docs/design/architecture/Mon, 01 Jan 0001 00:00:00 +0000https://kwok.sigs.k8s.io/docs/design/architecture/Architecture # This document will introduce the architecture of KWOK. kwok # kwok is a resource controller, similar to kube-controller-manager, that is responsible for simulating the lifecycle of fake nodes, pods, and other Kubernetes API resources. It can run in any environment, use in-cluster or kubeconfig to connect to a kube-apiserver of cluster, and then manage the resources of the cluster. So far, kwok provides two types of controllers:https://kwok.sigs.k8s.io/docs/examples/argo/Mon, 01 Jan 0001 00:00:00 +0000https://kwok.sigs.k8s.io/docs/examples/argo/Argo # More information about Argo can be found at Argo quick start. Custom Pod Behavior # The Argo Workflow is a custom resource for creating Pods, not using Job. So we need to change the behavior of the Pod to make it work. wget https://raw.githubusercontent.com/kubernetes-sigs/kwok/release-0.3/stages/pod-fast.yaml sed 's/Job/Workflow/g' pod-fast.yaml > workflow-fast.yaml Set up Cluster # kwokctl create cluster --runtime kind -c workflow-fast.yaml Create Node # kubectl apply -f https://kwok.sigs.k8s.io/examples/node.yaml Deploy Argo # kubectl create namespace argo kubectl apply -n argo -f https://github.https://kwok.sigs.k8s.io/docs/examples/dra/Mon, 01 Jan 0001 00:00:00 +0000https://kwok.sigs.k8s.io/docs/examples/dra/DRA (Dynamic Resource Allocation) # More information about DRA can be found at here. Setup Cluster # Create a cluster with DRA feature enabled kwokctl create cluster --kube-feature-gates="kube:DynamicResourceAllocation=true" --kube-runtime-config="resource.k8s.io/v1beta1=true" Create Device Class # Create a DeviceClass for GPU resources kubectl apply -f - <<EOF apiVersion: resource.k8s.io/v1beta1 kind: DeviceClass metadata: name: kwok.x-k8s.io spec: selectors: - cel: expression: device.driver == 'gpu.kwok.x-k8s' EOF Create GPU Node # Create a fake GPU nodehttps://kwok.sigs.k8s.io/docs/design/introduction/Mon, 01 Jan 0001 00:00:00 +0000https://kwok.sigs.k8s.io/docs/design/introduction/Introduction # This document will introduce the design of KWOK. User Stories # Scheduler # As a scheduler developer, I want to test the scheduler with a large number of Nodes and Pods. CRD Controller # As a CRD controller developer, I want to test the controller without fake clients. Control Plane Performance # As a control plane performance tester, I want to test the performance of the control plane at a low cost.https://kwok.sigs.k8s.io/docs/examples/istio/Mon, 01 Jan 0001 00:00:00 +0000https://kwok.sigs.k8s.io/docs/examples/istio/Istio # More information about Istio can be found at Istio quick start. Set up Cluster # kwokctl create cluster --runtime kind Create Node # kubectl apply -f https://kwok.sigs.k8s.io/examples/node.yaml Deploy Istio # istioctl install -y Migrate Controllers to Real Node # kubectl patch deploy istiod -n istio-system --type=json -p='[{"op":"add","path":"/spec/template/spec/nodeName","value":"kwok-kwok-control-plane"}]' kubectl patch deploy istio-ingressgateway -n istio-system --type=json -p='[{"op":"add","path":"/spec/template/spec/nodeName","value":"kwok-kwok-control-plane"}]' Create Pod and Inject Sidecar # kubectl label namespace default istio-injection=enabled kubectl apply -f https://raw.https://kwok.sigs.k8s.io/docs/examples/kueue/Mon, 01 Jan 0001 00:00:00 +0000https://kwok.sigs.k8s.io/docs/examples/kueue/Kueue # More information about Kueue can be found here. Set up Cluster # Create a new KWOK cluster using the kind runtime kwokctl create cluster --runtime kind Create Node # Create KWOK fake nodes (the example node already has type=kwok label) kubectl apply -f https://kwok.sigs.k8s.io/examples/node.yaml Verify that the nodes have the correct label using a label selector kubectl get nodes -l type=kwok kwok-node-0 Ready agent 4s kwok-v0.7.0 Deploy Kueue # Install Kueue controller-managerhttps://kwok.sigs.k8s.io/docs/examples/linkerd/Mon, 01 Jan 0001 00:00:00 +0000https://kwok.sigs.k8s.io/docs/examples/linkerd/Linkerd # More information about Linkerd can be found at Linkerd quick start. Set up Cluster # kwokctl create cluster --runtime kind Create Node # kubectl apply -f https://kwok.sigs.k8s.io/examples/node.yaml Deploy Linkerd # linkerd check --pre linkerd install --crds | kubectl apply -f - linkerd install | kubectl apply -f - Migrate Controllers to Real Node # kubectl patch deploy linkerd-destination -n linkerd --type=json -p='[{"op":"add","path":"/spec/template/spec/nodeName","value":"kwok-kwok-control-plane"}]' kubectl patch deploy linkerd-identity -n linkerd --type=json -p='[{"op":"add","path":"/spec/template/spec/nodeName","value":"kwok-kwok-control-plane"}]' kubectl patch deploy linkerd-proxy-injector -n linkerd --type=json -p='[{"op":"add","path":"/spec/template/spec/nodeName","value":"kwok-kwok-control-plane"}]' Create Pod and Inject Sidecar # kubectl apply -f https://run.https://kwok.sigs.k8s.io/docs/examples/metrics-server/Mon, 01 Jan 0001 00:00:00 +0000https://kwok.sigs.k8s.io/docs/examples/metrics-server/ Metrics Server # More information about an be found at Metrics and ResourceUsage. Fetch default metrics usage # wget https://github.com/kubernetes-sigs/kwok/releases/download/v0.5.1/metrics-usage.yaml Set up Cluster # kwokctl create cluster --enable-metrics-server -c metrics-usage.yaml Create Node and Pod # kwokctl scale node --replicas 2 kwokctl scale pod --replicas 8 Test Metrics # Wait about 45 seconds for the metrics server to collect the data. kubectl top node kubectl top podhttps://kwok.sigs.k8s.io/docs/examples/prometheus/Mon, 01 Jan 0001 00:00:00 +0000https://kwok.sigs.k8s.io/docs/examples/prometheus/Prometheus # This document walks you through how to enable Prometheus on clusters Create a cluster with Prometheus # kwokctl create cluster --prometheus-port 9090 Create Grafana dashboard with Prometheus data source # docker run -d --name=grafana -p 3000:3000 docker.io/grafana/grafana:9.4.7 Open your web browser and go to http://localhost:3000 On the login page, enter admin for username and password. Add the Prometheus data source, http://host.docker.internal:9090 or http://hostIP:9090 on Grafana, replacing hostIP with the IP address of your host.https://kwok.sigs.k8s.io/docs/examples/ray/Mon, 01 Jan 0001 00:00:00 +0000https://kwok.sigs.k8s.io/docs/examples/ray/Ray # More information about Ray can be found here. Set up Cluster # Create a new KWOK cluster using the kind runtime kwokctl create cluster --runtime kind Create Node # Create a KWOK fake node kubectl apply -f https://kwok.sigs.k8s.io/examples/node.yaml Verify that the nodes are created and running kubectl get node NAME STATUS ROLES AGE VERSION kwok-kwok-control-plane Ready,SchedulingDisabled control-plane 3m33s v1.33.0 kwok-node-0 Ready agent 3m11s kwok-v0.7.0 Deploy Ray Operator # Add the KubeRay Helm repository and install the KubeRay operator using Helm