(formerly known as kubectl-enter)
Start a root shell in the node's host OS running. Uses an alpine pod with nsenter for Linux nodes and a HostProcess pod with PowerShell for Windows nodes.
using krew:
Plugin can be installed from the official krew repository:
kubectl krew install node-shell
Or from our own krew repository:
kubectl krew index add kvaps https://github.com/kvaps/krew-index kubectl krew install kvaps/node-shell
or using curl:
curl -LO https://github.com/kvaps/kubectl-node-shell/raw/master/kubectl-node_shell chmod +x ./kubectl-node_shell sudo mv ./kubectl-node_shell /usr/local/bin/kubectl-node_shell# Get standard bash shell kubectl node-shell <node> # Use custom image for pod kubectl node-shell <node> --image <image> # Use X-mode (mount /host, and do not enter host namespace) kubectl node-shell -x <node> # Skip specific namespace types to enter, choose any of ipc, mount, pid, net, uts kubectl node-shell <node> --no-ipc # Execute custom command kubectl node-shell <node> -- echo 123 # Use stdin cat /etc/passwd | kubectl node-shell <node> -- sh -c 'cat > /tmp/passwd' # Run oneliner script kubectl node-shell <node> -- sh -c 'cat /tmp/passwd; rm -f /tmp/passwd'X-mode can be useful for debugging minimal systems that do not have a built-in shell (eg. Talos).
Here's an example of how you can debug the network for a rootless kube-apiserver container without a filesystem:
kubectl node-shell -x <node> # Download crictl wget https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.28.0/crictl-v1.28.0-linux-amd64.tar.gz -O- | \ tar -xzf- -C /usr/local/bin/ # Setup CRI endpoint export CONTAINER_RUNTIME_ENDPOINT=unix:///host/run/containerd/containerd.sock # Find your container crictl ps | grep kube-apiserver #3ff4626a9f10e e7972205b6614 6 hours ago Running kube-apiserver 0 215107b47bd7e kube-apiserver-talos-rzq-nkg # Find pid of the container crictl inspect 3ff4626a9f10e | grep pid # "pid": 2152, # "pid": 1 # "type": "pid" # "getpid", # "getppid", # "pidfd_open", # "pidfd_send_signal", # "waitpid", # Go to network namespace of the pid, but keep mount namespace of the debug container nsenter -t 2152 -nYou need to be able to start privileged containers for that.
You can mount volumes from your CSI storage layer using the -m flag. This allows you to move data to/from node devices seamlessly. The PVC will be mounted at /opt-pvc. This is useful for failover in minimal systems that do not have a built in shell (eg. Talos). Here is an example of how you can retrieve zfs/lvm data from a volume on a failed CSI node and put it back in your distributed storage layer:
k node-shell -n <namespace> -x <node_with_data> -m <pvc_name> # install rsync apk add rsync # Add lvm/zfs libs # ZFS mount -o bind /host/dev /dev mount -o bind /host/usr/local /usr/local touch /lib/libuuid.so.1 mount -o bind /host/lib/libuuid.so.1 /lib/libuuid.so.1 touch /lib/libuuid.so.1.3.0 mount -o bind /host/lib/libuuid.so.1.3.0 /lib/libuuid.so.1.3.0 touch /lib/libblkid.so.1 mount -o bind /host/lib/libblkid.so.1 /lib/libblkid.so.1 touch /lib/libblkid.so.1.1.0 mount -o bind /host/lib/libblkid.so.1.1.0 /lib/libblkid.so.1.1.0 #LVM touch /usr/lib/libaio.so.1 mount -o bind /host/usr/lib/libaio.so.1.0.2 /usr/lib/libaio.so.1 touch /usr/lib/libudev.so.1 mount -o bind /host/usr/lib/libudev.so.1 /usr/lib/libudev.so.1 export PATH=$PATH:/host/sbin mkdir /lib/modules mount -o bind /host/lib/modules /lib/modules # look for data to recover zfs list NAME USED AVAIL REFER MOUNTPOINT hdd-1 15.9T 7.52T 96K /hdd-1 hdd-1/SOME-OLD-PVC-FROM-PREVIOUS-NODE-INSTALL 361G 7.52T 361G - - # mount the failed volume zfs set mountpoint=/mnt hdd-1/SOME-OLD-PVC-FROM-PREVIOUS-NODE-INSTALL zfs mount /hdd-1/SOME-OLD-PVC-FROM-PREVIOUS-NODE-INSTALL # recover the data : copy it to the mounted CSI volume rsync -avh --info=progress2 /mnt/ /opt-pvc/the above exemple assumes pvc_name already exists in namespace. You need to be able to start privileged containers.