kdigger
Tags: #kubernetes #security
9 October 2022
Hacking Kubernetes Like a Beginner With Kdigger
- Let's start with a scenario where you're inside a shell and you don't know anything about it. You don't know whether it's a shell of container or a VM. How will you explore more and get more information from the shell environment.
- If you're inside a shell then you can use the
idcommand and if it's equal or more than 1000 then you're a normal/unprivileged user otherwise you're a superuser.
kdigger
- You can use kdigger to explore the same environment in detail? You may have many questions in your mind like
- Am I inside a container?
- Am I a privileged user?
- If I'm inside a container then what are the mounts & capabilities I'm having?
Buckets in Kdigger
-
Buckets are something like checks that you want to do. Think of it like a area/section you want to test. For e.g. You might want to check the capabilities present and that's a bucket.
-
The main command is the dig command and it allows you to use various to get buckets insights from your system. To list out different buckets use
kdigger ls -
kdigger can do a lot of things for you and on top of that it's extensible so if you've any new idea then you can share it with community.
-
The
kdigger dig admwill create a priviliged pod in the cluster and it will check whether the privileged pods are allowed in the cluster or not. Is there some admission controller blocking the pod? -
The
cdetectplugin will figure out whether you're running inside a container or a VM. Let's look what happens when I'm running it inside github codespaces.
$ kdigger dig cdetect
### CONTAINERDETECT ###
Comments:
- A majority of true hints might imply running in a container.
+-------------------------------+--------+
| HINT | RESULT |
+-------------------------------+--------+
| systemd is not PID 1 | true |
| kthreadd is not PID 2 | true |
| inode number of root is not 2 | true |
| root is an overlay fs | true |
| /etc/fstab is empty | true |
| /boot is empty | true |
+-------------------------------+--------+
- Running the environment plugin you'll get whether you're running on top of kubernetes or not. I'm executing the same in github codespaces.
$ ./kdigger dig env
### ENVIRONMENT ###
Comments:
- Typical Kubernetes API service env var KUBERNETES_SERVICE_HOST was not found, we might not be running inside a pod.
- Looks like we are not running on top of Kubernetes.
- Now let's check the cgroup part in codespaces.
$ ./kdigger dig cgroup
### CGROUPS ###
+-------------+------------------+---------------------------------------------+
| HIERARCHYID | CONTROLLERLIST | CGROUPPATH |
+-------------+------------------+---------------------------------------------+
| 12 | memory | /docker/176cba129a7ff30ede08dae6e8c934c21bd |
| | | 6596f49247c0d25ca17d6e8719cad |
| 11 | hugetlb | /docker/176cba129a7ff30ede08dae6e8c934c21bd |
| | | 6596f49247c0d25ca17d6e8719cad |
| 10 | rdma | /docker/176cba129a7ff30ede08dae6e8c934c21bd |
| | | 6596f49247c0d25ca17d6e8719cad |
| 9 | pids | /docker/176cba129a7ff30ede08dae6e8c934c21bd |
| | | 6596f49247c0d25ca17d6e8719cad |
| 8 | blkio | /docker/176cba129a7ff30ede08dae6e8c934c21bd |
| | | 6596f49247c0d25ca17d6e8719cad |
| 7 | devices | /docker/176cba129a7ff30ede08dae6e8c934c21bd |
| | | 6596f49247c0d25ca17d6e8719cad |
| 6 | cpu,cpuacct | /docker/176cba129a7ff30ede08dae6e8c934c21bd |
| | | 6596f49247c0d25ca17d6e8719cad |
| 5 | perf_event | /docker/176cba129a7ff30ede08dae6e8c934c21bd |
| | | 6596f49247c0d25ca17d6e8719cad |
| 4 | freezer | /docker/176cba129a7ff30ede08dae6e8c934c21bd |
| | | 6596f49247c0d25ca17d6e8719cad |
| 3 | cpuset | /docker/176cba129a7ff30ede08dae6e8c934c21bd |
| | | 6596f49247c0d25ca17d6e8719cad |
| 2 | net_cls,net_prio | /docker/176cba129a7ff30ede08dae6e8c934c21bd |
| | | 6596f49247c0d25ca17d6e8719cad |
| 1 | name=systemd | /docker/176cba129a7ff30ede08dae6e8c934c21bd |
| | | 6596f49247c0d25ca17d6e8719cad |
| 0 | | /docker/176cba129a7ff30ede08dae6e8c934c21bd |
| | | 6596f49247c0d25ca17d6e8719cad |
+-------------+------------------+---------------------------------------------+
- If you'll notice carefully then it seems that we are inside a docker container. Now at this point I'm excited about what's mounted and what capabilities I'm having. Let's check that too.
$ ./kdigger dig mount
### MOUNT ###
Comments:
- 36 devices are mounted.
+------------+---------------------------------+------------+---------------------------------+
| DEVICE | PATH | FILESYSTEM | FLAGS |
+------------+---------------------------------+------------+---------------------------------+
| overlay | / | overlay | rw,relatime,lowerdir=/var/lib/d |
| | | | ocker/overlay2/l/JSCWZSZ33U2OSI |
| | | | WUGNAHU5W5TA:/var/lib/docker/ov |
| | | | erlay2/l/Z2AF6KBIEEU5BKO4A2E5RO |
| | | | YGKF:/var/lib/docker/overlay2/l |
| | | | /V5TAFXV5JYS2ZCL4UMFZ7DIQCW:/va |
| | | | r/lib/docker/overlay2/l/PMPDEQB |
| | | | 5POSYJK7TLOC6YCKYS4:/var/lib/do |
| | | | cker/overlay2/l/7FDQDUFWWGLSOEB |
| | | | 7CYQN3LHGZZ:/var/lib/docker/ove |
| | | | rlay2/l/Z5N4BZGUAQJODUFJE77HM66 |
| | | | YJK:/var/lib/docker/overlay2/l/ |
| | | | KMEPSIT4HF4WGTRS26YNCJ3VEP:/var |
| | | | /lib/docker/overlay2/l/M6ORVVQ2 |
| | | | 5QVN6ZC5KFPBQKCMGS:/var/lib/doc |
| | | | ker/overlay2/l/JXFQAM537Y26VBHB |
| | | | QFJTBSNXVE:/var/lib/docker/over |
| | | | lay2/l/UBNMEL22VHWPBY2ZFHKQLIVQ |
| | | | FR:/var/lib/docker/overlay2/l/G |
| | | | BMMJU36YNDCSY6C4A745VMMD7:/var/ |
| | | | lib/docker/overlay2/l/2UX6GNBRO |
| | | | EHB7YHQY7OR4CRHN3:/var/lib/dock |
| | | | er/overlay2/l/EIW4PRDFG6H7ES4MU |
| | | | ZDHA3E6QM:/var/lib/docker/overl |
| | | | ay2/l/KJKIOL3A4K7VZQQ6SIN2ATHWI |
| | | | Q:/var/lib/docker/overlay2/l/42 |
| | | | GLD4YAGFJFYZN3OKWZPOKIEW:/var/l |
| | | | ib/docker/overlay2/l/WFW3XR3DXC |
| | | | PC4RPE5RB7IVY63J:/var/lib/docke |
| | | | r/overlay2/l/IZYGZSWSPWFJDR7I2B |
| | | | DSNDAWY2:/var/lib/docker/overla |
| | | | y2/l/TTSBSVX74IVOOHR4SOV3CVLFM6 |
| | | | :/var/lib/docker/overlay2/l/K65 |
| | | | GKNQVMKJCYHCK3ZJCO73H24:/var/li |
| | | | b/docker/overlay2/l/QUARESXF4ZI |
| | | | OSOVSFFYT5BIZ76:/var/lib/docker |
| | | | /overlay2/l/E7Z5Y6EIU7WS2Q2CTHR |
| | | | 3OQBSBB:/var/lib/docker/overlay |
| | | | 2/l/UD5DWGFUTTSJ3CU7TMZXF5QVQ5: |
| | | | /var/lib/docker/overlay2/l/3PJK |
| | | | JV6XHHAKIM2NH5VQGJWKN3:/var/lib |
| | | | /docker/overlay2/l/B3T7FVUYHYS5 |
| | | | M6G275N4U5M5SX:/var/lib/docker/ |
| | | | overlay2/l/EDDUO3YYBYQPIHZFR5RR |
| | | | GLIR25:/var/lib/docker/overlay2 |
| | | | /l/JI5K3X2JSIEQE7WAXJBFMERQLF:/ |
| | | | var/lib/docker/overlay2/l/D6DZ7 |
| | | | 7373CBRZ53WULCZTCGKF6:/var/lib/ |
| | | | docker/overlay2/l/PT5RSNPUEUN6I |
| | | | VNUO3LWVWSV47:/var/lib/docker/o |
| | | | verlay2/l/GDS7TFMIYOI726YERTTHR |
| | | | K6HC6:/var/lib/docker/overlay2/ |
| | | | l/2DU5AJ7HVOTSQYM26Y5UKVKTOG:/v |
| | | | ar/lib/docker/overlay2/l/IZ7CVU |
| | | | 6TH4MRN53DIH33ECHFYO:/var/lib/d |
| | | | ocker/overlay2/l/AQODFFLMLKTAYS |
| | | | EDU37ZMYWE2N,upperdir=/var/lib/ |
| | | | docker/overlay2/bf10ee8bf8ba09b |
| | | | 420641a7a50dea1c0fcfcb211596724 |
| | | | 0df24ce81cd78c4a36/diff,workdir |
| | | | =/var/lib/docker/overlay2/bf10e |
| | | | e8bf8ba09b420641a7a50dea1c0fcfc |
| | | | b2115967240df24ce81cd78c4a36/wo |
| | | | rk,xino=off |
| proc | /proc | proc | rw,nosuid,nodev,noexec,relatime |
| tmpfs | /dev | tmpfs | rw,nosuid,size=65536k,mode=755 |
| devpts | /dev/pts | devpts | rw,nosuid,noexec,relatime,gid=5 |
| | | | ,mode=620,ptmxmode=666 |
| sysfs | /sys | sysfs | rw,nosuid,nodev,noexec,relatime |
| tmpfs | /sys/fs/cgroup | tmpfs | rw,nosuid,nodev,noexec,relatime |
| | | | ,mode=755 |
| cgroup | /sys/fs/cgroup/systemd | cgroup | rw,nosuid,nodev,noexec,relatime |
| | | | ,xattr,name=systemd |
| cgroup | /sys/fs/cgroup/net_cls,net_prio | cgroup | rw,nosuid,nodev,noexec,relatime |
| | | | ,net_cls,net_prio |
| cgroup | /sys/fs/cgroup/cpuset | cgroup | rw,nosuid,nodev,noexec,relatime |
| | | | ,cpuset |
| cgroup | /sys/fs/cgroup/freezer | cgroup | rw,nosuid,nodev,noexec,relatime |
| | | | ,freezer |
| cgroup | /sys/fs/cgroup/perf_event | cgroup | rw,nosuid,nodev,noexec,relatime |
| | | | ,perf_event |
| cgroup | /sys/fs/cgroup/cpu,cpuacct | cgroup | rw,nosuid,nodev,noexec,relatime |
| | | | ,cpu,cpuacct |
| cgroup | /sys/fs/cgroup/devices | cgroup | rw,nosuid,nodev,noexec,relatime |
| | | | ,devices |
| cgroup | /sys/fs/cgroup/blkio | cgroup | rw,nosuid,nodev,noexec,relatime |
| | | | ,blkio |
| cgroup | /sys/fs/cgroup/pids | cgroup | rw,nosuid,nodev,noexec,relatime |
| | | | ,pids |
| cgroup | /sys/fs/cgroup/rdma | cgroup | rw,nosuid,nodev,noexec,relatime |
| | | | ,rdma |
| cgroup | /sys/fs/cgroup/hugetlb | cgroup | rw,nosuid,nodev,noexec,relatime |
| | | | ,hugetlb |
| cgroup | /sys/fs/cgroup/memory | cgroup | rw,nosuid,nodev,noexec,relatime |
| | | | ,memory |
| mqueue | /dev/mqueue | mqueue | rw,nosuid,nodev,noexec,relatime |
| shm | /dev/shm | tmpfs | rw,nosuid,nodev,noexec,relatime |
| | | | ,size=65536k |
| /dev/sdb1 | /usr/sbin/docker-init | ext4 | ro,relatime,discard |
| /dev/sdb1 | /vscode | ext4 | rw,relatime,discard |
| /dev/loop0 | /workspaces | ext4 | rw,nodev,relatime |
| /dev/sda1 | /tmp | ext4 | rw,relatime |
| /dev/sdb1 | /.codespaces/bin | ext4 | rw,relatime,discard |
| /dev/loop0 | /etc/resolv.conf | ext4 | rw,nodev,relatime |
| /dev/loop0 | /etc/hostname | ext4 | rw,nodev,relatime |
| /dev/loop0 | /etc/hosts | ext4 | rw,nodev,relatime |
| /dev/loop0 | /workspaces/.codespaces/.persis | ext4 | rw,nodev,relatime |
| | tedshare | | |
| /dev/sdb1 | /workspaces/.codespaces/shared | ext4 | rw,relatime,discard |
| /dev/loop0 | /var/lib/docker | ext4 | rw,nodev,relatime |
| none | /sys/kernel/security | securityfs | rw,relatime |
| overlay | /var/lib/docker/overlay2/cbf37e | overlay | rw,relatime,lowerdir=/var/lib/d |
| | a880fce4a0e82ad80c83aa78a7cf339 | | ocker/overlay2/l/2JQTCVLSDCNCSJ |
| | 3825c7f6c5772be64f8d2caacce/mer | | YHCTACXPVPU3:/var/lib/docker/ov |
| | ged | | erlay2/l/ZC4XRAP2SVFEA7J4HBA6DP |
| | | | Y4YR:/var/lib/docker/overlay2/l |
| | | | /FPFEUR2UUFEBQYMNOL4NIW7CSD,upp |
| | | | erdir=/var/lib/docker/overlay2/ |
| | | | cbf37ea880fce4a0e82ad80c83aa78a |
| | | | 7cf3393825c7f6c5772be64f8d2caac |
| | | | ce/diff,workdir=/var/lib/docker |
| | | | /overlay2/cbf37ea880fce4a0e82ad |
| | | | 80c83aa78a7cf3393825c7f6c5772be |
| | | | 64f8d2caacce/work,xino=off |
| nsfs | /run/docker/netns/f73869d81728 | nsfs | rw |
| overlay | /var/lib/docker/overlay2/731dcb | overlay | rw,relatime,lowerdir=/var/lib/d |
| | d3ac53d1ef8127fc27872f09995486f | | ocker/overlay2/l/57WYWOOED5SOFB |
| | 375bbba5a3fc13cb497c0f1cd48/mer | | CDT3HMMBQ7Z5:/var/lib/docker/ov |
| | ged | | erlay2/l/Y4YS7MTXWFLPXFPXWVNMUK |
| | | | WE5O:/var/lib/docker/overlay2/l |
| | | | /4KWOFQIGEJOF5PCJZWLRFPFS4Y:/va |
| | | | r/lib/docker/overlay2/l/M6CHIDG |
| | | | V56GZODY3GPUWXVZCWS:/var/lib/do |
| | | | cker/overlay2/l/CESOC5KKFNG23EI |
| | | | 4D5TL3OPLHR:/var/lib/docker/ove |
| | | | rlay2/l/NOUUEUH7WGP3VZNBJXEABSH |
| | | | H7A:/var/lib/docker/overlay2/l/ |
| | | | WSWF3VVTD4NWFHX3SI7OBKJKNX:/var |
| | | | /lib/docker/overlay2/l/HV4W7ZRC |
| | | | A7KV6PDBYKQUUSJDXA:/var/lib/doc |
| | | | ker/overlay2/l/B75S7YKA6J3RTGFH |
| | | | BVUKY2XLQP:/var/lib/docker/over |
| | | | lay2/l/OMJZONY7MGOUWLJNBY3VLN5N |
| | | | KM:/var/lib/docker/overlay2/l/Z |
| | | | FLHEDHCK5HFIPVIFBTBWPCXVM:/var/ |
| | | | lib/docker/overlay2/l/WI7L2NNBZ |
| | | | RKNUGTJEUSV2ZRKXO,upperdir=/var |
| | | | /lib/docker/overlay2/731dcbd3ac |
| | | | 53d1ef8127fc27872f09995486f375b |
| | | | bba5a3fc13cb497c0f1cd48/diff,wo |
| | | | rkdir=/var/lib/docker/overlay2/ |
| | | | 731dcbd3ac53d1ef8127fc27872f099 |
| | | | 95486f375bbba5a3fc13cb497c0f1cd |
| | | | 48/work,xino=off |
| nsfs | /run/docker/netns/738249662b75 | nsfs | rw |
+------------+---------------------------------+------------+---------------------------------+
- Looking at the mount it seems that we have a lot of flexibility in the container. Look at the mount over
/sysand/proc
And now let's check capabilities too.
$ ./kdigger dig capabilities
### CAPABILITIES ###
Comments:
- The bounding set contains 38 caps and you have CAP_SYS_ADMIN, you might be running a privileged container, check the number of devices available.
- NoNewPrivs flag is set to false.
+-------------+--------------------------------------------------------------------+
| SET | CAPABILITIES |
+-------------+--------------------------------------------------------------------+
| inheritable | [] |
| bounding | [chown dac_override dac_read_search fowner fsetid kill setgid |
| | setuid setpcap linux_immutable net_bind_service net_broadcast |
| | net_admin net_raw ipc_lock ipc_owner sys_module sys_rawio |
| | sys_chroot sys_ptrace sys_pacct sys_admin sys_boot sys_nice |
| | sys_resource sys_time sys_tty_config mknod lease audit_write |
| | audit_control setfcap mac_override mac_admin syslog wake_alarm |
| | block_suspend audit_read] |
| ambient | [] |
| effective | [] |
| permitted | [] |
+-------------+--------------------------------------------------------------------+
And here we go, Looks like we have SYS_ADMIN and if not all then we can do most of the things in the cluster.
- Now in codespaces let's create a kubernetes cluster and see what all we can do there. This is just to simulate what kdigger can do.
$ ./kdigger dig adm -s
### ADMISSION ###
+------------------------+---------+-------+
| POD | SUCCESS | ERROR |
+------------------------+---------+-------+
| hostPathPod | true | |
| privilegedPod | true | |
| runAsRootPod | true | |
| privilegeEscalationPod | true | |
| hostNetworkPod | true | |
| hostPIDPod | true | |
+------------------------+---------+-------+
- The admission plugin will give you a check on the types of pods you're can run with whether it's allowed or not.
- You can also check what you're authorized to do in the cluster.
$ ./kdigger dig auth
### AUTHORIZATION ###
Comments:
- Checking current context/token permissions in the "default" namespace.
+---------------------------------+-----------------+----------------+----------+
| RESOURCES | NONRESOURCEURLS | RESSOURCENAMES | VERBS |
+---------------------------------+-----------------+----------------+----------+
| *.* | [] | [] | [*] |
| | [*] | [] | [*] |
| selfsubjectaccessreviews.author | [] | [] | [create] |
| ization.k8s.io | | | |
| selfsubjectrulesreviews.authori | [] | [] | [create] |
| zation.k8s.io | | | |
| | [/api/*] | [] | [get] |
| | [/api] | [] | [get] |
| | [/apis/*] | [] | [get] |
| | [/apis] | [] | [get] |
| | [/healthz] | [] | [get] |
| | [/healthz] | [] | [get] |
| | [/livez] | [] | [get] |
| | [/livez] | [] | [get] |
| | [/openapi/*] | [] | [get] |
| | [/openapi] | [] | [get] |
| | [/readyz] | [] | [get] |
| | [/readyz] | [] | [get] |
| | [/version/] | [] | [get] |
| | [/version/] | [] | [get] |
| | [/version] | [] | [get] |
| | [/version] | [] | [get] |
+---------------------------------+-----------------+----------------+----------+
- It's same as
kubectl auth can-i --list
And the list goes on. One of the most important features I found is that you can use flags to generate vulnerable pods with busybox image buy default.
- You can explore more buckets that ships along with kdigger. I'm looking forward to use this tool.