#Docker swarm vs kubernetes self killing manual
#Docker swarm vs kubernetes self killing code
#Docker swarm vs kubernetes self killing series

A good example for this is вЂњcontainerdвЂќ.ДЇigure 5: Docker stack! Image credit: internet It provides ability to run containers.Д«) Image management: How images are packed, unpacked, pushed, pulled etc comes under this umbrella.

#Docker swarm vs kubernetes self killing manual

Takeaway 1: Either Container Runtime/Interface or OS Kernel killed my process because supposedly it was misbehaving and causing the out-of-memory issue!Д®ssentially, I am ruling out the manual kill because that was simply not the case! Deep-dive into factors at play hereД¬ontainer runtime (in fig 4) is responsible for two things:ДЄ) Running containers: Comes from open container initiative (OCI) (about 2013) open sourced by Docker called вЂњruncвЂќ. System-level failure and to nip mischieve in the bud. OOMKilled represent a kill event (SIGKILL) triggered to a process because someone in-charge suspected of the process to be the culprit of a memory surge that may lead to an out of memory event. It can also kill and almost always kills badly behaving processes! It is the mighty power that be for all the processes on the host including the container process and its children.

OS kernel: The OS kernel is responsible for the life cycle of processes running on the host.

#Docker swarm vs kubernetes self killing series

Image Credit: Ian Lewis! Borrowed from his 4 part container runtime series container runtime that I highly recommend readingДЎ.3. It can kill and almost always kills badly behaving pods!ДЇigure 4: Container runtime interface. It communicates through container runtime to manage the container lifecycle. Container Runtime/Interface: Kubelet the process running on the host machine that manages running Kube workload is the power that be for containers. Manually (human): Triggering CTRL+C or using other means of manually sending SIGKILL or even manually killing process.ДЎ.2. SIGKILL on Kube can only be produced using one of the following means:ДЎ.1.

#Docker swarm vs kubernetes self killing code

Let me explain a few things about the failure first:Д®rror code 137 indicates that the container process received the SIGKILL and thus was killed by the OS kernel. The provisioned app pods started to get killed as frequently as every 20 mins or more with error code 137 and reason OOMKilled.ДЇigure 3: The killer is on the loose! - Whodunit? Ok, so the app was deployed and now, we will look at the line of investigation: AppвЂ™s on Kube: day 1 Having said that, itвЂ™s an absolutely acceptable and supported workload. It would be awesome to have constant non-flapping resource requirement needs - so clearly some work needs to happen on the app here. I hear you, the resource utilization is not following a zero gradient line (fig 2)! The average resource requirement, for this app, is very fluctuating yet predictable (in a given range).ДЄt least so we thought looking at our metrics:ДЇigure 2: Average resource requirements of the app when run on VMs or bare metal Other gory details besides resource requirements of the app is irrelevant for this discussion. The workloads are more memory-intensive as they operate on rich multi-media content. The app runs some intensive numpy and Tensorflow computations to produce some artifacts and associative metadata. I am going to talk about this particular incident and the insights I have uncovered about both Kube and Linux kernels. Has also been done on by Line Corp in their excellent blog however, I have a different story to tell! This sparked a massive hunt for the culprit, and some interesting insights were discovered. I am not a total Kube noob!Д«ut, I could not figure out what the fudge was going on actually!Д«esides, this app has been thoroughly tested and profiled and ran fine on bare metal and virtual environments. Now, I know a thing or two about Kubernetes 1, 2. The pods were getting OOMKilled with error code 137 left and right!

A few days ago, I deployed a brand new application onto a self-managed Kubernetes cluster (hereafter referred to as Kube).