[ICN-654] K8s deployment with Nodus results in OOMKilled

Thu, 18 Jan 2024 21:45:28 +0000

When installing Kubernetes clusters on the 5GNS clusters with the Nodus CNI, along with the EMCO services (not verified to be connected yet), the nfn-agent-xyz pod crashes and restarts with OOMKilled. This doesn't seem to occur when idling, but will often happen when installing or uninstall EMCO via helm or killing EMCO pods manually.

Attaching some logs I was able to capture when this was first seen.

Additional situations that may be related:

Pods take a long time to start up or get destroyed (across all EMCO microservices it has been seen to take up to 5 minutes to destroy all pods.
Using Flannel as the CNI plugin instead of Nodus doesn't result in OOMKilled crashes, but causes restarts of the kube-flannel-xyz pod with no indication of any error in the log files. However, increasing the memory limit and request for flannel seems to solve this. See the flannel dmesg log to see the memory commitment issues when using the default '50Mi'.

Akraino JIRA

[ICN-654] K8s deployment with Nodus results in OOMKilled