[ICN-654] K8s deployment with Nodus results in OOMKilled Created: 08/Jun/22  Updated: 18/Jan/24

Status: To Do
Project: Integrated Cloud Native NFV
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Medium
Reporter: Igor DC Assignee: Kuralamudhan Ramakrishnan
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File nfn-agent-logs-p.txt     Text File nodus-dmesg.txt    
Epic Link: 5G net slicing

 Description   

When installing Kubernetes clusters on the 5GNS clusters with the Nodus CNI, along with the EMCO services (not verified to be connected yet), the nfn-agent-xyz pod crashes and restarts with OOMKilled. This doesn't seem to occur when idling, but will often happen when installing or uninstall EMCO via helm or killing EMCO pods manually.

 

Attaching some logs I was able to capture when this was first seen.

 

Additional situations that may be related:

  • Pods take a long time to start up or get destroyed (across all EMCO microservices it has been seen to take up to 5 minutes to destroy all pods.
  • Using Flannel as the CNI plugin instead of Nodus doesn't result in OOMKilled crashes, but causes restarts of the kube-flannel-xyz pod with no indication of any error in the log files. However, increasing the memory limit and request for flannel seems to solve this. See the flannel dmesg log to see the memory commitment issues when using the default '50Mi'.

 



 Comments   
Comment by Igor DC [ 08/Jun/22 ]

v1vk palaniap 

Generated at Sat Feb 10 06:01:43 UTC 2024 using Jira 9.4.5#940005-sha1:e3094934eac4fd8653cf39da58f39364fb9cc7c1.