https://jira.akraino.org This file is an XML representation of an issue en-us 9.4.5 940005 11-04-2023 https://jira.akraino.org/browse/ICN-24 Integrated Cloud Native NFV <p><b>Prerequisite:</b> KUD installation step and documentation</p> <p><b>Resources:</b> Baremetal server, Storage device (/dev/sdb etc)</p> <p><b>Components Version:</b> OS Ubuntu 18.04, KUD version (Kubespray 2.9.0, Kubernetes 1.13.5), Rook 1.0.4, Ceph 13.2.2 (Mimic)</p> <p><b>Task Description:</b> Integration of Rook daemonset with KUD, bring up Rook operator and Ceph cluster in each K8s Node, and config Ceph-monitor and Ceph-osd with certain policy, provide storage provisioning function to K8s application.</p> <p><b>Testing:</b> Checking Rook operator and Ceph cluster status and scheduling mechanism, and check the CSI function for volumes.</p> <p><b>Expected Deliverables:</b> Daemonset yaml, install_package, configuration and launching script along with KUD integration. </p> <p><b>Gerrit Patch:</b> <a href="https://gerrit.akraino.org/r/#/c/icn/+/1282/" class="external-link" target="_blank" rel="nofollow noopener">https://gerrit.akraino.org/r/#/c/icn/+/1282/</a></p> <p> </p> ICN-24

Integrating Rook plugin with KuD

Story Medium Done Done Tingjie Chen Kuralamudhan Ramakrishnan Wed, 31 Jul 2019 01:24:37 +0000 Tue, 20 Aug 2019 02:07:36 +0000 Thu, 15 Aug 2019 01:27:26 +0000 0 3 <p>The Rook operator and Ceph cluster deployment information, by default the Ceph version is Mimic: v13.2.2</p> <p>----------------------------------------------------------</p> <p>$ kubectl get pod -n rook-ceph<br/> NAME READY STATUS RESTARTS AGE<br/> csi-cephfsplugin-provisioner-0 2/2 Running 0 10m<br/> csi-cephfsplugin-z8r2t 2/2 Running 0 10m<br/> csi-rbdplugin-977dn 2/2 Running 0 10m<br/> csi-rbdplugin-provisioner-0 4/4 Running 0 10m<br/> rook-ceph-agent-gctv8 1/1 Running 0 10m<br/> rook-ceph-mgr-a-7d5766c65d-bx49r 1/1 Running 0 9m17s<br/> rook-ceph-mon-a-94665555f-zr7pl 1/1 Running 0 9m54s<br/> rook-ceph-mon-b-54d5d78659-rqzkb 1/1 Running 0 9m45s<br/> rook-ceph-mon-c-67dbf65685-rjnlf 1/1 Running 0 9m31s<br/> rook-ceph-operator-948f8f84c-749zb 1/1 Running 0 11m<br/> rook-ceph-osd-0-6ff8fb6d6b-hcrlx 1/1 Running 0 8m57s<br/> rook-ceph-osd-prepare-localhost-pk2tc 0/2 Completed 0 9m3s<br/> rook-ceph-tools-8b46fc6f-lqn5q 1/1 Running 0 11m<br/> rook-discover-wnbqj 1/1 Running 0 10m<br/> marvin@localhost:~/rook/rook_yaml$<br/> marvin@localhost:~/rook/rook_yaml$ kubectl exec -ti rook-ceph-operator-948f8f84c-749zb -n rook-ceph &#8211; bash<br/> <span class="error">&#91;root@rook-ceph-operator-948f8f84c-749zb /&#93;</span># ceph -s<br/> cluster:<br/> id: 81afbf78-e3d4-4651-a65e-194ddc28bad0<br/> health: HEALTH_OK</p> <p>services:<br/> mon: 3 daemons, quorum b,a,c<br/> mgr: a(active)<br/> osd: 1 osds: 1 up, 1 in</p> <p>data:<br/> pools: 0 pools, 0 pgs<br/> objects: 0 objects, 0 B<br/> usage: 116 GiB used, 1.6 TiB / 1.7 TiB avail<br/> pgs:</p> <p><span class="error">&#91;root@rook-ceph-operator-948f8f84c-749zb /&#93;</span># ceph version<br/> ceph version 13.2.2 (02899bfda814146b021136e9d8e80eba494e1126) mimic (stable)</p> <p> </p> <p><a href="https://jira.akraino.org/secure/ViewProfile.jspa?name=r.kuralamudhan" class="user-hover" rel="r.kuralamudhan">r.kuralamudhan</a> <a href="https://jira.akraino.org/secure/ViewProfile.jspa?name=akhilakishore" class="user-hover" rel="akhilakishore">akhilakishore</a></p> <p>I have upgrade to K8s 1.13.5 and deploy Rook with CSI successfully, you can refer the changes in KUD code base.</p> <p>if something is wrong with kubectl server version, you can manually upgrade as command: kubeadm upgrade apply v1.13.5</p> <p>after upgraded, the information as following:</p> <p>$ kubectl version<br/> Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.5", GitCommit:"2166946f41b36dea2c4626f90a77706f426cdea2", GitTreeState:"clean", BuildDate:"2019-03-25T15:19:22Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}<br/> Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.5", GitCommit:"2166946f41b36dea2c4626f90a77706f426cdea2", GitTreeState:"clean", BuildDate:"2019-03-25T15:19:22Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}</p> <p> </p> <p>The code changes in KUD as following:</p> <p>--------------------------------------------------------------------------------------------------</p> <p>diff --git a/kud/deployment_infra/playbooks/kud-vars.yml b/kud/deployment_infra/playbooks/kud-vars.yml<br/> index b11672f..eb9b24c 100644<br/> — a/kud/deployment_infra/playbooks/kud-vars.yml<br/> +++ b/kud/deployment_infra/playbooks/kud-vars.yml<br/> @@ -57,8 +57,8 @@ ovn4nfv_source_type: "source"<br/> ovn4nfv_version: aa14577f6bc672bc8622edada8a487825fdebce1<br/> ovn4nfv_url: "https://git.opnfv.org/ovn4nfv-k8s-plugin/"</p> <p><font color="#ff0000">-go_version: '1.12.4'</font><br/> <font color="#ff0000">-kubespray_version: 2.8.2</font><br/> <font color="#ff0000">-helm_client_version: 2.9.1</font><br/> <font color="#00875a">+go_version: '1.12.5'</font><br/> <font color="#00875a">+kubespray_version: 2.9.0</font><br/> <font color="#00875a">+helm_client_version: 2.13.1</font></p> <ol> <li>kud playbooks not compatible with 2.8.0 - see MULTICLOUD-634<br/> ansible_version: 2.7.10</li> </ol> <p>diff --git a/kud/hosting_providers/vagrant/inventory/group_vars/k8s-cluster.yml b/kud/hosting_providers/vagrant/inventory/group_vars/k8s-cluster.yml<br/> index 9966ba8..cacb4b3 100644<br/> — a/kud/hosting_providers/vagrant/inventory/group_vars/k8s-cluster.yml<br/> +++ b/kud/hosting_providers/vagrant/inventory/group_vars/k8s-cluster.yml<br/> @@ -48,7 +48,7 @@ local_volumes_enabled: true<br/> local_volume_provisioner_enabled: true</p> <ol> <li> <ol> <li>Change this to use another Kubernetes version, e.g. a current beta release<br/> <font color="#de350b">-kube_version: v1.12.3</font><br/> <font color="#00875a">+kube_version: v1.13.5</font></li> </ol> </li> </ol> <p><font color="#172b4d"># Helm deployment</font><br/> helm_enabled: true</p> <p><a href="https://jira.akraino.org/secure/ViewProfile.jspa?name=tingjiec" class="user-hover" rel="tingjiec">tingjiec</a> Thanks for bringing it up. It is really good that we identified issues here. <a href="https://jira.akraino.org/secure/ViewProfile.jspa?name=akhilakishore" class="user-hover" rel="akhilakishore">akhilakishore</a> could you please suggest us, is that possible for us by Aug 15th to update K8s version ? or you afraid that it will break current KuD offline operation. </p> <p><a href="https://jira.akraino.org/secure/ViewProfile.jspa?name=r.kuralamudhan" class="user-hover" rel="r.kuralamudhan">r.kuralamudhan</a> May I ask the version of K8s in KUD deployment env? since in Rook we have CSI supported, which can have storage service provision for K8s applications.</p> <p>And the CSI 1.0 require the K8s version must be &gt;= 1.13, currently in our KUD env, the K8s version is 1.12.3, so the CSI plugins will not bring up and the functionality is invalid.</p> <p>I have submit gerrit patch for Rook deployment: <a href="https://gerrit.akraino.org/r/c/icn/+/1282" class="external-link" target="_blank" rel="nofollow noopener">https://gerrit.akraino.org/r/c/icn/+/1282</a></p> <p>There has Ceph storage(osd), by default I set to save in folder: /rook/storage-dir for database and journal for 10GB each, please reserve enough space in local disk.</p> <p>Just sent out the setup instructions to them</p> <p><a href="https://jira.akraino.org/secure/ViewProfile.jspa?name=akhilakishore" class="user-hover" rel="akhilakishore">akhilakishore</a> <span class="nobr"><a href="mailto:ritu.sood@intel.com" class="external-link" target="_blank" rel="nofollow noopener">ritu.sood@intel.com^{<img class="rendericon" src="https://jira.akraino.org/images/icons/mail_small.gif" height="12" width="13" align="absmiddle" alt="" border="0"/>}</a></span> Please provide the KUD installation to <a href="https://jira.akraino.org/secure/ViewProfile.jspa?name=tingjiec" class="user-hover" rel="tingjiec">tingjiec</a> <a href="https://jira.akraino.org/secure/ViewProfile.jspa?name=hle2" class="user-hover" rel="hle2">hle2</a> -  <a href="https://github.com/onap/multicloud-k8s/tree/master/kud" class="external-link" target="_blank" rel="nofollow noopener">https://github.com/onap/multicloud-k8s/tree/master/kud &amp; </a> <a href="https://github.com/onap/multicloud-k8s/tree/master/kud/deployment_infra/playbooks" class="external-link" target="_blank" rel="nofollow noopener">https://github.com/onap/multicloud-k8s/tree/master/kud/deployment_infra/playbooks</a></p> <p>AR for <a href="https://jira.akraino.org/secure/ViewProfile.jspa?name=r.kuralamudhan" class="user-hover" rel="r.kuralamudhan">r.kuralamudhan</a> to check the kubespray k8s version ?</p> <p>Tin - AR: Can you send me the details on Storage device?</p> Development Epic Link ICN-6 Rank 0|i0013r: Sprint ICN Sprint 1 ICN Sprint 2 Story Points 4.0