[ovs-discuss] SB flows not being created in OVN K8 Stateful set

Brendan Doyle brendan.doyle at oracle.com
Wed Aug 5 15:14:52 UTC 2020


Folks,

I'm stumped here, I have the k8 ovnkube-db-raft Stateful set up and running.
But when I create a simple network, no SB flows are generated.

ovn-nbctl show shows my network. ovn-sbctl show shows the physicals 
systems in my network.
But I can't ping between any hosts because ovn-sbctl lflow-list is 
empty, and there are no
errors or warnings in the logs. The ovn cluster says it is up and healthy.

Anybody got any ideas why this might be?


ovn-nbctl 
--db="tcp:253.255.0.33:6641,tcp:253.255.0.34:6641,tcp:253.255.0.35:6641" 
show
------------------------------------------------------------------------------------------------------------------------------
switch 5eb7330c-bb58-454b-9b0f-78f33e477f9a (ls_vcn1)
     port ls_vcn1_net2-lr_vcn1_net2
         type: router
         addresses: ["40:44:00:00:00:40"]
         router-port: lr_vcn1_net2-ls_vcn1_net2
     port ls_vcn1_net1-lr_vcn1_net1
         type: router
         addresses: ["40:44:00:00:00:30"]
         router-port: lr_vcn1_net1-ls_vcn1_net1
     port 00bff7c0-2e2d-41ba-9485-3b5fa9801365
         addresses: ["52:54:00:e6:4f:46 192.16.1.5"]
     port 47433b54-ac10-42f1-ae84-cc6fbb580297
         addresses: ["52:54:00:be:06:16 192.16.1.6"]
     port 1cb7d760-90b0-4201-9517-88cb2de31c79
         addresses: ["52:54:00:80:d0:c8 192.16.2.5"]
switch c03b6ad3-be18-47a6-951a-65bd1d64d912 (ls_vcn1_backbone)
     port lsb_vcn1_net1-lr_vcn1_net1
         type: router
         router-port: lr_vcn1_net1-lsb_vcn1_net1
     port lsb_vcn1_net2-lr_vcn1_net2
         type: router
         router-port: lr_vcn1_net2-lsb_vcn1_net2
router c931466d-2768-4a4d-9528-6bb3ac530084 (lr_vcn1_net2)
     port lr_vcn1_net2-ls_vcn1_net2
         mac: "40:44:00:00:00:40"
         networks: ["192.16.2.1/24"]
     port lr_vcn1_net2-lsb_vcn1_net2
         mac: "40:44:00:00:00:60"
         networks: ["253.255.130.2/23"]
router 76b42f83-f508-43b7-b1a1-887b3b900740 (lr_vcn1_net1)
     port lr_vcn1_net1-ls_vcn1_net1
         mac: "40:44:00:00:00:30"
         networks: ["192.16.1.1/24"]
     port lr_vcn1_net1-lsb_vcn1_net1
         mac: "40:44:00:00:00:50"
         networks: ["253.255.130.1/23"]

ovn-sbctl 
--db="tcp:253.255.0.33:6642,tcp:253.255.0.34:6642,tcp:253.255.0.35:6642" 
show
---------------------------------------------------------------------------------------------------------------------------
Chassis ca-rain05
     hostname: ca-rain05.us.oracle.com
     Encap geneve
         ip: "253.255.1.1"
         options: {csum="true"}
Chassis ca-rain17
     hostname: ca-rain17.us.oracle.com
     Encap geneve
         ip: "253.255.3.1"
         options: {csum="true"}
Chassis ca-rain03
     hostname: ca-rain03.us.oracle.com
     Encap geneve
         ip: "253.255.0.35"
         options: {csum="true"}
Chassis ca-rain06
     hostname: ca-rain06.us.oracle.com
     Encap geneve
         ip: "253.255.2.1"
         options: {csum="true"}
Chassis ca-rain02
     hostname: ca-rain02.us.oracle.com
     Encap geneve
         ip: "253.255.0.34"
         options: {csum="true"}
Chassis ca-rain01
     hostname: ca-rain01.us.oracle.com
     Encap geneve
         ip: "253.255.0.33"
         options: {csum="true"}

ovn-sbctl 
--db="tcp:253.255.0.33:6642,tcp:253.255.0.34:6642,tcp:253.255.0.35:6642" 
lflow-list
----------------------------------------------------------------------------------------------------------------------------------
empty

k8 stuff
----------
kubectl get services -o wide --namespace=ovn-kubernetes
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP PORT(S)             
AGE   SELECTOR
ovnkube-db   ClusterIP   None         <none> 6641/TCP,6642/TCP   33m   
<none>

  kubectl get StatefulSet -o wide --namespace=ovn-kubernetes
NAME         READY   AGE   CONTAINERS          IMAGES
ovnkube-db   3/3     33m   nb-ovsdb,sb-ovsdb 
ovn-daemonset:latest,ovn-daemonset:latest

kubectl get pods -o wide --namespace=ovn-kubernetes
NAME           READY   STATUS    RESTARTS   AGE   IP NODE        
NOMINATED NODE   READINESS GATES
ovnkube-db-0   2/2     Running   0          33m   253.255.0.33 
ca-rain01   <none>           <none>
ovnkube-db-1   2/2     Running   0          33m   253.255.0.35 
ca-rain03   <none>           <none>
ovnkube-db-2   2/2     Running   0          33m   253.255.0.34 
ca-rain02   <none>           <none>

kubectl describe StatefulSet --namespace=ovn-kubernetes
Name:               ovnkube-db
Namespace:          ovn-kubernetes
CreationTimestamp:  Wed, 05 Aug 2020 10:22:50 -0400
Selector:           name=ovnkube-db
Labels:             <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"apps/v1","kind":"StatefulSet","metadata":{"annotations":{"kubernetes.io/description":"This 
statefulset launches the OVN Nor...
                     kubernetes.io/description: This statefulset 
launches the OVN Northbound/Southbound Database raft clusters.
Replicas:           3 desired | 3 total
Update Strategy:    RollingUpdate
   Partition:        824644587688
Pods Status:        3 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
   Labels:           component=network
                     kubernetes.io/os=linux
                     name=ovnkube-db
                     type=infra
   Annotations:      scheduler.alpha.kubernetes.io/critical-pod:
   Service Account:  ovn
   Containers:
    nb-ovsdb:
     Image:      ovn-daemonset:latest
     Port:       <none>
     Host Port:  <none>
     Command:
       /root/ovnkube.sh
       nb-ovsdb-raft
     Requests:
       cpu:      100m
       memory:   300Mi
     Readiness:  exec [/usr/bin/ovn-kube-util readiness-probe -t 
ovnnb-db-raft] delay=30s timeout=30s period=60s #success=1 #failure=3
     Environment:
       OVN_DAEMONSET_VERSION:       3
       OVN_LOGLEVEL_NB:             -vconsole:info -vfile:info
       K8S_APISERVER:               <set to the key 'k8s_apiserver' of 
config map 'ovn-config'>  Optional: false
       OVN_KUBERNETES_NAMESPACE:     (v1:metadata.namespace)
       POD_NAME:                     (v1:metadata.name)
       K8S_NODE_IP:                  (v1:status.hostIP)
       OVN_SSL_ENABLE:              no
       OVN_NB_RAFT_ELECTION_TIMER:  1000
       OVN_NB_PORT:                 6641
       OVN_NB_RAFT_PORT:            6643
     Mounts:
       /etc/openvswitch/ from host-var-lib-ovs (rw)
       /etc/ovn/ from host-var-lib-ovs (rw)
       /ovn-cert from host-ovn-cert (ro)
       /var/log/openvswitch/ from host-var-log-ovs (rw)
       /var/log/ovn/ from host-var-log-ovs (rw)
       /var/run/openvswitch/ from host-var-run-ovs (rw)
       /var/run/ovn/ from host-var-run-ovs (rw)
    sb-ovsdb:
     Image:      ovn-daemonset:latest
     Port:       <none>
     Host Port:  <none>
     Command:
       /root/ovnkube.sh
       sb-ovsdb-raft
     Requests:
       cpu:      100m
       memory:   300Mi
     Readiness:  exec [/usr/bin/ovn-kube-util readiness-probe -t 
ovnsb-db-raft] delay=30s timeout=30s period=60s #success=1 #failure=3
     Environment:
       OVN_DAEMONSET_VERSION:       3
       OVN_LOGLEVEL_SB:             -vconsole:info -vfile:info
       K8S_APISERVER:               <set to the key 'k8s_apiserver' of 
config map 'ovn-config'>  Optional: false
       OVN_KUBERNETES_NAMESPACE:     (v1:metadata.namespace)
       POD_NAME:                     (v1:metadata.name)
       K8S_NODE_IP:                  (v1:status.hostIP)
       OVN_SSL_ENABLE:              no
       OVN_SB_RAFT_ELECTION_TIMER:  1000
       OVN_SB_PORT:                 6642
       OVN_SB_RAFT_PORT:            6644
     Mounts:
       /etc/openvswitch/ from host-var-lib-ovs (rw)
       /etc/ovn/ from host-var-lib-ovs (rw)
       /ovn-cert from host-ovn-cert (ro)
       /var/log/openvswitch/ from host-var-log-ovs (rw)
       /var/log/ovn/ from host-var-log-ovs (rw)
       /var/run/openvswitch/ from host-var-run-ovs (rw)
       /var/run/ovn/ from host-var-run-ovs (rw)
   Volumes:
    host-var-log-ovs:
     Type:          HostPath (bare host directory volume)
     Path:          /usr/var/log/ovn
     HostPathType:
    host-var-lib-ovs:
     Type:          HostPath (bare host directory volume)
     Path:          /usr/etc/ovn
     HostPathType:
    host-var-run-ovs:
     Type:          HostPath (bare host directory volume)
     Path:          /usr/var/run/ovn/
     HostPathType:
    host-ovn-cert:
     Type:          HostPath (bare host directory volume)
     Path:          /usr/etc/ovn
     HostPathType:  DirectoryOrCreate
Volume Claims:     <none>
Events:
   Type    Reason            Age   From                    Message
   ----    ------            ----  ----                    -------
   Normal  SuccessfulCreate  36m   statefulset-controller  create Pod 
ovnkube-db-0 in StatefulSet ovnkube-db successful
   Normal  SuccessfulCreate  36m   statefulset-controller  create Pod 
ovnkube-db-1 in StatefulSet ovnkube-db successful
   Normal  SuccessfulCreate  36m   statefulset-controller  create Pod 
ovnkube-db-2 in StatefulSet ovnkube-db successful


kubectl describe services --namespace=ovn-kubernetes
Name:              ovnkube-db
Namespace:         ovn-kubernetes
Labels:            <none>
Annotations:       kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"ovnkube-db","namespace":"ovn-kubernetes"},"spec":{"clusterIP":"No...
Selector:          <none>
Type:              ClusterIP
IP:                None
Port:              north  6641/TCP
TargetPort:        6641/TCP
Endpoints: 253.255.0.33:6641,253.255.0.34:6641,253.255.0.35:6641
Port:              south  6642/TCP
TargetPort:        6642/TCP
Endpoints: 253.255.0.33:6642,253.255.0.34:6642,253.255.0.35:6642
Session Affinity:  None
Events:            <none>


kubectl describe pods --namespace=ovn-kubernetes
Name:               ovnkube-db-0
Namespace:          ovn-kubernetes
Priority:           0
PriorityClassName:  <none>
Node:               ca-rain01/253.255.0.33
Start Time:         Wed, 05 Aug 2020 10:22:50 -0400
Labels:             component=network
                     controller-revision-hash=ovnkube-db-bdb977
                     kubernetes.io/os=linux
                     name=ovnkube-db
                     statefulset.kubernetes.io/pod-name=ovnkube-db-0
                     type=infra
Annotations:        scheduler.alpha.kubernetes.io/critical-pod:
Status:             Running
IP:                 253.255.0.33
Controlled By:      StatefulSet/ovnkube-db
Containers:
   nb-ovsdb:
     Container ID: 
cri-o://f9d13d99d5efdff8e3be0014d59296a31dcd1b4d489bbfe307d3e0f1eed8ea5a
     Image:         ovn-daemonset:latest
     Image ID: 
docker.io/library/ovn-daemonset at sha256:624f03ceb671c8e9176a2d3a6988e2351680064fee060bf8e8cf40a4eb0cb362
     Port:          <none>
     Host Port:     <none>
     Command:
       /root/ovnkube.sh
       nb-ovsdb-raft
     State:          Running
       Started:      Wed, 05 Aug 2020 10:22:50 -0400
     Ready:          True
     Restart Count:  0
     Requests:
       cpu:      100m
       memory:   300Mi
     Readiness:  exec [/usr/bin/ovn-kube-util readiness-probe -t 
ovnnb-db-raft] delay=30s timeout=30s period=60s #success=1 #failure=3
     Environment:
       OVN_DAEMONSET_VERSION:       3
       OVN_LOGLEVEL_NB:             -vconsole:info -vfile:info
       K8S_APISERVER:               <set to the key 'k8s_apiserver' of 
config map 'ovn-config'>  Optional: false
       OVN_KUBERNETES_NAMESPACE:    ovn-kubernetes (v1:metadata.namespace)
       POD_NAME:                    ovnkube-db-0 (v1:metadata.name)
       K8S_NODE_IP:                  (v1:status.hostIP)
       OVN_SSL_ENABLE:              no
       OVN_NB_RAFT_ELECTION_TIMER:  1000
       OVN_NB_PORT:                 6641
       OVN_NB_RAFT_PORT:            6643
     Mounts:
       /etc/openvswitch/ from host-var-lib-ovs (rw)
       /etc/ovn/ from host-var-lib-ovs (rw)
       /ovn-cert from host-ovn-cert (ro)
       /var/log/openvswitch/ from host-var-log-ovs (rw)
       /var/log/ovn/ from host-var-log-ovs (rw)
       /var/run/openvswitch/ from host-var-run-ovs (rw)
       /var/run/ovn/ from host-var-run-ovs (rw)
       /var/run/secrets/kubernetes.io/serviceaccount from 
ovn-token-srjkt (ro)
   sb-ovsdb:
     Container ID: 
cri-o://0fd6d297cc849eef4d5601c8cc3079222cb2e8a47009d70fa1edbaef23a50b0f
     Image:         ovn-daemonset:latest
     Image ID: 
docker.io/library/ovn-daemonset at sha256:624f03ceb671c8e9176a2d3a6988e2351680064fee060bf8e8cf40a4eb0cb362
     Port:          <none>
     Host Port:     <none>
     Command:
       /root/ovnkube.sh
       sb-ovsdb-raft
     State:          Running
       Started:      Wed, 05 Aug 2020 10:22:51 -0400
     Ready:          True
     Restart Count:  0
     Requests:
       cpu:      100m
       memory:   300Mi
     Readiness:  exec [/usr/bin/ovn-kube-util readiness-probe -t 
ovnsb-db-raft] delay=30s timeout=30s period=60s #success=1 #failure=3
     Environment:
       OVN_DAEMONSET_VERSION:       3
       OVN_LOGLEVEL_SB:             -vconsole:info -vfile:info
       K8S_APISERVER:               <set to the key 'k8s_apiserver' of 
config map 'ovn-config'>  Optional: false
       OVN_KUBERNETES_NAMESPACE:    ovn-kubernetes (v1:metadata.namespace)
       POD_NAME:                    ovnkube-db-0 (v1:metadata.name)
       K8S_NODE_IP:                  (v1:status.hostIP)
       OVN_SSL_ENABLE:              no
       OVN_SB_RAFT_ELECTION_TIMER:  1000
       OVN_SB_PORT:                 6642
       OVN_SB_RAFT_PORT:            6644
     Mounts:
       /etc/openvswitch/ from host-var-lib-ovs (rw)
       /etc/ovn/ from host-var-lib-ovs (rw)
       /ovn-cert from host-ovn-cert (ro)
       /var/log/openvswitch/ from host-var-log-ovs (rw)
       /var/log/ovn/ from host-var-log-ovs (rw)
       /var/run/openvswitch/ from host-var-run-ovs (rw)
       /var/run/ovn/ from host-var-run-ovs (rw)
       /var/run/secrets/kubernetes.io/serviceaccount from 
ovn-token-srjkt (ro)
Conditions:
   Type              Status
   Initialized       True
   Ready             True
   ContainersReady   True
   PodScheduled      True
Volumes:
   host-var-log-ovs:
     Type:          HostPath (bare host directory volume)
     Path:          /usr/var/log/ovn
     HostPathType:
   host-var-lib-ovs:
     Type:          HostPath (bare host directory volume)
     Path:          /usr/etc/ovn
     HostPathType:
   host-var-run-ovs:
     Type:          HostPath (bare host directory volume)
     Path:          /usr/var/run/ovn/
     HostPathType:
   host-ovn-cert:
     Type:          HostPath (bare host directory volume)
     Path:          /usr/etc/ovn
     HostPathType:  DirectoryOrCreate
   ovn-token-srjkt:
     Type:        Secret (a volume populated by a Secret)
     SecretName:  ovn-token-srjkt
     Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:
Events:
   Type    Reason     Age   From                Message
   ----    ------     ----  ----                -------
   Normal  Scheduled  37m   default-scheduler   Successfully assigned 
ovn-kubernetes/ovnkube-db-0 to ca-rain01
   Normal  Pulled     37m   kubelet, ca-rain01  Container image 
"ovn-daemonset:latest" already present on machine
   Normal  Created    37m   kubelet, ca-rain01  Created container nb-ovsdb
   Normal  Started    37m   kubelet, ca-rain01  Started container nb-ovsdb
   Normal  Pulled     37m   kubelet, ca-rain01  Container image 
"ovn-daemonset:latest" already present on machine
   Normal  Created    37m   kubelet, ca-rain01  Created container sb-ovsdb
   Normal  Started    37m   kubelet, ca-rain01  Started container sb-ovsdb


Name:               ovnkube-db-1
Namespace:          ovn-kubernetes
Priority:           0
PriorityClassName:  <none>
Node:               ca-rain03/253.255.0.35
Start Time:         Wed, 05 Aug 2020 10:22:50 -0400
Labels:             component=network
                     controller-revision-hash=ovnkube-db-bdb977
                     kubernetes.io/os=linux
                     name=ovnkube-db
                     statefulset.kubernetes.io/pod-name=ovnkube-db-1
                     type=infra
Annotations:        scheduler.alpha.kubernetes.io/critical-pod:
Status:             Running
IP:                 253.255.0.35
Controlled By:      StatefulSet/ovnkube-db
Containers:
   nb-ovsdb:
     Container ID: 
cri-o://329d696874354bb5666da830faaa02efab60e573588c700c7977d85be11f8741
     Image:         ovn-daemonset:latest
     Image ID: 
docker.io/library/ovn-daemonset at sha256:624f03ceb671c8e9176a2d3a6988e2351680064fee060bf8e8cf40a4eb0cb362
     Port:          <none>
     Host Port:     <none>
     Command:
       /root/ovnkube.sh
       nb-ovsdb-raft
     State:          Running
       Started:      Wed, 05 Aug 2020 10:22:50 -0400
     Ready:          True
     Restart Count:  0
     Requests:
       cpu:      100m
       memory:   300Mi
     Readiness:  exec [/usr/bin/ovn-kube-util readiness-probe -t 
ovnnb-db-raft] delay=30s timeout=30s period=60s #success=1 #failure=3
     Environment:
       OVN_DAEMONSET_VERSION:       3
       OVN_LOGLEVEL_NB:             -vconsole:info -vfile:info
       K8S_APISERVER:               <set to the key 'k8s_apiserver' of 
config map 'ovn-config'>  Optional: false
       OVN_KUBERNETES_NAMESPACE:    ovn-kubernetes (v1:metadata.namespace)
       POD_NAME:                    ovnkube-db-1 (v1:metadata.name)
       K8S_NODE_IP:                  (v1:status.hostIP)
       OVN_SSL_ENABLE:              no
       OVN_NB_RAFT_ELECTION_TIMER:  1000
       OVN_NB_PORT:                 6641
       OVN_NB_RAFT_PORT:            6643
     Mounts:
       /etc/openvswitch/ from host-var-lib-ovs (rw)
       /etc/ovn/ from host-var-lib-ovs (rw)
       /ovn-cert from host-ovn-cert (ro)
       /var/log/openvswitch/ from host-var-log-ovs (rw)
       /var/log/ovn/ from host-var-log-ovs (rw)
       /var/run/openvswitch/ from host-var-run-ovs (rw)
       /var/run/ovn/ from host-var-run-ovs (rw)
       /var/run/secrets/kubernetes.io/serviceaccount from 
ovn-token-srjkt (ro)
   sb-ovsdb:
     Container ID: 
cri-o://e8807808b518b8a71e2a4e439a34fa3e1ddf201dc522214a487a8f711fb82289
     Image:         ovn-daemonset:latest
     Image ID: 
docker.io/library/ovn-daemonset at sha256:624f03ceb671c8e9176a2d3a6988e2351680064fee060bf8e8cf40a4eb0cb362
     Port:          <none>
     Host Port:     <none>
     Command:
       /root/ovnkube.sh
       sb-ovsdb-raft
     State:          Running
       Started:      Wed, 05 Aug 2020 10:22:51 -0400
     Ready:          True
     Restart Count:  0
     Requests:
       cpu:      100m
       memory:   300Mi
     Readiness:  exec [/usr/bin/ovn-kube-util readiness-probe -t 
ovnsb-db-raft] delay=30s timeout=30s period=60s #success=1 #failure=3
     Environment:
       OVN_DAEMONSET_VERSION:       3
       OVN_LOGLEVEL_SB:             -vconsole:info -vfile:info
       K8S_APISERVER:               <set to the key 'k8s_apiserver' of 
config map 'ovn-config'>  Optional: false
       OVN_KUBERNETES_NAMESPACE:    ovn-kubernetes (v1:metadata.namespace)
       POD_NAME:                    ovnkube-db-1 (v1:metadata.name)
       K8S_NODE_IP:                  (v1:status.hostIP)
       OVN_SSL_ENABLE:              no
       OVN_SB_RAFT_ELECTION_TIMER:  1000
       OVN_SB_PORT:                 6642
       OVN_SB_RAFT_PORT:            6644
     Mounts:
       /etc/openvswitch/ from host-var-lib-ovs (rw)
       /etc/ovn/ from host-var-lib-ovs (rw)
       /ovn-cert from host-ovn-cert (ro)
       /var/log/openvswitch/ from host-var-log-ovs (rw)
       /var/log/ovn/ from host-var-log-ovs (rw)
       /var/run/openvswitch/ from host-var-run-ovs (rw)
       /var/run/ovn/ from host-var-run-ovs (rw)
       /var/run/secrets/kubernetes.io/serviceaccount from 
ovn-token-srjkt (ro)
Conditions:
   Type              Status
   Initialized       True
   Ready             True
   ContainersReady   True
   PodScheduled      True
Volumes:
   host-var-log-ovs:
     Type:          HostPath (bare host directory volume)
     Path:          /usr/var/log/ovn
     HostPathType:
   host-var-lib-ovs:
     Type:          HostPath (bare host directory volume)
     Path:          /usr/etc/ovn
     HostPathType:
   host-var-run-ovs:
     Type:          HostPath (bare host directory volume)
     Path:          /usr/var/run/ovn/
     HostPathType:
   host-ovn-cert:
     Type:          HostPath (bare host directory volume)
     Path:          /usr/etc/ovn
     HostPathType:  DirectoryOrCreate
   ovn-token-srjkt:
     Type:        Secret (a volume populated by a Secret)
     SecretName:  ovn-token-srjkt
     Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:
Events:
   Type    Reason     Age   From                Message
   ----    ------     ----  ----                -------
   Normal  Scheduled  37m   default-scheduler   Successfully assigned 
ovn-kubernetes/ovnkube-db-1 to ca-rain03
   Normal  Pulled     37m   kubelet, ca-rain03  Container image 
"ovn-daemonset:latest" already present on machine
   Normal  Created    37m   kubelet, ca-rain03  Created container nb-ovsdb
   Normal  Started    37m   kubelet, ca-rain03  Started container nb-ovsdb
   Normal  Pulled     37m   kubelet, ca-rain03  Container image 
"ovn-daemonset:latest" already present on machine
   Normal  Created    37m   kubelet, ca-rain03  Created container sb-ovsdb
   Normal  Started    37m   kubelet, ca-rain03  Started container sb-ovsdb


Name:               ovnkube-db-2
Namespace:          ovn-kubernetes
Priority:           0
PriorityClassName:  <none>
Node:               ca-rain02/253.255.0.34
Start Time:         Wed, 05 Aug 2020 10:21:14 -0400
Labels:             component=network
                     controller-revision-hash=ovnkube-db-bdb977
                     kubernetes.io/os=linux
                     name=ovnkube-db
                     statefulset.kubernetes.io/pod-name=ovnkube-db-2
                     type=infra
Annotations:        scheduler.alpha.kubernetes.io/critical-pod:
Status:             Running
IP:                 253.255.0.34
Controlled By:      StatefulSet/ovnkube-db
Containers:
   nb-ovsdb:
     Container ID: 
cri-o://2a3634666c21ebc859dfbb5a67a8f76e6792e94d165581fe0638b73aeb880698
     Image:         ovn-daemonset:latest
     Image ID: 
docker.io/library/ovn-daemonset at sha256:624f03ceb671c8e9176a2d3a6988e2351680064fee060bf8e8cf40a4eb0cb362
     Port:          <none>
     Host Port:     <none>
     Command:
       /root/ovnkube.sh
       nb-ovsdb-raft
     State:          Running
       Started:      Wed, 05 Aug 2020 10:21:14 -0400
     Ready:          True
     Restart Count:  0
     Requests:
       cpu:      100m
       memory:   300Mi
     Readiness:  exec [/usr/bin/ovn-kube-util readiness-probe -t 
ovnnb-db-raft] delay=30s timeout=30s period=60s #success=1 #failure=3
     Environment:
       OVN_DAEMONSET_VERSION:       3
       OVN_LOGLEVEL_NB:             -vconsole:info -vfile:info
       K8S_APISERVER:               <set to the key 'k8s_apiserver' of 
config map 'ovn-config'>  Optional: false
       OVN_KUBERNETES_NAMESPACE:    ovn-kubernetes (v1:metadata.namespace)
       POD_NAME:                    ovnkube-db-2 (v1:metadata.name)
       K8S_NODE_IP:                  (v1:status.hostIP)
       OVN_SSL_ENABLE:              no
       OVN_NB_RAFT_ELECTION_TIMER:  1000
       OVN_NB_PORT:                 6641
       OVN_NB_RAFT_PORT:            6643
     Mounts:
       /etc/openvswitch/ from host-var-lib-ovs (rw)
       /etc/ovn/ from host-var-lib-ovs (rw)
       /ovn-cert from host-ovn-cert (ro)
       /var/log/openvswitch/ from host-var-log-ovs (rw)
       /var/log/ovn/ from host-var-log-ovs (rw)
       /var/run/openvswitch/ from host-var-run-ovs (rw)
       /var/run/ovn/ from host-var-run-ovs (rw)
       /var/run/secrets/kubernetes.io/serviceaccount from 
ovn-token-srjkt (ro)
   sb-ovsdb:
     Container ID: 
cri-o://b2532fe5a419b91e6fe80c36b3d79bd410e7a70a08451ada586fd321c7927e73
     Image:         ovn-daemonset:latest
     Image ID: 
docker.io/library/ovn-daemonset at sha256:624f03ceb671c8e9176a2d3a6988e2351680064fee060bf8e8cf40a4eb0cb362
     Port:          <none>
     Host Port:     <none>
     Command:
       /root/ovnkube.sh
       sb-ovsdb-raft
     State:          Running
       Started:      Wed, 05 Aug 2020 10:21:14 -0400
     Ready:          True
     Restart Count:  0
     Requests:
       cpu:      100m
       memory:   300Mi
     Readiness:  exec [/usr/bin/ovn-kube-util readiness-probe -t 
ovnsb-db-raft] delay=30s timeout=30s period=60s #success=1 #failure=3
     Environment:
       OVN_DAEMONSET_VERSION:       3
       OVN_LOGLEVEL_SB:             -vconsole:info -vfile:info
       K8S_APISERVER:               <set to the key 'k8s_apiserver' of 
config map 'ovn-config'>  Optional: false
       OVN_KUBERNETES_NAMESPACE:    ovn-kubernetes (v1:metadata.namespace)
       POD_NAME:                    ovnkube-db-2 (v1:metadata.name)
       K8S_NODE_IP:                  (v1:status.hostIP)
       OVN_SSL_ENABLE:              no
       OVN_SB_RAFT_ELECTION_TIMER:  1000
       OVN_SB_PORT:                 6642
       OVN_SB_RAFT_PORT:            6644
     Mounts:
       /etc/openvswitch/ from host-var-lib-ovs (rw)
       /etc/ovn/ from host-var-lib-ovs (rw)
       /ovn-cert from host-ovn-cert (ro)
       /var/log/openvswitch/ from host-var-log-ovs (rw)
       /var/log/ovn/ from host-var-log-ovs (rw)
       /var/run/openvswitch/ from host-var-run-ovs (rw)
       /var/run/ovn/ from host-var-run-ovs (rw)
       /var/run/secrets/kubernetes.io/serviceaccount from 
ovn-token-srjkt (ro)
Conditions:
   Type              Status
   Initialized       True
   Ready             True
   ContainersReady   True
   PodScheduled      True
Volumes:
   host-var-log-ovs:
     Type:          HostPath (bare host directory volume)
     Path:          /usr/var/log/ovn
     HostPathType:
   host-var-lib-ovs:
     Type:          HostPath (bare host directory volume)
     Path:          /usr/etc/ovn
     HostPathType:
   host-var-run-ovs:
     Type:          HostPath (bare host directory volume)
     Path:          /usr/var/run/ovn/
     HostPathType:
   host-ovn-cert:
     Type:          HostPath (bare host directory volume)
     Path:          /usr/etc/ovn
     HostPathType:  DirectoryOrCreate
   ovn-token-srjkt:
     Type:        Secret (a volume populated by a Secret)
     SecretName:  ovn-token-srjkt
     Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:
Events:
   Type    Reason     Age   From                Message
   ----    ------     ----  ----                -------
   Normal  Pulled     39m   kubelet, ca-rain02  Container image 
"ovn-daemonset:latest" already present on machine
   Normal  Created    39m   kubelet, ca-rain02  Created container nb-ovsdb
   Normal  Started    39m   kubelet, ca-rain02  Started container nb-ovsdb
   Normal  Pulled     39m   kubelet, ca-rain02  Container image 
"ovn-daemonset:latest" already present on machine
   Normal  Created    39m   kubelet, ca-rain02  Created container sb-ovsdb
   Normal  Started    39m   kubelet, ca-rain02  Started container sb-ovsdb
   Normal  Scheduled  37m   default-scheduler   Successfully assigned 
ovn-kubernetes/ovnkube-db-2 to ca-rain02



OVN Cluster
-----------------
kubectl exec -n ovn-kubernetes ovnkube-db-0 -c nb-ovsdb -- ovs-appctl -t 
/var/run/ovn/ovnnb_db.ctl cluster/status OVN_Northbound
b124
Name: OVN_Northbound
Cluster ID: 75d4 (75d41b86-9bf2-4da6-a8a6-8e17f8d83d61)
Server ID: b124 (b1242b82-83e7-4764-a30a-e861b40fe580)
Address: tcp:[253.255.0.33]:6643
Status: cluster member
Role: leader
Term: 1
Leader: self
Vote: self

Election timer: 1000
Log: [2, 40]
Entries not yet committed: 0
Entries not yet applied: 0
Connections: <-8c28 ->8c28 <-5043 ->5043
Servers:
     8c28 (8c28 at tcp:[253.255.0.34]:6643) next_index=40 match_index=39
     5043 (5043 at tcp:[253.255.0.35]:6643) next_index=40 match_index=39
     b124 (b124 at tcp:[253.255.0.33]:6643) (self) next_index=2 
match_index=39
[root at ca-rain01 yaml-pca]# kubectl exec -n ovn-kubernetes ovnkube-db-0 
-c sb-ovsdb -- ovs-appctl -t /var/run/ovn/ovnsb_db.ctl cluster/status 
OVN_Southbound
615d
Name: OVN_Southbound
Cluster ID: b0c0 (b0c077ab-c0aa-4944-9852-b936881f3d64)
Server ID: 615d (615d66f0-b1c9-477b-9c3d-9d88857cba1e)
Address: tcp:[253.255.0.33]:6644
Status: cluster member
Role: leader
Term: 1
Leader: self
Vote: self

Election timer: 1000
Log: [2, 13]
Entries not yet committed: 0
Entries not yet applied: 0
Connections: <-ccf5 ->ccf5 <-257f ->257f
Servers:
     615d (615d at tcp:[253.255.0.33]:6644) (self) next_index=2 
match_index=12
     257f (257f at tcp:[253.255.0.35]:6644) next_index=13 match_index=12
     ccf5 (ccf5 at tcp:[253.255.0.34]:6644) next_index=13 match_index=12



=======================

kubectl exec -n ovn-kubernetes ovnkube-db-1 -c nb-ovsdb -- ovs-appctl -t 
/var/run/ovn/ovnnb_db.ctl cluster/status OVN_Northbound
5043
Name: OVN_Northbound
Cluster ID: 75d4 (75d41b86-9bf2-4da6-a8a6-8e17f8d83d61)
Server ID: 5043 (50437142-0ab8-4eff-8a20-76fc5cff82b8)
Address: tcp:[253.255.0.35]:6643
Status: cluster member
Role: follower
Term: 1
Leader: b124
Vote: unknown

Election timer: 1000
Log: [2, 40]
Entries not yet committed: 0
Entries not yet applied: 0
Connections: ->0000 ->8c28 <-b124 <-8c28
Servers:
     8c28 (8c28 at tcp:[253.255.0.34]:6643)
     5043 (5043 at tcp:[253.255.0.35]:6643) (self)
     b124 (b124 at tcp:[253.255.0.33]:6643)
[root at ca-rain01 yaml-pca]# kubectl exec -n ovn-kubernetes ovnkube-db-1 
-c sb-ovsdb -- ovs-appctl -t /var/run/ovn/ovnsb_db.ctl cluster/status 
OVN_Southbound
257f
Name: OVN_Southbound
Cluster ID: b0c0 (b0c077ab-c0aa-4944-9852-b936881f3d64)
Server ID: 257f (257f45e1-2ffa-403e-a162-4bad3fd58b21)
Address: tcp:[253.255.0.35]:6644
Status: cluster member
Role: follower
Term: 1
Leader: 615d
Vote: unknown
Election timer: 1000
Log: [2, 13]
Entries not yet committed: 0
Entries not yet applied: 0
Connections: ->0000 ->ccf5 <-615d <-ccf5
Servers:
     615d (615d at tcp:[253.255.0.33]:6644)
     257f (257f at tcp:[253.255.0.35]:6644) (self)
     ccf5 (ccf5 at tcp:[253.255.0.34]:6644)



=============================
kubectl exec -n ovn-kubernetes ovnkube-db-2 -c nb-ovsdb -- ovs-appctl -t 
/var/run/ovn/ovnnb_db.ctl cluster/status OVN_Northbound
8c28
Name: OVN_Northbound
Cluster ID: 75d4 (75d41b86-9bf2-4da6-a8a6-8e17f8d83d61)
Server ID: 8c28 (8c28d0c0-2027-4fee-bf14-a7f33dd650b2)
Address: tcp:[253.255.0.34]:6643
Status: cluster member
Role: follower
Term: 1
Leader: b124
Vote: unknown

Election timer: 1000
Log: [2, 40]
Entries not yet committed: 0
Entries not yet applied: 0
Connections: ->0000 <-b124 <-5043 ->5043
Servers:
     8c28 (8c28 at tcp:[253.255.0.34]:6643) (self)
     5043 (5043 at tcp:[253.255.0.35]:6643)
     b124 (b124 at tcp:[253.255.0.33]:6643)
[root at ca-rain01 yaml-pca]# kubectl exec -n ovn-kubernetes ovnkube-db-2 
-c sb-ovsdb -- ovs-appctl -t /var/run/ovn/ovnsb_db.ctl cluster/status 
OVN_Southbound
ccf5
Name: OVN_Southbound
Cluster ID: b0c0 (b0c077ab-c0aa-4944-9852-b936881f3d64)
Server ID: ccf5 (ccf55bc7-5185-4f83-b686-5d697255a44e)
Address: tcp:[253.255.0.34]:6644
Status: cluster member
Role: follower
Term: 1
Leader: 615d
Vote: unknown

Election timer: 1000
Log: [2, 13]
Entries not yet committed: 0
Entries not yet applied: 0
Connections: ->0000 <-615d <-257f ->257f
Servers:
     615d (615d at tcp:[253.255.0.33]:6644)
     257f (257f at tcp:[253.255.0.35]:6644)
     ccf5 (ccf5 at tcp:[253.255.0.34]:6644) (self)




More information about the discuss mailing list