[ovs-dev] [OVN][Scale] Conjunctive matches exponentially increase in Table 45

Girish Moodalbail gmoodalbail at gmail.com
Mon Mar 8 16:13:44 UTC 2021


Essentially the number of OpenFlow rules in Table 45 can be calculated with
the equation below:



OpenFlow rules := (Number of LS with more than one port * Number of
conjunction requirements * number of LSP in port group)



If the cluster size is 1000 and that each of the LS has more than one port
in the same namespace, then the number of OpenFlow rules would be:



1000 * 2 * 2000 = 4M flows


The thinking is if we make the conjunctive match requirements to be 3-way
between -- metadata, reg0, and nw_src -- then the number of OpenFlow rules
would be more linear.


Regards,

~Girish

On Mon, Mar 8, 2021 at 8:00 AM Girish Moodalbail <gmoodalbail at gmail.com>
wrote:

> Hello all,
>
>
>
> If a K8s NetworkPolicy is defined as below...
>
>
>
> kind: NetworkPolicy
>
> metadata:
>
>   name: allow_from_within_namespace
>
>   namespace: ns1
>
> spec:
>
>   podSelector: {}
>
>   ingress:
>
>   - from:
>
>     - podSelector: {}
>
>
>
> ... which basically allows all packets from all Pods to all the Pods
> within the same namespace (ns1) then ovn-k8s translates that to the
> following OVN ACL...
>
>
>
> "ip4.src == {$address_set_name} && outport == @port_group_name"
>
>
>
> ...which basically says all packets with source ip from
> logical_switch_ports in ns1 to logical_switch_ports in ns1 are allowed.
>
>
>
> This particular OVN ACL results in an explosion of OpenFlow rules in Table
> 45 on each of the OVN Chassis in certain cases.
>
>
>
> In the following case, no conjunctive match is used and there is no
> explosion.
>
>
>
> 1. Consider a K8s cluster with 10 nodes
>
> 2. There are 10 LSes (LS1, LS2, ... LS10), one per each node
>
> 3. Each node has one pod (aka LSP -- LSP1 on LS1, LSP2 on LS2, ....) in
> ns1 namespace
>
> 4. Table 45 doesn't contain any conjunction rules and everything is fine.
>
>
>
> Extending the example above in the manner shown below results in addition
> of conjunction flows in Table 45.
>
>
>
> 5. Now schedule a 11th Pod on node1
>
> 6. That is, LSP11 on LS1
>
> 7. This results in addition of OpenFlow rule with conj_id field and two
> requirements for LS1
>
>
>
> So, the conjunction is two-way
> -----------8<--------------8<------
>
> priority=2001,conj_id=4979,ip,reg0=0x80/0x80,metadata=LS1
> actions=load:0x1->NXM_NX_XXREG0[97],resubmit(,46)
>
> priority=2001,ip,reg0=0x80/0x80,metadata=LS1,nw_src=10.244.202.4
> actions=conjunction(4979,1/2)
>
> priority=2001,ip,reg0=0x80/0x80,metadata=LS1,nw_src=10.244.111.4
> actions=conjunction(4979,1/2)
>
> priority=2001,ip,reg0=0x80/0x80,metadata=LS1,nw_src=10.244.180.4
> actions=conjunction(4979,1/2)
>
> <7 more lines snipped>
>
> -----------8<--------------8<------
>
>
> Extending the example above in the manner shown below results in addition
> of more conjunction flows in Table 45.
>
>
>
> 8. Now schedule a 12th pod on node2
>
> 9. That is, LSP21 on LS2
>
> 10. This results in addition of NEW OpenFlow rule with conj_id field and
> two requirements for the LS2
>
>
>
> So, the conjunction is two-way
> -----------8<--------------8<------
>
> priority=2001,conj_id=4980,ip,reg0=0x80/0x80,metadata=LS2
> actions=load:0x1->NXM_NX_XXREG0[97],resubmit(,46)
>
> priority=2001,ip,reg0=0x80/0x80,metadata=LS2,nw_src=10.244.202.4
> actions=conjunction(4979,1/2)
>
> priority=2001,ip,reg0=0x80/0x80,metadata=LS2,nw_src=10.244.111.4
> actions=conjunction(4979,1/2)
>
> priority=2001,ip,reg0=0x80/0x80,metadata=LS2,nw_src=10.244.180.4
> actions=conjunction(4979,1/2)
>
> <7 more lines snipped>
>
> -----------8<--------------8<------
>
>
>
> As you can see, addition of the 2nd port to each of the LS results in an
> exponential number of OpenFlow flows in Table 45.
>
>
>
> Should the conjunction be with 3 requirements -- metadata, nw_src, and
> reg0? Right now, the conjunction is between nw_src and reg0.
>
>
>
> Thanks,
>
> ~Girish
>


More information about the dev mailing list