[ovs-discuss] Temporary packet loss on existing connections during acl updates

Blank-Burian, Dr. Markus blankburian at uni-muenster.de
Tue Dec 10 17:49:20 UTC 2019


Hello,

we are using OVS/OVN v2.10.1 as networking backend for OpenStack. We found a problem, that leeds to temporarily stuck
connections during OpenStack Security Group updates. It only affects connections which traverse Security Groups / ACLs
including port ranges. The  connections show complete packet loss within OVS for >90 seconds. After this period, packets
will again be transmitted. The translation of ACLs with port ranges to OpenFlow rules consists of multiple flow entries,
including conjunctions and bitwise tests. During the update, we expect some packets to be dropped, but tcp
retransmissions will resend the missing ones.
However, due to "ovn: Apply ACL changes to existing connections" (
https://github.com/ovn-org/ovn/commit/9918edfeb164c1948520b6dab3b14e91356b2961), the whole connections are marked as
"ct_label.blocked == 1". This results in all follow up packages to be blocked by the corresponding ls_in_acl rule. We
did not examine the mechanism, by which the connection is unmarked, so packets can again be transmitted.
For our tests, we replaced the "ct_commit(ct_label=1/1);" action with a simple "drop;" (see patch below). This leads to
the expected behavior, that packets of established connections not matching any ACLs will be dropped. Also, after the
ACL change is complete, tcp retransmissions quickly catch up and the service interruption is minimal. As I am no expert
on OVN, I am not sure of any side effects, this change might cause. However, this seems to be the only place, where the
ct_label.blocked is set by ovn-northd.

Best regards,
Markus


--- a/ovn/northd/ovn-northd.c
+++ b/ovn/northd/ovn-northd.c
@@ -3547,7 +3547,7 @@ consider_acl(struct hmap *lflows, struct ovn_datapath *od,
             ds_clear(&match);
             ds_clear(&actions);
             ds_put_cstr(&match, "ct.est && ct_label.blocked == 0");
-            ds_put_cstr(&actions, "ct_commit(ct_label=1/1); ");
+            ds_put_cstr(&actions, "drop; ");
             if (!strcmp(acl->action, "reject")) {
                 build_reject_acl_rules(od, lflows, stage, acl, &match,
                                        &actions);


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 7521 bytes
Desc: not available
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20191210/618cce4f/attachment.bin>


More information about the discuss mailing list