[ovs-dev] Issues with the use of the clone action for resubmission to the pipeline

[Mickey Spiegel]

One of the motivations for clone is to use it as a lightweight way to
resubmit to an earlier table at the beginning of the pipeline, without
incurring all of the overhead associated with openflow patch ports.
One such usage is in OVN, where a recent patch set replaced the
use of openflow patch ports with clone, for OVN patch ports within
the same bridge (br-int).

Over the holidays, some issues arose related to this usage of clone
(see the thread ovn ping from VM to external gateway IP failed).

While investigating these issues, I noticed some significant
differences between the way flow and other context information
is saved and restored when using openflow patch ports, versus
the way it is done with clone. At least one such difference, with
regard to conntrack, is causing failures.

I would classify the differences (those for which there is no current
workaround by specifying nested actions within the clone) into two
categories:
1. State that is kept outside of flow, which is currently saved,
   cleared, and restored when using openflow patch ports.
   This includes:
   ctx->wc->masks.tunnel
   ctx->conntracked
   ctx->was_mpls
   ctx->xin->tables_version (not an issue if bridge does not change)
   ctx->stack
   ctx->action_set
   ctx->xbridge (not an issue if bridge does not change)
   ctx->mirrors (not an issue if bridge does not change)
2. State that resides inside the flow, but for which there is no
   workaround to clear or reset the fields. This includes:
   flow->tunnel ?
   flow->tunnel.metadata.tab (not an issue if bridge does not change)
   flow->ct_state (read only)
   flow->ct_mark (restricted so can only be set in ct action)
   flow->ct_label (restricted so can only be set in ct action)

The question is which of these need to be included in the
clone action in order to use clone for resubmission to the
pipeline within the same bridge?

Issues have already been seen with regard to conntrack,
so it needs to be cleared for this usage.
Options include:
1. Always clear conntrack state as part of the clone action.
   Is this OK for all envisioned uses of clone?
2. Add one or more flags indicating whether conntrack
   state should be cleared. This may cause the structure
   to deviate from ext_action_header followed by one or
   more embedded openflow actions.
3. Loosen the restrictions on ct values, so that they can be
   written from actions nested inside a ct action or inside a
   clone action. Note that this would not resolve
   ctx->conntracked, which could become a problem if
   there are recirc reasons other than conntrack.

Do we foresee any issues due to any of the other
differences?
It seems like we should think about ctx->stack (always
dangerous, the question is whether it is more so when
using clone?), ctx->action_set, and flow->tunnel.

While flow->tunnel.metadata.tab was an issue for OVN,
this was because removal of openflow patch ports
revealed an underlying bug. Once this bug is fixed,
flow->tunnel.metadata.tab should not be an issue when
using clone inside a bridge.

Mickey