[ovs-discuss] OVN - Opaque metadata on logical switch ports to simplify network policies

Dumitru Ceara dceara at redhat.com
Fri Apr 24 10:45:04 UTC 2020


On 4/17/20 7:52 AM, Han Zhou wrote:
> 
> 
> On Thu, Apr 16, 2020 at 9:54 PM Han Zhou <hzhou at ovn.org
> <mailto:hzhou at ovn.org>> wrote:
>>
>>
>>
>> On Thu, Apr 16, 2020 at 2:41 AM Dumitru Ceara <dceara at redhat.com
> <mailto:dceara at redhat.com>> wrote:
>> >
>> > On 4/11/20 1:03 AM, Han Zhou wrote:
>> > >
>> > >
>> > > On Thu, Apr 9, 2020 at 4:54 PM Mark Michelson <mmichels at redhat.com
> <mailto:mmichels at redhat.com>
>> > > <mailto:mmichels at redhat.com <mailto:mmichels at redhat.com>>> wrote:
>> > >>
>> > >> On 4/2/20 7:30 AM, Dumitru Ceara wrote:
>> > >> > Hi all,
>> > >> >
>> > >> > I'd like to bring up to discussion the following feature
> request that
>> > >> > would make the life of some CMSs easier when implementing network
>> > >> > policies with OVN ACLs.
>> > >> >
>> > >> > If OVN would allow a CMS to attach an opaque metadata value to
> logical
>> > >> > switch ports (e.g., a 32-bit integer) and also support defining
> ACLs
>> > >> > that match on a specific value of that opaque metadata then the
>> > >> > following type of policies would result in less openflow
> entries and
>> > >> > would also be easier to configure.
>> > >> >
>> > >> > Let's assume the logical topology:
>> > >> >
>> > >> > Logical_Switch LS1 --- Logical_Router LR --- Logical_Switch LS2
>> > >> >     |         |                                  |         |
>> > >> >     |         |                                  |         |
>> > >> >    VM1.1     VM1.2                              VM2.1     VM2.2
>> > >> >
>> > >> > And a default, drop-all-traffic, policy:
>> > >> >
>> > >> > PortGroup:
>> > >> > - name: "pg_drop_all"
>> > >> > - ports: VM1.1, VM1.2, VM2.1, VM2.2
>> > >> >
>> > >> > ACL:
>> > >> > - priority: 1000
>> > >> > - direction: to-lport
>> > >> > - match: "outport == @pg_drop_all && ipv4"
>> > >> > - action: "drop"
>> > >> > - applied on PortGroup "pg_drop_all".
>> > >> >
>> > >> > With the current OVN implementation, if we want to allow
> traffic between
>> > >> > VM1.1 and VM2.1, for example, for TCP port 80 we have to add
>> > > something like:
>> > >> >
>> > >> > PortGroup:
>> > >> > - name: "pg_allow"
>> > >> > - ports: VM1.1, VM2.1
>> > >> >
>> > >> > AddressSet:
>> > >> > - name: "as_allow"
>> > >> > - IPs: IP-VM1.1, IP-VM2.1
>> > >> >
>> > >> > ACL:
>> > >> > - priority: 2000
>> > >> > - direction: to-lport
>> > >> > - match: "ipv4.src == @as_allow && outport == @pg_allow && tcp.dest
>> > > == 80"
>> > >> > - action: "allow"/"allow-related"
>> > >> > - applied on PortGroup "pg_allow"
>> > >> >
>> > >> > Assuming that we could attach some opaque metadata to traffic
> received
>> > >> > on specific logical switch ports, e.g. value 42:
>> > >> >
>> > >> > ovn-nbctl set-lsp-metadata VM1.1 42
>> > >> > ovn-nbctl set-lsp-metadata VM2.1 42
>> > >> >
>> > >> > The configuration required to enforce the same type of allow
> network
>> > >> > policy would change to:
>> > >> >
>> > >> > ACL:
>> > >> > - priority: 2000
>> > >> > - direction: to-lport
>> > >> > - match "lsp-metadata == 42 && outport == @pg_allow && tcp.dest
> == 80"
>> > >> > - action: "allow"/"allow-related"
>> > >> > - applied on PortGroup "pg_allow"
>> > >> >
>> > >> > This would generate 50% less openflow entries because we don't
> need to
>> > >> > validate that traffic comes from IPs in AddressSet "as_allow"
> as that's
>> > >> > implicitly done by matching on the "lsp-metadata" value.
>> > >> >
>> > >> >  From an implementation perspective as far as I see we'd need to:
>> > >> > - in openflow table 0: store the lsp-metadata value in an
> unused 32 bit
>> > >> > register (e.g., REG7). This is either passed through tunnel header
>> > >> > fields or read from the port_binding associated with the
> ingress logical
>> > >> > switch port.
>> > >> > - in openflow table 34: when moving the packet to the egress
> pipeline,
>> > >> > skip clearing the register used to store lsp-metadata (e.g., REG7)
>> > >> > - in openflow table 32: when packets need to be tunneled to a
> remote
>> > >> > hypervisor, put_encapsulation() should also encode the
> lsp-metadata.
>> > >> >
>> > >> > Do you think there would be any major problems with such an
> approach?
>> > >> >
>> > >> > Thanks,
>> > >> > Dumitru
>> > >>
>> >
>> > Hi, Mark, Han,
>> >
>> > Thanks for your feedback!
>> >
>> > >> Hi Dumitru,
>> > >>
>> > >> The proposal is essentially a way of expressing a port group as a
>> > >> user-defined "hash". Instead of having to check for set
> membership of a
>> > >> port in this port group, the user can express an integer that
> represents
>> > >> the group instead. A single integer check is more efficient than
> having
>> > >> to generate multiple flows to check for set membership.
>> > >>
>> >
>> > Yes.
>> >
>> > >> The first thing that came to mind here is that we probably need to
>> > >> consider both the source lsp metadata and the destination lsp
> metadata.
>> > >> In the proposal, the source lsp metadata is all that is
> considered, but
>> > >> the destination lsp metadata could be just as important. This
> would mean
>> > >> needing two registers to store the values instead of one, and
>> > >> potentially encapulsating two values instead of one.
>> > >>
>> >
>> > I'm not completely sure what you mean here. Do you mean to store another
>> > piece of metadata based on the egress port determined by the logical
>> > switch ingress pipeline? Do you have an example in mind where this would
>> > be useful?
>> >
>> > >> There are other thoughts I have about this, but I think it
> muddies the
>> > >> scope of what you've proposed. For instance, could there be a way
> to set
>> > >> multiple metadata values on a switch port and choose which to use
> for a
>> > >> particular packet based on certain criteria? Could this metadata
> concept
>> > >> be used internally as a method of implementing port group matches?
>> > >>
>> >
>> > I think this would be harder to implement because we'd need something
>> > like an ACL to match on the criteria and determine if a given metadata
>> > would be set for the packet.
>> >
>> > >> Mark Michelson
>> > >>
>> > >> _______________________________________________
>> > >> discuss mailing list
>> > >> discuss at openvswitch.org <mailto:discuss at openvswitch.org>
> <mailto:discuss at openvswitch.org <mailto:discuss at openvswitch.org>>
>> > >> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>> > >
>> > > Hi Dumitru,
>> > >
>> > > Thanks for the proposal. I had a similar idea and presented here:
>> > >
> https://www.slideshare.net/hanzhou1978/large-scale-overlay-networks-with-ovn-problems-and-solutions/31
>> > >
>> >
>> > Thanks Han for pointing this out!
>> >
>> > > Instead of providing a separate interface to set opaque metadata, my
>> > > idea was focusing on using metadata as a way to implement port-group.
>> > > This fits in what Mark has just mentioned above.
>> > >
>> > > In fact I am more inclined to just port-group implementation
> instead of
>> > > exposing this in NB, because I think the metadata is more of an
>> > > implementation bit, which is better to be in SB only, and hidden from
>> > > end user.
>> > >
>> >
>> > So if I understand correctly, assuming that a port is part of single
>> > port_group, a user would write ACLs of the form:
>> >
>> > - direction: to-lport
>> > - match: "outport == @pg_allow && src_group_id == @pg_allow_id &&
>> > tcp.dest == 80"
>> > - action: "allow"/"allow-related"
>>
>> Here is the idea. Usually, an ACL that allows group A to access group
> B, on destination side (ingress rule), may look like:
>>
>> direction: to-lport
>> match: "outport == @pg_B && src_group_id == get_group_id(pg_A) &&
> tcp.dest == 80"
>> action: "allow"/"allow-related"
>>
>> "src_group_id" here is a new field that can be defined in lflow, which
> translates to a logical field, which can come from tunnel metadata or
> directly populated if it is from local.
>> "get_group_id(pg_A)" is a function that can be translated to an ID
> that maps to pg_A in SB DB. Each port group that is allowed to use this
> mechanism (if a port belongs to multiple groups, only limited number of
> groups, or maybe only one, can use this mechanism) will be assigned an
> ID by northd and write to SB DB. Port groups that are allowed to use
> this mechanism can have a column in NB like "use_group_id_metadata" = True.

Ack.

>>
>> >
>> > How would we determine the "src_group_id"?
>> > Would we add something in openflow table 0 that based on the port_id and
>> > membership to port_groups would set "src_group_id" in a register?
>> > And if needed we'd also add this to the geneve tunnel header?
>> >
>>
>> On the sending side, src_group_id is always populated in a register
> (in tunnel header as well when sending to remote), if the src port
> belongs to a group that has "use_group_id_metadata" = True.
> 
> Sorry, I forgot to add that, to populated the src_group_id, it can be
> done in the ls_in_port_sec_l2 stage (if that is what you meant by table
> 0), and try to combine with the port-security flows, to avoid adding
> extra flows (for scalability consideration).
> 

Sounds good.

>>
>> > > I also mentioned some challenges for using metadata to implement
> port-group.
>> > > 1. A port can belong to multiple groups. Since we can only encode
>> > > limited number of group IDs in metadata of tunnel header, we need to
>> > > handle it properly. One idea is to specify in NB port-group
> whether this
>> > > group is supposed to be encoded in metadata.
>> > >
>> >
>> > I guess that if a port is part of multiple port groups we'd have to use
>> > multiple registers (one per port group?) to store all potential
>> > src_group_ids, right?
>>
>> It depends on the size of a group we want to support and the number of
> bit we have in register(s) and metadata header.
>>
>> >
>> > It might be better then to allow a port to be part of at most one such
>> > port group. What do you think?
>>
>> Yes, that might be reasonable.
>>

OK.

I'll start working on an PoC to support for this and share it on the
mailing list as RFC once I have it ready. It might be easier then to
discuss all the implications and details.

Thanks,
Dumitru

>> >
>> > > 2. The approach works for ingress (to-lport) ACL rules only, to
> specify
>> > > source group id (id in metadata), and the receiving side host is
>> > > supposed to check the metadata to verify if the packet is from a
> source
>> > > port of Group X. For egress rules, i.e. ACL that checks if a packet is
>> > > sent to certain destination, the only information in the packet is IP,
>> > > so expanding to IP addresses based flows seems to be the only way for
>> > > egress rules. @Mark mentioned above about destination metadata, but I
>> > > didn't understand how could we get the destination metadata when
> packet
>> > > is being sent out from the source - if we have a mapping between the
>> > > destination IP and metadata it is equally big amount of flows needed
>> > > just like how port group is implemented today.
>> > >
>> >
>> > I agree with this but maybe Mark has a concrete example in mind.
>> >
>> > > Have you thought about these and what's your idea?
>> > >
>> > > Thanks,
>> > > Han
>> > >
>> > >
>> >
>> > Thanks,
>> > Dumitru
>> >



More information about the discuss mailing list