[ovs-dev] [PATCH ovn v2] Fix the routing for external logical ports of bridged logical switches.
Numan Siddique
numans at ovn.org
Wed Jul 22 19:24:30 UTC 2020
On Mon, Jul 13, 2020 at 11:56 AM Numan Siddique <numans at ovn.org> wrote:
> +Daniel Alvarez Sanchez <dalvarez at redhat.com>
> +Lucas Alvares Gomes Martins <lmartins at redhat.com>
>
>
> On Mon, Jul 13, 2020 at 11:29 AM Ankur Sharma <ankur.sharma at nutanix.com>
> wrote:
>
>> Hi Numan,
>>
>> Thank you so much for the details.
>>
>>
> Hi Ankur,
>
> Thanks for the detailed email. Your analysis is correct. I have few
> comments.
> Please see below.
>
>
>
Hi Ankur,
Did you get any chance to look into my comments ? Just checking.
Thanks
Numan
> Following is my analysis on the feature:
>> a. Port of type EXTERNAL means that we create a logical switch port in
>> OVN without a VIF backing.
>> b. i.e the physical port corresponding to external port is NOT behind OVN
>> managed vswitch (for SRIOV specific case it is not behind any vswitch,
>> since PNIC sends the packet directly yo guest driver).
>> Just for the sake of further discussion we will refer the PHYSICAL
>> PORT/VM corresponding to external port as SRIOV PORT/VM.
>> c. Now from OVN perspective, packets from SRIOV VM will enter the OVN
>> flow pipeline via localnet port (on the Active HA Chassis).
>> d. For DHCP requests, the logical switch pipeline responds.
>> e. Now, we were trying to get the routing working.
>>
>> Based on the understanding mentioned above, i tried following scenario
>> and observed following:
>> a. SRIOV VM could talk to endpoints on other LS attached to LR via Active
>> gateway/HA chassis.
>>
>>
>> https://docs.google.com/document/d/117yskeP1S3qHmkNrBrZ0PxXCvMJCCVJhcPG4phw1Qls/edit?usp=sharing
>> [
>> https://lh5.googleusercontent.com/vhEZ3Ws4BXXkqoxc7naEXKPFC9qMqhJxKhumRzSyXsw9L1jbDc4B4r8cHBZZfOl9J6vYwgtyVA=w1200-h630-p
>> ]<
>> https://docs.google.com/document/d/117yskeP1S3qHmkNrBrZ0PxXCvMJCCVJhcPG4phw1Qls/edit?usp=sharing
>> >
>> OVN EXTERNAL PORT ROUTING<
>> https://docs.google.com/document/d/117yskeP1S3qHmkNrBrZ0PxXCvMJCCVJhcPG4phw1Qls/edit?usp=sharing
>> >
>> OVN EXTERNAL PORT EW ROUTING LOGICAL TOPOLOGY CONFIGURATION LOGICAL
>> ROUTER router 2bd894b1-81a0-4095-9c58-0472aa5de19d (router) port
>> router-to-gvlan1 mac: "00:00:01:01:02:03" networks: ["20.0.0.1/24"] port
>> router-to-underlay mac: "00:00:01:01:02:...
>> docs.google.com
>>
>> Explanation:
>> a. Since packets are coming through localnet port, hence from Router
>> perspective, it is an external endpoint, i.e NS.
>> b. Now for such cases, Router is designed to respond to ARP requests ONLY
>> on gateway chassis and since we have centralized a chassis for NS now,
>> hence from the Gateway chassis there is no MAC replacement.
>> c. Based on a. and b. above, i attached the same gateway chassis to LRP
>> (Logical Router Port) connecting to SRIOV PORT's LS.
>> d. And routing across LR connected Switches worked fine.
>> e. Mac table on TOR and ARP table on SRIOV VM was also fine, i.e ARP
>> cache had <router_port_ip, router_port_mac> and Mac table had an entry for
>> router port mac.
>>
>>
>> Inference:
>> a. For Routing, traffic from localnet ports has to enter via gateway node.
>> b. Hence, the Router port which connects to SR IOV VM Logical Switch has
>> to be on same gateway node as corresponding external port (Which can be
>> achieved easily by attaching same HA chassis group to both).
>>
>
> If you see the BZ here -
> https://bugzilla.redhat.com/show_bug.cgi?id=1829762, Openstack
> networking ovn folks want this to be decoupled. According to them, it will
> be hard to maintain the logic to collocate the
> ha chassis group of external ports with the gateway chassis ports. I have
> CC'd them to get more comments from them.
>
> There is another problem:
> Let's say we have 3 logical switches - ls_vlan4 (10.0.0.0/24) , ls_vlan5 (
> 20.0.0.0/24) and ls_public (172.168.0.0/24) connected to a logical router
> lr0. And all are VLAN bridge networks
> but only ls_public provides N/S traffic. ls_vlan4 and ls_vlan5 are tenant
> bridged networks. I think this is a very common use case.
>
> In this scenario, you will have one l3gateway port - lr0-public
> connecting to logical switch ls_public. So there will be a set of
> gateway_chassis created for this.
>
> To solve this issue, we can also make the router ports - lr0-vlan4 and
> lr0-vlan5 (connecting to logical switches ls_vlan4 and ls_vlan5) as gateway
> ports by (once we enhance OVN
> to support multiple gateway ports), but if you see the logical
> swithes ls_vlan4 and ls_vlan5 in this example don't provide any N/S
> connectivity and hence it may not be accurate
> to consider them as having gateway ports.
>
> And the subnets (here 10.0.0.0/24 and 20.0.0.0/24) of ls_vlan4 and
> ls_vlan5 are internal to OVN and ideally there should be no external entity
> sharing the IP address from these subnets and using the same
> VLAN. Although practically it's possible to do so. If there is such a
> requirement, I think CMS should consider using 'external' port for this. I
> had noticed the issue you mentioned below in (c) long time back,
> I didn't think to bother much because of the reason I mentioned now. (i.e
> these subnets are internal to OVN)
>
> That's why I think it may be OK to take the approach of this patch. i.e
> replace the chassis mac with the actual router mac in the ARP replies for
> the router IPs.
>
> In the case of ls_public (which provides N/S connectivity) I would expect
> an external router to handle routing for this. Please correct me if I'm
> wrong.
>
> Let me know your thoughts.
>
> Thanks
> Numan
>
>
>>
>> Improvements:
>> a. Current state is slightly restrictive, because we support only ONE
>> l3gateway port per router. Which means that SRIOV Logical Switch has to be
>> connected to physical network gateway as well.
>> i.e SRIOV VMs have to on the same logical switch which has the gateway
>> for all the external traffic.
>>
>> b. A more generic and complete implementation would be to enhance OVN to
>> support multiple gateway ports in a distributed router.
>>
>> https://patchwork.ozlabs.org/project/openvswitch/patch/20180312090911.9608-1-ligs@dtdream.com/
>>
>> c. I did observe 1 bug, where we are NOT blocking ARP requests for router
>> port via localnet port in the absence of a gateway port configuration.
>> Absence of guard, leads to multiple ARP responses and duplicate ICMP reply
>> packets.I will submit a fix for this.
>>
>>
>> Please let me know your thoughts on the same and please feel free to call
>> out, if i missed something.
>>
>> Thanks
>>
>> Regards,
>> Ankur
>>
>> ________________________________
>> From: Numan Siddique <numans at ovn.org>
>> Sent: Friday, July 10, 2020 6:18 AM
>> To: Ankur Sharma <ankur.sharma at nutanix.com>
>> Cc: dev at openvswitch.org <dev at openvswitch.org>
>> Subject: Re: [ovs-dev] [PATCH ovn v2] Fix the routing for external
>> logical ports of bridged logical switches.
>>
>>
>>
>> On Fri, Jul 10, 2020 at 4:41 PM Numan Siddique <numans at ovn.org<mailto:
>> numans at ovn.org>> wrote:
>>
>>
>> On Fri, Jul 10, 2020 at 12:45 AM Ankur Sharma <ankur.sharma at nutanix.com
>> <mailto:ankur.sharma at nutanix.com>> wrote:
>> Hi Numan, Daniel,
>>
>> I have not looked at the patch yet. But replacing arp.sha with chassis
>> mac is not the correct approach from networking perspective.
>> Chassic mac is NOT meant to replace the IP-MAC binding of router port, it
>> is ONLY meant to ensure that for EW traffic a distributed router port mac
>> does not show on multiple TOR ports.
>> Both for NS and EW, ARP resolution for router port ip should be responded
>> with router port mac ONLY.
>>
>> I am trying to understand the use case and we can discuss an alternative
>> in this thread.
>> Can you share the repro steps, i can try the same and will try to come up
>> with an alternative.
>>
>>
>> Hi Ankur,
>>
>> In this particular case, the originator of the traffic is from a logical
>> port of type 'external'.
>>
>> One example of using external ports is for SRIOV VMs. The traffic from
>> these VMs are not seen
>> by the local ovn-controller. And we want to provide E-W routing and other
>> OVN services like DHCP, DNS etc
>> to these VMS.
>>
>> So one of the controller nodes (which can receive the traffic sent by
>> these SRIOV VMs) binds these external ports
>> and it responds to the ARP requests and does the routing for it.
>>
>> To reproduce the issue, can you please use own-fake-multi node setup from
>> here ? -
>> https://github.com/numansiddique/ovn-fake-multinode/tree/vlan_chassis_mac_issue
>> [github.com]<
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_numansiddique_ovn-2Dfake-2Dmultinode_tree_vlan-5Fchassis-5Fmac-5Fissue&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=pi-QEhO1EOFRK_o1vb_1vCRSCTS1g5cBvqlc_EKo4oE&s=08JHsDFgxW2FTSXgjxSJtnLy7M1MTPORv9hizGS6GcM&e=
>> >
>>
>> The steps are:
>> 1. Build OVN containers.
>> ./ovn_cluster.sh build
>>
>>
>> Please note, before the 'start', you need to start openvswitch on the
>> host.
>>
>> Thanks
>> Numan
>>
>> 2. ./ovn_cluster.sh start
>>
>> Run
>> 3. sudo ip netns exec sw0-ext1 ping -c3 20.0.0.3
>> PING 20.0.0.3 (20.0.0.3) 56(84) bytes of data.
>> 64 bytes from 20.0.0.3 [20.0.0.3]<
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__20.0.0.3&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=pi-QEhO1EOFRK_o1vb_1vCRSCTS1g5cBvqlc_EKo4oE&s=62q3nvlIlZ0o_lW4MD9nBcQeO3dFMImwlVb2ImFAYwM&e=>:
>> icmp_seq=1 ttl=63 time=0.074 ms
>> 64 bytes from 20.0.0.3 [20.0.0.3]<
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__20.0.0.3&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=pi-QEhO1EOFRK_o1vb_1vCRSCTS1g5cBvqlc_EKo4oE&s=62q3nvlIlZ0o_lW4MD9nBcQeO3dFMImwlVb2ImFAYwM&e=>:
>> icmp_seq=1 ttl=63 time=0.086 ms (DUP!)
>> 64 bytes from 20.0.0.3 [20.0.0.3]<
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__20.0.0.3&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=pi-QEhO1EOFRK_o1vb_1vCRSCTS1g5cBvqlc_EKo4oE&s=62q3nvlIlZ0o_lW4MD9nBcQeO3dFMImwlVb2ImFAYwM&e=>:
>> icmp_seq=1 ttl=63 time=0.089 ms (DUP!)
>> 64 bytes from 20.0.0.3 [20.0.0.3]<
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__20.0.0.3&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=pi-QEhO1EOFRK_o1vb_1vCRSCTS1g5cBvqlc_EKo4oE&s=62q3nvlIlZ0o_lW4MD9nBcQeO3dFMImwlVb2ImFAYwM&e=>:
>> icmp_seq=2 ttl=63 time=0.105 ms
>> 64 bytes from 20.0.0.3 [20.0.0.3]<
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__20.0.0.3&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=pi-QEhO1EOFRK_o1vb_1vCRSCTS1g5cBvqlc_EKo4oE&s=62q3nvlIlZ0o_lW4MD9nBcQeO3dFMImwlVb2ImFAYwM&e=>:
>> icmp_seq=2 ttl=63 time=0.120 ms (DUP!)
>> 64 bytes from 20.0.0.3 [20.0.0.3]<
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__20.0.0.3&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=pi-QEhO1EOFRK_o1vb_1vCRSCTS1g5cBvqlc_EKo4oE&s=62q3nvlIlZ0o_lW4MD9nBcQeO3dFMImwlVb2ImFAYwM&e=>:
>> icmp_seq=2 ttl=63 time=0.124 ms (DUP!)
>> 64 bytes from 20.0.0.3 [20.0.0.3]<
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__20.0.0.3&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=pi-QEhO1EOFRK_o1vb_1vCRSCTS1g5cBvqlc_EKo4oE&s=62q3nvlIlZ0o_lW4MD9nBcQeO3dFMImwlVb2ImFAYwM&e=>:
>> icmp_seq=3 ttl=63 time=0.145 ms
>>
>> --- 20.0.0.3 ping statistics ---
>> 3 packets transmitted, 3 received, +4 duplicates, 0% packet loss, time
>> 2036ms
>> rtt min/avg/max/mdev = 0.074/0.106/0.145/0.023 ms
>>
>> You will see a few DUP packets.
>>
>> $sudo ip netns exec sw0-ext1 ping -c3 10.0.0.1
>> PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data.
>> 64 bytes from 10.0.0.1 [10.0.0.1]<
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__10.0.0.1&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=pi-QEhO1EOFRK_o1vb_1vCRSCTS1g5cBvqlc_EKo4oE&s=wkUosQoosSX262Atr1UvYSkmSeaaizNOw4D4Yh4sNLM&e=>:
>> icmp_seq=1 ttl=254 time=0.298 ms
>> 64 bytes from 10.0.0.1 [10.0.0.1]<
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__10.0.0.1&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=pi-QEhO1EOFRK_o1vb_1vCRSCTS1g5cBvqlc_EKo4oE&s=wkUosQoosSX262Atr1UvYSkmSeaaizNOw4D4Yh4sNLM&e=>:
>> icmp_seq=1 ttl=254 time=0.358 ms (DUP!)
>> 64 bytes from 10.0.0.1 [10.0.0.1]<
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__10.0.0.1&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=pi-QEhO1EOFRK_o1vb_1vCRSCTS1g5cBvqlc_EKo4oE&s=wkUosQoosSX262Atr1UvYSkmSeaaizNOw4D4Yh4sNLM&e=>:
>> icmp_seq=1 ttl=254 time=0.384 ms (DUP!)
>> 64 bytes from 10.0.0.1 [10.0.0.1]<
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__10.0.0.1&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=pi-QEhO1EOFRK_o1vb_1vCRSCTS1g5cBvqlc_EKo4oE&s=wkUosQoosSX262Atr1UvYSkmSeaaizNOw4D4Yh4sNLM&e=>:
>> icmp_seq=2 ttl=254 time=0.598 ms
>> 64 bytes from 10.0.0.1 [10.0.0.1]<
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__10.0.0.1&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=pi-QEhO1EOFRK_o1vb_1vCRSCTS1g5cBvqlc_EKo4oE&s=wkUosQoosSX262Atr1UvYSkmSeaaizNOw4D4Yh4sNLM&e=>:
>> icmp_seq=2 ttl=254 time=0.594 ms (DUP!)
>> 64 bytes from 10.0.0.1 [10.0.0.1]<
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__10.0.0.1&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=pi-QEhO1EOFRK_o1vb_1vCRSCTS1g5cBvqlc_EKo4oE&s=wkUosQoosSX262Atr1UvYSkmSeaaizNOw4D4Yh4sNLM&e=>:
>> icmp_seq=2 ttl=254 time=0.656 ms (DUP!)
>> 64 bytes from 10.0.0.1 [10.0.0.1]<
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__10.0.0.1&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=pi-QEhO1EOFRK_o1vb_1vCRSCTS1g5cBvqlc_EKo4oE&s=wkUosQoosSX262Atr1UvYSkmSeaaizNOw4D4Yh4sNLM&e=>:
>> icmp_seq=3 ttl=254 time=0.715 ms
>>
>> --- 10.0.0.1 ping statistics ---
>> 3 packets transmitted, 3 received, +4 duplicates, 0% packet loss, time
>> 2088ms
>> rtt min/avg/max/mdev = 0.298/0.514/0.715/0.152 ms
>>
>> In the setup, sw0-ext1 represents an external logical switch port. If you
>> see the script here [1],
>> sw0-ext1 is claimed by ovn-chassis-1 node.
>>
>> And when sw0-ext1 sends ARP request to 10.0.0.1, the arp request is
>> handled by ovn-chassis-1
>> and the reply has - arp.sha = router mac and eth.src = chassis mac of
>> ovn-chassis-1.
>>
>> And hence sw0-ext1 sends ping packets with the destination mac of router
>> port IP - 10.0.0.1.
>> And all the 3 nodes reply - ovn-chassis-1, ovn-chassis-2 and ovn-gw-1.
>>
>> I'm not sure if you have played with ovn-fake-multinode before. If you
>> run "docker ps", you will see a docker
>> container representing each chassis.
>>
>> Please do "docker exec -it ovn-central bash" and run a few
>> ovn-nbctl/ovn-sbctl commands to know more.
>>
>> You can also see the script in [1] and reproduce the issue in your setup.
>>
>> I didn't find any other way to solve this issue. Also in normal
>> situations where external ports are not used,
>> any arp request to the router IP from bridge logical switch ports don't
>> leave the chassis since the local
>> ovn-controller itself replies. This is for tenant bridged VLAN logical
>> switches. I guess for provider VLAN networks
>> (which provide the N/S traffic, I guess the arp request for the router
>> port can come from the physical network).
>>
>>
>> [1] -
>> https://github.com/numansiddique/ovn-fake-multinode/blob/vlan_chassis_mac_issue/ovn_cluster.sh#L501
>> [github.com]<
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_numansiddique_ovn-2Dfake-2Dmultinode_blob_vlan-5Fchassis-5Fmac-5Fissue_ovn-5Fcluster.sh-23L501&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=pi-QEhO1EOFRK_o1vb_1vCRSCTS1g5cBvqlc_EKo4oE&s=Z81yBhdW1o0FHPWqPcFDOeU7DOgPOdbCPDt9jNS8hf4&e=
>> >
>>
>>
>> Thanks
>> Numan
>>
>>
>>
>> Regards,
>> Ankur
>> ________________________________
>> From: numans at ovn.org<mailto:numans at ovn.org> <numans at ovn.org<mailto:
>> numans at ovn.org>>
>> Sent: Thursday, July 9, 2020 2:11 AM
>> To: dev at openvswitch.org<mailto:dev at openvswitch.org> <dev at openvswitch.org
>> <mailto:dev at openvswitch.org>>
>> Cc: Numan Siddique <numans at ovn.org<mailto:numans at ovn.org>>; Daniel
>> Alvarez <dalvarez at redhat.com<mailto:dalvarez at redhat.com>>; Ankur Sharma <
>> ankur.sharma at nutanix.com<mailto:ankur.sharma at nutanix.com>>
>> Subject: [PATCH ovn v2] Fix the routing for external logical ports of
>> bridged logical switches.
>>
>> From: Numan Siddique <numans at ovn.org<mailto:numans at ovn.org>>
>>
>> Routing for external logical ports is broken if these ports belonged
>> to bridged logical switches (with localnet port) and
>> 'ovn-chassis-mac-mappings'
>> is configured. External logical ports are those which are external to OVN,
>> but there is a logical port for it and it is claimed by one of the HA
>> chassis.
>> The claimed chassis provides routing and other native OVN serices like
>> dhcp and dns.
>>
>> When the external port sends ARP request for the router IP, the claimed
>> chassis
>> replies for the ARP request, but the arp.sha is set to the actual router
>> mac instead
>> of the chassis mac. This causes the traffic from external port
>> VM/container to be handled
>> incorrectly. A ping to the router ip, is replied by all the chassis which
>> can see this
>> packet instead of just the claimed HA chassis.
>>
>> To fix this, this patch does 2 things.
>>
>> 1. In the table - OFTABLE_LOG_TO_PHY (65), it adds a 160 priority flow to
>> modify the ARP packets arp.sha to store the chassis mac.
>>
>> 2. And when the packet destined to the chassis mac is received, it
>> replaces the
>> chassis mac with the actual router mac in table 0.
>>
>> Reported-at:
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__bugzilla.redhat.com_show-5Fbug.cgi-3Fid-3D1829762&d=DwIDAg&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=u_maNAEOYzfy4_tzUirBX0TdPn35ePuIddtQDl4B8fs&s=T6SxlTDjkPxA6_Lsv_KjWkOSUSfesz0LIVnovPxBXlc&e=
>> Reported-by
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__bugzilla.redhat.com_show-5Fbug.cgi-3Fid-3D1829762&d=DwIDAg&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=u_maNAEOYzfy4_tzUirBX0TdPn35ePuIddtQDl4B8fs&s=T6SxlTDjkPxA6_Lsv_KjWkOSUSfesz0LIVnovPxBXlc&e=Reported-by>
>> <
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__bugzilla.redhat.com_show-5Fbug.cgi-3Fid-3D1829762&d=DwIDAg&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=u_maNAEOYzfy4_tzUirBX0TdPn35ePuIddtQDl4B8fs&s=T6SxlTDjkPxA6_Lsv_KjWkOSUSfesz0LIVnovPxBXlc&e=Reported-by>:
>> Daniel Alvarez <dalvarez at redhat.com<mailto:dalvarez at redhat.com>>
>> CC: Ankur Sharma <ankur.sharma at nutanix.com<mailto:
>> ankur.sharma at nutanix.com>>
>> Signed-off-by: Numan Siddique <numans at ovn.org<mailto:numans at ovn.org>>
>> ---
>>
>> v1 -> v2
>> ----
>> * Rebased.
>>
>> controller/chassis.c | 48 ++++++++------
>> controller/chassis.h | 2 +
>> controller/physical.c | 145 +++++++++++++++++++++++++++++++++++++++---
>> tests/ovn.at [ovn.at]<
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__ovn.at&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=pi-QEhO1EOFRK_o1vb_1vCRSCTS1g5cBvqlc_EKo4oE&s=Q2fRVpcNqWezUjPuzhRmq0flEygq4AKjSbnWP1tVpJk&e=>
>> | 131 ++++++++++++++++++++++++++++++++++++++
>> 4 files changed, 299 insertions(+), 27 deletions(-)
>>
>> diff --git a/controller/chassis.c b/controller/chassis.c
>> index eec270ea39..25146d75f2 100644
>> --- a/controller/chassis.c
>> +++ b/controller/chassis.c
>> @@ -645,10 +645,11 @@ chassis_run(struct ovsdb_idl_txn *ovnsb_idl_txn,
>> }
>>
>> bool
>> -chassis_get_mac(const struct sbrec_chassis *chassis_rec,
>> - const char *bridge_mapping,
>> - struct eth_addr *chassis_mac)
>> +chassis_get_mac_mappings(const struct sbrec_chassis *chassis_rec,
>> + struct smap *chassis_mappings)
>> {
>> + smap_init(chassis_mappings);
>> +
>> const char *tokens
>> = get_chassis_mac_mappings(&chassis_rec->other_config);
>> if (!tokens[0]) {
>> @@ -656,7 +657,6 @@ chassis_get_mac(const struct sbrec_chassis
>> *chassis_rec,
>> }
>>
>> char *save_ptr = NULL;
>> - bool ret = false;
>> char *tokstr = xstrdup(tokens);
>>
>> /* Format for a chassis mac configuration is:
>> @@ -669,24 +669,36 @@ chassis_get_mac(const struct sbrec_chassis
>> *chassis_rec,
>> char *chassis_mac_bridge = strtok_r(token, ":", &save_ptr2);
>> char *chassis_mac_str = strtok_r(NULL, "", &save_ptr2);
>>
>> - if (!strcmp(chassis_mac_bridge, bridge_mapping)) {
>> - struct eth_addr temp_mac;
>> + smap_replace(chassis_mappings, chassis_mac_bridge,
>> chassis_mac_str);
>> + }
>>
>> - /* Return the first chassis mac. */
>> - char *err_str = str_to_mac(chassis_mac_str, &temp_mac);
>> - if (err_str) {
>> - free(err_str);
>> - continue;
>> - }
>> + free(tokstr);
>> + return true;
>> +}
>>
>> - ret = true;
>> - *chassis_mac = temp_mac;
>> - break;
>> - }
>> +bool
>> +chassis_get_mac(const struct sbrec_chassis *chassis_rec,
>> + const char *bridge_mapping,
>> + struct eth_addr *chassis_mac)
>> +{
>> + struct smap chassis_mappings;
>> +
>> + if (!chassis_get_mac_mappings(chassis_rec, &chassis_mappings)) {
>> + return false;
>> }
>>
>> - free(tokstr);
>> - return ret;
>> + const char *chassis_mac_str = smap_get_def(&chassis_mappings,
>> + bridge_mapping, "");
>> + struct eth_addr temp_mac;
>> +
>> + char *err_str = str_to_mac(chassis_mac_str, &temp_mac);
>> + if (err_str) {
>> + free(err_str);
>> + return false;
>> + }
>> +
>> + *chassis_mac = temp_mac;
>> + return true;
>> }
>>
>> /* Returns true if the database is all cleaned up, false if more work is
>> diff --git a/controller/chassis.h b/controller/chassis.h
>> index 178d2957e8..dae761312d 100644
>> --- a/controller/chassis.h
>> +++ b/controller/chassis.h
>> @@ -42,6 +42,8 @@ bool chassis_cleanup(struct ovsdb_idl_txn
>> *ovnsb_idl_txn,
>> bool chassis_get_mac(const struct sbrec_chassis *chassis,
>> const char *bridge_mapping,
>> struct eth_addr *chassis_mac);
>> +bool chassis_get_mac_mappings(const struct sbrec_chassis *,
>> + struct smap *chassis_mappings);
>> const char *chassis_get_id(void);
>> const char * get_chassis_mac_mappings(const struct smap *ext_ids);
>>
>> diff --git a/controller/physical.c b/controller/physical.c
>> index 6d7d8e93bc..b43a157b94 100644
>> --- a/controller/physical.c
>> +++ b/controller/physical.c
>> @@ -62,7 +62,8 @@ load_logical_ingress_metadata(const struct
>> sbrec_port_binding *binding,
>> /* UUID to identify OF flows not associated with ovsdb rows. */
>> static struct uuid *hc_uuid = NULL;
>>
>> -#define CHASSIS_MAC_TO_ROUTER_MAC_CONJID 100
>> +#define CHASSIS_MAC_TO_ROUTER_SRC_MAC_CONJID 100
>> +#define CHASSIS_MAC_TO_ROUTER_DST_MAC_CONJID 101
>>
>> void
>> physical_register_ovs_idl(struct ovsdb_idl *ovs_idl)
>> @@ -148,6 +149,18 @@ put_move(enum mf_field_id src, int src_ofs,
>> move->dst.n_bits = n_bits;
>> }
>>
>> +static void
>> +put_value(const uint8_t *data, size_t len,
>> + enum mf_field_id dst, int ofs, int n_bits,
>> + struct ofpbuf *ofpacts)
>> +{
>> + struct ofpact_set_field *sf = ofpact_put_set_field(ofpacts,
>> + mf_from_id(dst),
>> NULL,
>> + NULL);
>> + bitwise_copy(data, len, 0, sf->value, sf->field->n_bytes, ofs,
>> n_bits);
>> + bitwise_one(ofpact_set_field_mask(sf), sf->field->n_bytes, ofs,
>> n_bits);
>> +}
>> +
>> static void
>> put_resubmit(uint8_t table_id, struct ofpbuf *ofpacts)
>> {
>> @@ -494,11 +507,10 @@ put_chassis_mac_conj_id_flow(const struct
>> sbrec_chassis_table *chassis_table,
>> ofpbuf_clear(ofpacts_p);
>> match_init_catchall(&match);
>>
>> -
>> match_set_dl_src(&match, chassis_mac);
>>
>> conj = ofpact_put_CONJUNCTION(ofpacts_p);
>> - conj->id = CHASSIS_MAC_TO_ROUTER_MAC_CONJID;
>> + conj->id = CHASSIS_MAC_TO_ROUTER_SRC_MAC_CONJID;
>> conj->n_clauses = 2;
>> conj->clause = 0;
>> ofctrl_add_flow(flow_table, OFTABLE_PHY_TO_LOG, 180,
>> @@ -507,6 +519,51 @@ put_chassis_mac_conj_id_flow(const struct
>> sbrec_chassis_table *chassis_table,
>> }
>>
>> free_remote_chassis_macs();
>> +
>> + /* We need to replace the packet destined to the chassis mac
>> (eth.dst)
>> + * with the router mac. This is required to support external ports.
>> + * These ports don't see the router mac at all since we send the
>> + * chassis MAC in the ARP reply for any ARP requests to the router
>> IPs.
>> + * Without these flows, the packets will not enter the router
>> pipeline
>> + * if they need to be routed.
>> + * Please see put_replace_chassis_mac_flows() for the 2nd clause of
>> + * conj id - CHASSIS_MAC_TO_ROUTER_DST_MAC_CONJID.
>> + * */
>> + struct smap chassis_mac_mappings =
>> SMAP_INITIALIZER(&chassis_mac_mappings);
>> + if (chassis_get_mac_mappings(chassis, &chassis_mac_mappings)) {
>> + struct smap_node *node;
>> + struct sset macs = SSET_INITIALIZER(&macs);
>> + SMAP_FOR_EACH (node, &chassis_mac_mappings) {
>> + struct eth_addr chassis_mac;
>> +
>> + char *err_str = str_to_mac(node->value, &chassis_mac);
>> + if (err_str) {
>> + free(err_str);
>> + continue;
>> + }
>> +
>> + if (!sset_add(&macs, node->value)) {
>> + /* The OF flow for the mac is already added. */
>> + continue;
>> + }
>> +
>> + ofpbuf_clear(ofpacts_p);
>> + match_init_catchall(&match);
>> +
>> + match_set_dl_dst(&match, chassis_mac);
>> +
>> + struct ofpact_conjunction *conj;
>> + conj = ofpact_put_CONJUNCTION(ofpacts_p);
>> + conj->id = CHASSIS_MAC_TO_ROUTER_DST_MAC_CONJID;
>> + conj->n_clauses = 2;
>> + conj->clause = 0;
>> + ofctrl_add_flow(flow_table, OFTABLE_PHY_TO_LOG, 180,
>> + 0, &match, ofpacts_p, hc_uuid);
>> + }
>> + sset_destroy(&macs);
>> + }
>> +
>> + smap_destroy(&chassis_mac_mappings);
>> }
>>
>> static void
>> @@ -555,7 +612,7 @@ put_replace_chassis_mac_flows(const struct simap
>> *ct_zones,
>>
>> /* Match on ingress port, vlan_id and conjunction id */
>> match_set_in_port(&match, ofport);
>> - match_set_conj_id(&match, CHASSIS_MAC_TO_ROUTER_MAC_CONJID);
>> + match_set_conj_id(&match, CHASSIS_MAC_TO_ROUTER_SRC_MAC_CONJID);
>>
>> if (tag) {
>> match_set_dl_vlan(&match, htons(tag), 0);
>> @@ -572,6 +629,37 @@ put_replace_chassis_mac_flows(const struct simap
>> *ct_zones,
>> replace_mac = ofpact_put_SET_ETH_SRC(ofpacts_p);
>> replace_mac->mac = router_port_mac;
>>
>> + /* Resubmit to first logical ingress pipeline table. */
>> + put_resubmit(OFTABLE_LOG_INGRESS_PIPELINE, ofpacts_p);
>> + ofctrl_add_flow(flow_table, OFTABLE_PHY_TO_LOG, 180,
>> + rport_binding->header_.uuid.parts[0],
>> + &match, ofpacts_p, hc_uuid);
>> +
>> + ofpbuf_clear(ofpacts_p);
>> + match_init_catchall(&match);
>> +
>> + /* Add flow, which will match on conjunction id and will
>> + * replace destination mac with router port mac */
>> +
>> + /* Match on ingress port, vlan_id and conjunction id */
>> + match_set_in_port(&match, ofport);
>> + match_set_conj_id(&match, CHASSIS_MAC_TO_ROUTER_DST_MAC_CONJID);
>> +
>> + if (tag) {
>> + match_set_dl_vlan(&match, htons(tag), 0);
>> + } else {
>> + match_set_dl_tci_masked(&match, 0, htons(VLAN_CFI));
>> + }
>> +
>> + /* Actions */
>> +
>> + if (tag) {
>> + ofpact_put_STRIP_VLAN(ofpacts_p);
>> + }
>> + load_logical_ingress_metadata(localnet_port, &zone_ids,
>> ofpacts_p);
>> + replace_mac = ofpact_put_SET_ETH_DST(ofpacts_p);
>> + replace_mac->mac = router_port_mac;
>> +
>> /* Resubmit to first logical ingress pipeline table. */
>> put_resubmit(OFTABLE_LOG_INGRESS_PIPELINE, ofpacts_p);
>> ofctrl_add_flow(flow_table, OFTABLE_PHY_TO_LOG, 180,
>> @@ -579,7 +667,7 @@ put_replace_chassis_mac_flows(const struct simap
>> *ct_zones,
>> &match, ofpacts_p, hc_uuid);
>>
>> /* Provide second search criteria, i.e localnet port's
>> - * vlan ID for conjunction flow */
>> + * vlan ID for conjunction flows. */
>> struct ofpact_conjunction *conj;
>> ofpbuf_clear(ofpacts_p);
>> match_init_catchall(&match);
>> @@ -591,12 +679,19 @@ put_replace_chassis_mac_flows(const struct simap
>> *ct_zones,
>> }
>>
>> conj = ofpact_put_CONJUNCTION(ofpacts_p);
>> - conj->id = CHASSIS_MAC_TO_ROUTER_MAC_CONJID;
>> + conj->id = CHASSIS_MAC_TO_ROUTER_SRC_MAC_CONJID;
>> + conj->n_clauses = 2;
>> + conj->clause = 1;
>> +
>> + conj = ofpact_put_CONJUNCTION(ofpacts_p);
>> + conj->id = CHASSIS_MAC_TO_ROUTER_DST_MAC_CONJID;
>> conj->n_clauses = 2;
>> conj->clause = 1;
>> +
>> ofctrl_add_flow(flow_table, OFTABLE_PHY_TO_LOG, 180,
>> rport_binding->header_.uuid.parts[0],
>> &match, ofpacts_p, hc_uuid);
>> +
>> }
>> }
>>
>> @@ -665,9 +760,6 @@ put_replace_router_port_mac_flows(struct
>> ovsdb_idl_index
>> * a. Flow replaces ingress router port mac with a chassis mac.
>> * b. Flow appends the vlan id localnet port is configured with.
>> */
>> - match_init_catchall(&match);
>> - ofpbuf_clear(ofpacts_p);
>> -
>> ovs_assert(rport_binding->n_mac == 1);
>> char *err_str = str_to_mac(rport_binding->mac[0],
>> &router_port_mac);
>> if (err_str) {
>> @@ -679,6 +771,9 @@ put_replace_router_port_mac_flows(struct
>> ovsdb_idl_index
>> }
>>
>> /* Replace Router mac flow */
>> + match_init_catchall(&match);
>> + ofpbuf_clear(ofpacts_p);
>> +
>> match_set_metadata(&match, htonll(dp_key));
>> match_set_reg(&match, MFF_LOG_OUTPORT - MFF_REG0, port_key);
>> match_set_dl_src(&match, router_port_mac);
>> @@ -698,6 +793,38 @@ put_replace_router_port_mac_flows(struct
>> ovsdb_idl_index
>> ofctrl_add_flow(flow_table, OFTABLE_LOG_TO_PHY, 150,
>> localnet_port->header_.uuid.parts[0],
>> &match, ofpacts_p, &localnet_port->header_.uuid);
>> +
>> + /* Replace Router mac in the ARP packets (arp.sha) to the
>> chassis MAC.
>> + * This is very important and required for external logical
>> ports and
>> + * when these ports send ARP for their router IPs, the chassis
>> mac
>> + * should be sent which has claimed these external ports. */
>> + match_init_catchall(&match);
>> + ofpbuf_clear(ofpacts_p);
>> +
>> + match_set_metadata(&match, htonll(dp_key));
>> + match_set_reg(&match, MFF_LOG_OUTPORT - MFF_REG0, port_key);
>> + match_set_dl_src(&match, router_port_mac);
>> + match_set_dl_type(&match, htons(ETH_TYPE_ARP));
>> + match_set_arp_sha(&match, router_port_mac);
>> +
>> + replace_mac = ofpact_put_SET_ETH_SRC(ofpacts_p);
>> + replace_mac->mac = chassis_mac;
>> +
>> + if (tag) {
>> + struct ofpact_vlan_vid *vlan_vid;
>> + vlan_vid = ofpact_put_SET_VLAN_VID(ofpacts_p);
>> + vlan_vid->vlan_vid = tag;
>> + vlan_vid->push_vlan_if_needed = true;
>> + }
>> +
>> + put_value(chassis_mac.ea, sizeof chassis_mac.ea, MFF_ARP_SHA,
>> + 0, 48, ofpacts_p);
>> +
>> + ofpact_put_OUTPUT(ofpacts_p)->port = ofport;
>> +
>> + ofctrl_add_flow(flow_table, OFTABLE_LOG_TO_PHY, 160,
>> + localnet_port->header_.uuid.parts[0],
>> + &match, ofpacts_p, &localnet_port->header_.uuid);
>> }
>> }
>>
>> diff --git a/tests/ovn.at [ovn.at]<
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__ovn.at&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=pi-QEhO1EOFRK_o1vb_1vCRSCTS1g5cBvqlc_EKo4oE&s=Q2fRVpcNqWezUjPuzhRmq0flEygq4AKjSbnWP1tVpJk&e=>
>> b/tests/ovn.at [ovn.at]<
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__ovn.at&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=pi-QEhO1EOFRK_o1vb_1vCRSCTS1g5cBvqlc_EKo4oE&s=Q2fRVpcNqWezUjPuzhRmq0flEygq4AKjSbnWP1tVpJk&e=
>> >
>> index 24d93bc245..f033401410 100644
>> --- a/tests/ovn.at [ovn.at]<
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__ovn.at&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=pi-QEhO1EOFRK_o1vb_1vCRSCTS1g5cBvqlc_EKo4oE&s=Q2fRVpcNqWezUjPuzhRmq0flEygq4AKjSbnWP1tVpJk&e=
>> >
>> +++ b/tests/ovn.at [ovn.at]<
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__ovn.at&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=pi-QEhO1EOFRK_o1vb_1vCRSCTS1g5cBvqlc_EKo4oE&s=Q2fRVpcNqWezUjPuzhRmq0flEygq4AKjSbnWP1tVpJk&e=
>> >
>> @@ -14748,6 +14748,137 @@ AT_CHECK([cat ext1_v6.packets | cut -c -120],
>> [0], [expout])
>> cat ext1_v6.expected | cut -c 125- > expout
>> AT_CHECK([cat ext1_v6.packets | cut -c 125-], [0], [expout])
>>
>> +# Configure ovn-chassis-mac-mappings on all the hypervisors.
>> +as hv1
>> +ovs-vsctl set open .
>> external_ids:ovn-chassis-mac-mappings=phys:1e:02:ad:aa:bb:01
>> +
>> +as hv2
>> +ovs-vsctl set open .
>> external_ids:ovn-chassis-mac-mappings=phys:1e:02:ad:aa:bb:02
>> +
>> +as hv3
>> +ovs-vsctl set open .
>> external_ids:ovn-chassis-mac-mappings=phys:1e:02:ad:aa:bb:03
>> +
>> +OVS_WAIT_UNTIL([test 6 = $(as hv1 ovs-ofctl dump-flows br-int table=0 |
>> grep conj -c)])
>> +OVS_WAIT_UNTIL([test 6 = $(as hv2 ovs-ofctl dump-flows br-int table=0 |
>> grep conj -c)])
>> +OVS_WAIT_UNTIL([test 6 = $(as hv3 ovs-ofctl dump-flows br-int table=0 |
>> grep conj -c)])
>> +
>> +OVS_WAIT_UNTIL([test 1 = $(as hv1 ovs-ofctl dump-flows br-int table=0 | \
>> +grep conj | grep "dl_dst=1e:02:ad:aa:bb:01" -c)])
>> +
>> +OVS_WAIT_UNTIL([test 1 = $(as hv2 ovs-ofctl dump-flows br-int table=0 | \
>> +grep conj | grep "dl_dst=1e:02:ad:aa:bb:02" -c)])
>> +
>> +OVS_WAIT_UNTIL([test 1 = $(as hv3 ovs-ofctl dump-flows br-int table=0 | \
>> +grep conj | grep "dl_dst=1e:02:ad:aa:bb:03" -c)])
>> +
>> +OVS_WAIT_UNTIL([test 1 = $(as hv1 ovs-ofctl dump-flows br-int
>> table=65,arp | \
>> +grep "load:0x1e02adaabb01->NXM_NX_ARP_SHA" -c)])
>> +
>> +OVS_WAIT_UNTIL([test 0 = $(as hv2 ovs-ofctl dump-flows br-int
>> table=65,arp | \
>> +grep "load:0x1e02adaabb01->NXM_NX_ARP_SHA" -c)])
>> +
>> +OVS_WAIT_UNTIL([test 1 = $(as hv2 ovs-ofctl dump-flows br-int
>> table=65,arp | \
>> +grep "load:0x1e02adaabb02->NXM_NX_ARP_SHA" -c)])
>> +
>> +OVS_WAIT_UNTIL([test 1 = $(as hv3 ovs-ofctl dump-flows br-int
>> table=65,arp | \
>> +grep "load:0x1e02adaabb03->NXM_NX_ARP_SHA" -c)])
>> +
>> +as hv1
>> +reset_pcap_file hv1-ext1 hv1/ext1
>> +
>> +send_arp_request() {
>> + local inport=$1 eth_src=$2 eth_dst=$3 spa=$4 tpa=$5
>> + local reply_src_mac=$6 reply_dst_mac=$7
>> + local reply_sha=$8 reply_tha=$9
>> +
>> + local eth_type=0806
>> + local eth=${eth_dst}${eth_src}${eth_type}
>> +
>> + local arp=0001080006040001${eth_src}${spa}${eth_dst}${tpa}
>> +
>> + local request=${eth}${arp}
>> + as hv1 ovs-appctl netdev-dummy/receive hv${inport}-ext${inport}
>> $request
>> +
>> + local reply=${reply_dst_mac}${reply_src_mac}${eth_type}
>> + reply=${reply}0001080006040002${reply_sha}${tpa}${reply_tha}${spa}
>> + echo $reply > hv1-ext${inport}.expected
>> +}
>> +
>> +src_mac=f00000000003
>> +dst_mac=ffffffffffff
>> +reply_src_mac=1e02adaabb03
>> +repl_dst_mac=f00000000003
>> +# Send ARP request to router ip - 10.0.0.1
>> +send_arp_request 1 ${src_mac} ${dst_mac} $(ip_to_hex 10 0 0 6)
>> $(ip_to_hex 10 0 0 1) \
>> +${reply_src_mac} ${repl_dst_mac} ${reply_src_mac} ${repl_dst_mac}
>> +
>> +OVS_WAIT_UNTIL([test 1 = $(as hv3 ovs-ofctl dump-flows br-int
>> table=65,arp | \
>> +grep "load:0x1e02adaabb03->NXM_NX_ARP_SHA" | grep "n_packets=1" -c)])
>> +
>> +OVN_CHECK_PACKETS([hv1/ext1-tx.pcap], [hv1-ext1.expected])
>> +
>> +as hv1
>> +reset_pcap_file hv1-ext1 hv1/ext1
>> +
>> +# Send unicast ARP request destined to the chassis mac of hv3.
>> +src_mac=f00000000003
>> +dst_mac=1e02adaabb03
>> +reply_src_mac=1e02adaabb03
>> +repl_dst_mac=f00000000003
>> +send_arp_request 1 ${src_mac} ${dst_mac} $(ip_to_hex 10 0 0 6)
>> $(ip_to_hex 10 0 0 1) \
>> +${reply_src_mac} ${repl_dst_mac} ${reply_src_mac} ${repl_dst_mac}
>> +
>> +OVS_WAIT_UNTIL([test 1 = $(as hv3 ovs-ofctl dump-flows br-int
>> table=65,arp | \
>> +grep "load:0x1e02adaabb03->NXM_NX_ARP_SHA" | grep "n_packets=2" -c)])
>> +
>> +OVN_CHECK_PACKETS([hv1/ext1-tx.pcap], [hv1-ext1.expected])
>> +
>> +# Make hv2 active.
>> +ovn-nbctl ha-chassis-group-add-chassis hagrp1 hv2 60
>> +
>> +OVS_WAIT_UNTIL(
>> + [chassis=`ovn-sbctl --bare --columns chassis find port_binding \
>> +logical_port=ls1-lp_ext1`
>> + test "$chassis" = "$hv2_uuid"])
>> +
>> +reset_pcap_file hv1-ext1 hv1/ext1
>> +
>> +src_mac=f00000000003
>> +dst_mac=ffffffffffff
>> +reply_src_mac=1e02adaabb02
>> +repl_dst_mac=f00000000003
>> +# Send ARP request to router ip - 10.0.0.1. Should be replied by hv2.
>> +send_arp_request 1 ${src_mac} ${dst_mac} $(ip_to_hex 10 0 0 6)
>> $(ip_to_hex 10 0 0 1) \
>> +${reply_src_mac} ${repl_dst_mac} ${reply_src_mac} ${repl_dst_mac}
>> +
>> +OVS_WAIT_UNTIL([test 1 = $(as hv2 ovs-ofctl dump-flows br-int
>> table=65,arp | \
>> +grep "load:0x1e02adaabb02->NXM_NX_ARP_SHA" | grep "n_packets=1" -c)])
>> +
>> +OVN_CHECK_PACKETS([hv1/ext1-tx.pcap], [hv1-ext1.expected])
>> +
>> +as hv1
>> +reset_pcap_file hv1-ext1 hv1/ext1
>> +
>> +# Send unicast ARP request destined to the chassis mac of hv2.
>> +src_mac=f00000000003
>> +dst_mac=1e02adaabb02
>> +reply_src_mac=1e02adaabb02
>> +repl_dst_mac=f00000000003
>> +send_arp_request 1 ${src_mac} ${dst_mac} $(ip_to_hex 10 0 0 6)
>> $(ip_to_hex 10 0 0 1) \
>> +${reply_src_mac} ${repl_dst_mac} ${reply_src_mac} ${repl_dst_mac}
>> +
>> +OVS_WAIT_UNTIL([test 1 = $(as hv2 ovs-ofctl dump-flows br-int
>> table=65,arp | \
>> +grep "load:0x1e02adaabb02->NXM_NX_ARP_SHA" | grep "n_packets=2" -c)])
>> +
>> +OVN_CHECK_PACKETS([hv1/ext1-tx.pcap], [hv1-ext1.expected])
>> +
>> +ovn-nbctl ha-chassis-group-add-chassis hagrp1 hv3 70
>> +ovn-nbctl ha-chassis-group-add-chassis hagrp1 hv2 10
>> +OVS_WAIT_UNTIL(
>> + [chassis=`ovn-sbctl --bare --columns chassis find port_binding \
>> +logical_port=ls1-lp_ext1`
>> + test "$chassis" = "$hv3_uuid"])
>> +
>> # disconnect hv3 from the network, hv1 should take over
>> as hv3
>> port=${sandbox}_br-phys
>> --
>> 2.26.2
>>
>> _______________________________________________
>> dev mailing list
>> dev at openvswitch.org<mailto:dev at openvswitch.org>
>> https://mail.openvswitch.org/mailman/listinfo/ovs-dev [
>> mail.openvswitch.org]<
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddev&d=DwMFaQ&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=pi-QEhO1EOFRK_o1vb_1vCRSCTS1g5cBvqlc_EKo4oE&s=zN_7muosIOH3hM4whjjfwe4Jr-AWMZoDTgL9UqpVEIQ&e=
>> >
>>
>> _______________________________________________
>> dev mailing list
>> dev at openvswitch.org
>> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>>
>>
More information about the dev
mailing list