<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none"><!-- p { margin-top: 0px; margin-bottom: 0px; }--></style>
</head>
<body dir="ltr" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<p></p>
<div>To whom it may concern,</div>
<div><br>
</div>
<div>I am an undergraduate student currently working on a thesis involving an OVS setup. The setup consists of multiple Docker containers each running an OVS instance. The containers are interconnected by Containernet. For brevity, a small one-hop example is
described below:</div>
<div><br>
</div>
<div>In essence, node 1 GRE-encaps packets with ipv6_dst fc00::1 and fc00::2 into packets with ipv6_dst fcaa::1 and fcaa::2 respectively and forwards them to node 2. Pinging fc00::1 or fc00::2 generates traffic for node 2. In node 1 we have:</div>
<div><br>
</div>
<div># needed to get ipv6 working in containers</div>
<div>sysctl net.ipv6.conf.all.disable_ipv6=0</div>
<div><br>
</div>
<div>ovs-ctl start</div>
<div>ovs-vsctl add-br switch</div>
<div>ovs-vsctl add-port switch in -- set interface in ofport=1 type=internal</div>
<div>ovs-vsctl add-port switch allgre -- set interface allgre ofport=2 type=ip6gre options:packet_type=legacy_l2 options:remote_ip=flow</div>
<div><br>
</div>
<div>ip l set in up</div>
<div>IFACE_IN_MAC=$(cat /sys/class/net/in/address)</div>
<div><br>
</div>
<div>ip -6 r add fc00::1 dev in</div>
<div>ip -6 r add fc00::2 dev in</div>
<div>ip n add fc00::1 dev in lladdr $IFACE_IN_MAC</div>
<div>ip n add fc00::2 dev in lladdr $IFACE_IN_MAC</div>
<div><br>
</div>
<div>ip -6 r add fcaa::1 dev n1-eth0</div>
<div>ip -6 r add fcaa::2 dev n1-eth0</div>
<div><br>
</div>
<div># next hop MAC address, i.e. the MAC address of n2-eth0 on the second docker node</div>
<div># n1-eth0 and n2-eth0 are connected to each other via containernet</div>
<div>ip n add fcaa::1 dev n1-eth0 lladdr 00:00:00:00:00:02</div>
<div>ip n add fcaa::2 dev n1-eth0 lladdr 00:00:00:00:00:02</div>
<div><br>
</div>
<div>ovs-ofctl add-flow switch in_port=1,ipv6,ipv6_dst=fc00::1,actions="set_field:fcaa::1->tun_ipv6_dst",output:2</div>
<div>ovs-ofctl add-flow switch in_port=1,ipv6,ipv6_dst=fc00::2,actions="set_field:fcaa::2->tun_ipv6_dst",output:2</div>
<div><br>
</div>
<div>#ip l show n1-eth0</div>
<div># n1-eth0@if13: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000</div>
<div># link/ether 00:00:00:00:00:01 brd ff:ff:ff:ff:ff:ff link-netnsid 1</div>
<div><br>
</div>
<div><br>
</div>
<div>In docker node 2:</div>
<div><br>
</div>
<div>sysctl net.ipv6.conf.all.disable_ipv6=0 </div>
<div><br>
</div>
<div>ovs-vsctl add-br switch</div>
<div>ovs-vsctl add-port switch n2-eth0 -- set interface n2-eth0 ofport=1</div>
<div>ovs-vsctl add-port switch allgre -- set interface allgre ofport=2 type=ip6gre options:packet_type=legacy_l2 options:remote_ip=flow</div>
<div><br>
</div>
<div>ip l set switch up</div>
<div>ip -6 r add local fcaa::/16 dev switch # AnyIP kernel feature</div>
<div><br>
</div>
<div># use case intended: no need to decapsulate, swap the outermost destination ip, forward to the next hop, possibly a third docker node (hence the output to a "potential" port with OF number 3)</div>
<div>ovs-ofctl add-flow switch in_port=1,ipv6,ipv6_dst=fcaa::1,actions="set_field:fcaa::ff->ipv6_dst,output:3"</div>
<div><br>
</div>
<div># use case intended: decapsulate. Send the packet to the Linux networking stack, it will loop back and appear on the "allgre" port where further matching will be made</div>
<div>ovs-ofctl add-flow switch in_port=1,ipv6,ipv6_dst=fcaa::2,actions="output:LOCAL"</div>
<div><br>
</div>
<div># use case intended: do something useful with a decapsulated packet</div>
<div>ovs-ofctl add-flow switch in_port=2,tun_ipv6_dst=fcaa::2,actions="..."</div>
<div><br>
</div>
<div>#ip l show n2-eth0</div>
<div># n2-eth0@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000</div>
<div># link/ether 00:00:00:00:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 1</div>
<div><br>
</div>
<div><br>
</div>
<div>The problem is that the second docker node must also be able to GRE-encap packets received on Containernet ports and set their tun_ipv6_dst to an IP in the fcaa::/16 range. Thus, we have two conflicting use cases. Some packets in the fcaa::/16 prefix need
to be decapsulated and looped back to the switch for further processing/forwarding. Some packets of another prefix, e.g. fc00::/16, must be encapsulated with GRE, their outermost IP destination addresses must be set in the fcaa::/16 range based on flows, then
the (now encapsulated) packets must also loop back to the switch for forwarding. I am currently unable to implement the latter use case. Because the whole fcaa::/16 prefix is bound to the switch's LOCAL port, I can decapsulate any packet from this prefix and
then forward it via OVS, since I just "receive" it on the "allgre" port. However, my understanding is that tunnel ports in OVS utilise kernel routing, so my prefix bind would also mean that all packets which I encapsulate through "allgre" will bounce back
and get decapsulated right away: OVS sees a GRE packet and a matching generic tunneling port for it. I need a way to get the encapsulated packets back in the switch to perform forwarding, however, they should not land on the "allgre" port.</div>
<div><br>
</div>
<div>How could I solve the described problem? Is it perhaps achieved by some kind of datapath configuration (dpctl)? Any help will be greatly appreciated.</div>
<div><br>
</div>
<div>Best regards,</div>
<div>Lyubomir<br>
</div>
<p><br>
</p>
</body>
</html>