[ovs-dev] SST Tunnel missing from Open vSwitch v2.11.0?

Gregory Rose gvrose8192 at gmail.com
Thu Apr 11 21:06:25 UTC 2019


On 4/11/2019 10:48 AM, Nikolas Britton wrote:
> I'm not sure what you mean by pulled from the OOT drivers. I was able 
> to build the kernel modules with a few hacks to ./debian/rules and 
> everything necessary was already there. I was able to build the 
> openvswitch.ko, vport_geneve.ko, vport_vxlan.ko, vport_gre.ko, 
> vport_lisp.ko, and vport_stt.ko with dpdk appending the following in 
> ./debian/rules for configure:
>
> --with-dpdk=/usr/src/dpdk-stable-18.11.1/x86_64-native-linuxapp-gcc
> --with-linux=/usr/src/linux-headers-4.18.0-17-generic

Top posting... so hard to follow.

In any case I misunderstood - I was referring to the Linux upstream 
kernel where the stt and lisp drivers are
not included.  If Ubuntu adds them to their distro package then that 
changes the calculus.

So now I'm confused though - the stt driver is in the native Linux 
kernel datapath but you're using dpdk to
bypass the kernel datapath?

Maybe someone who knows DPDK better can help.

- Greg

>
> For some reason the dpdk 18.11 upstream debian packages doesn't build 
> / install the libraries necessary for the openvswitch configure script 
> to use on ubuntu 18.04, so I had to first manually build the dpdk 
> libraries by running make install T=x86_64-native-linuxapp-gcc in 
> /usr/src/dpdk-stable-18.11.1/. More information about that process is 
> documented here: http://docs.openvswitch.org/en/latest/intro/install/dpdk/
>
> I compiled it for skylake-avx512, here are my flags:
> export DEB_CFLAGS_APPEND="-m64 -Ofast -march=skylake-avx512 
> -mtune=skylake-avx512 -funroll-loops"
> export DEB_CXXFLAGS_APPEND="-m64 -Ofast -march=skylake-avx512 
> -mtune=skylake-avx512 -funroll-loops"
> export DEB_LDFLAGS_APPEND="-m64 -Ofast -march=skylake-avx512 
> -mtune=skylake-avx512 -funroll-loops"
> export DEB_OBJCFLAGS_APPEND="-m64 -Ofast -march=skylake-avx512 
> -mtune=skylake-avx512 -funroll-loops"
> export DEB_OBJCXXFLAGS_APPEND="-m64 -Ofast -march=skylake-avx512 
> -mtune=skylake-avx512 -funroll-loops"
> export DEB_CPPFLAGS_APPEND="-m64 -Ofast -march=skylake-avx512 
> -mtune=skylake-avx512 -funroll-loops"
>
> I uploaded all the packages to here: http://www.exabit.io/ubuntu/
>
> I did an initial test with the vport-stt.ko, and openvswitch.ko, 
> modules enabled and performance was amazing to say the least, iperf 
> test appeared to fully saturate my 16 Gbps connection between two nodes.
>
> root at bm001:~# ovs-vsctl show
> 224dea41-f166-4f33-aa31-372fa5617624
>     Bridge "overlay0"
>         Port "stt0"
> Interface "stt0"
> type: stt
> options: {remote_cert="/etc/openvswitch/bm002-cert.pem", 
> remote_ip="10.3.40.52"}
>         Port "overlay0"
> Interface "overlay0"
> type: internal
>     ovs_version: "2.11.1"
>
> root at bm001:~# ifconfig overlay0
> overlay0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>         inet 192.168.144.1  netmask 255.255.255.0  broadcast 
> 192.168.144.255
>         inet6 fe80::181b:5dff:fed5:bc4f  prefixlen 64  scopeid 0x20<link>
>         ether 1a:1b:5d:d5:bc:4f  txqueuelen 1000  (Ethernet)
>         RX packets 755447  bytes 36222552 (36.2 MB)
>         RX errors 0  dropped 0  overruns 0  frame 0
>         TX packets 714694  bytes 44007592220 (44.0 GB)
>         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
>
> root at bm001:~# ifconfig ens3
> ens3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9000
>         inet 10.3.40.51  netmask 255.255.255.0  broadcast 10.3.40.255
>         inet6 fe80::17ff:fe00:57ce  prefixlen 64  scopeid 0x20<link>
>         ether 02:00:17:00:57:ce  txqueuelen 1000  (Ethernet)
>         RX packets 13310513  bytes 1332012738 (1.3 GB)
>         RX errors 0  dropped 1570  overruns 0  frame 0
>         TX packets 20576288  bytes 45912880820 (45.9 GB)
>         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
>
> root at bm001:~# iperf -c 192.168.144.2
> ------------------------------------------------------------
> Client connecting to 192.168.144.2, TCP port 5001
> TCP window size: 85.0 KByte (default)
> ------------------------------------------------------------
> [  3] local 192.168.144.1 port 32834 connected with 192.168.144.2 port 
> 5001
> [ ID] Interval       Transfer   Bandwidth
> [  3]  0.0-10.0 sec  17.7 GBytes  15.2 Gbits/sec
>
> root at bm002:~# ovs-vsctl show
> 75ef1aaf-a31a-4e87-b3d3-94241d9b7125
>     Bridge "overlay0"
>         Port "overlay0"
> Interface "overlay0"
> type: internal
>         Port "stt0"
> Interface "stt0"
> type: stt
> options: {local_ip="10.3.40.52", 
> remote_cert="/etc/openvswitch/bm001-cert.pem", remote_ip="10.3.40.51"}
> ovs_version: "2.11.1"
>
> overlay0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>         inet 192.168.144.2  netmask 255.255.255.0  broadcast 
> 192.168.144.255
>         inet6 fe80::1097:22ff:fe53:1e49  prefixlen 64  scopeid 0x20<link>
>         ether 12:97:22:53:1e:49  txqueuelen 1000  (Ethernet)
>         RX packets 944565  bytes 44007205884 (44.0 GB)
>         RX errors 0  dropped 0  overruns 0  frame 0
>         TX packets 517923  bytes 34184906 (34.1 MB)
>         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
>
> ens3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9000
> inet 10.3.40.52  netmask 255.255.255.0  broadcast 10.3.40.255
> inet6 fe80::17ff:fe02:12b8  prefixlen 64  scopeid 0x20<link>
> ether 02:00:17:02:12:b8  txqueuelen 1000  (Ethernet)
>         RX packets 20679038  bytes 45910655685 (45.9 GB)
>         RX errors 0  dropped 0  overruns 0  frame 0
>         TX packets 13338314  bytes 1344124706 (1.3 GB)
>         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
>
> root at bm002:~# iperf -s -B 192.168.144.2
> ------------------------------------------------------------
> Server listening on TCP port 5001
> Binding to local address 192.168.144.2
> TCP window size: 85.3 KByte (default)
> ------------------------------------------------------------
> [  4] local 192.168.144.2 port 5001 connected with 192.168.144.1 port 
> 32834
> [ ID] Interval Transfer     Bandwidth
> [  4]  0.0-10.0 sec  17.7 GBytes  15.2 Gbits/sec
>
> On Wed, Apr 10, 2019 at 5:53 PM Gregory Rose <gvrose8192 at gmail.com 
> <mailto:gvrose8192 at gmail.com>> wrote:
>
>
>
>     On 4/10/2019 1:32 PM, Nikolas Britton wrote:
>     > Hi,
>     >
>     > I just finished compiling OVS 2.11.0 packages for Ubuntu 18.04
>     using the
>     > Debian upstream source package. I was experimenting with tunnels
>     to create
>     > an overlay network to connect multiple KVM hypervisors. One of
>     the options
>     > I tried to use was an STT tunnel. However, it appears that the
>     STT protocol
>     > didn't get included in the binary somehow. Does it need to be
>     enabled with
>     > a build flag? The STT option also doesn't work with IPsec, but
>     the manual
>     > says it should.
>
>     The STT driver isn't included in the upstream drivers.  You'd need to
>     pull from the OOT drivers at
>     github.
>
>     - Greg
>
>
>
> -- 
> *Nikolas Britton*
> *Senior Deployment Engineer, Mirantis*
> 12301-B Riata Trace Pkwy Bldg #2, Suite 150, Austin, Texas



More information about the dev mailing list