<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=gb2312"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:宋体;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@宋体";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
text-align:justify;
text-justify:inter-ideograph;
font-size:10.5pt;
font-family:"Calibri",sans-serif;}
h2
{mso-style-priority:9;
mso-style-link:"标题 2 Char";
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:18.0pt;
font-family:宋体;
font-weight:bold;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:宋体;}
code
{mso-style-priority:99;
font-family:宋体;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin:0cm;
margin-bottom:.0001pt;
text-align:justify;
text-justify:inter-ideograph;
text-indent:21.0pt;
font-size:10.5pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.2Char
{mso-style-name:"标题 2 Char";
mso-style-priority:9;
mso-style-link:"标题 2";
font-family:宋体;
font-weight:bold;}
p.simpara, li.simpara, div.simpara
{mso-style-name:simpara;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:宋体;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
/* Page Definitions */
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 90.0pt 72.0pt 90.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:504785666;
mso-list-template-ids:-18217436;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:36.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:72.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:"Courier New";
mso-bidi-font-family:"Times New Roman";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:108.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:144.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:180.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:216.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:252.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:288.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:324.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l1
{mso-list-id:993295500;
mso-list-type:hybrid;
mso-list-template-ids:-1053678668 -27629124 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l1:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:18.0pt;
text-indent:-18.0pt;}
@list l1:level2
{mso-level-number-format:alpha-lower;
mso-level-text:"%2\)";
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:42.0pt;
text-indent:-21.0pt;}
@list l1:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
margin-left:63.0pt;
text-indent:-21.0pt;}
@list l1:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:84.0pt;
text-indent:-21.0pt;}
@list l1:level5
{mso-level-number-format:alpha-lower;
mso-level-text:"%5\)";
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:105.0pt;
text-indent:-21.0pt;}
@list l1:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
margin-left:126.0pt;
text-indent:-21.0pt;}
@list l1:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:147.0pt;
text-indent:-21.0pt;}
@list l1:level8
{mso-level-number-format:alpha-lower;
mso-level-text:"%8\)";
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:168.0pt;
text-indent:-21.0pt;}
@list l1:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
margin-left:189.0pt;
text-indent:-21.0pt;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=ZH-CN link="#0563C1" vlink="#954F72" style='text-justify-trim:punctuation'><div class=WordSection1><p class=MsoNormal><span lang=EN-US>Hi, all<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>We’re trying OVS DPDK in openstack cloud, but a big warn makes us hesitate. Floating IP and qrouter use tap interfaces which are attached into br-int, SNAT also should use similar way, so OVS DPDK will impact on VM network performance significantly, I believe many cloud providers have deployed OVS DPDK, my questions are:<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoListParagraph style='margin-left:18.0pt;text-indent:-18.0pt;mso-list:l1 level1 lfo2'><![if !supportLists]><span lang=EN-US><span style='mso-list:Ignore'>1.<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span lang=EN-US>Do we have some known ways to improve this?<o:p></o:p></span></p><p class=MsoListParagraph style='margin-left:18.0pt;text-indent:-18.0pt;mso-list:l1 level1 lfo2'><![if !supportLists]><span lang=EN-US><span style='mso-list:Ignore'>2.<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span lang=EN-US>Is there any existing effort for this? Veth in kubernetes should have the same performance issue in OVS DPDK case.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>I also found a very weird issue. I added two veth pairs into ovs bridge and ovs DPDK bridge, for ovs case, iperf3 can work well, but it can’t for OVS DPDK case, what’s wrong.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>$ sudo ./my-ovs-vsctl show<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>2a67c1d9-51dc-4728-bb3e-405f2f49e2b1<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> Bridge br-int<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> Port "veth3-br"<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> Interface "veth3-br"<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> Port "dpdk0"<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> Interface "dpdk0"<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> type: dpdk<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> options: {dpdk-devargs="0000:00:08.0"}<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> Port br-int<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> Interface br-int<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> type: internal<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> Port "veth2-br"<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> Interface "veth2-br"<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> Port "dpdk1"<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> Interface "dpdk1"<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> type: dpdk<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> options: {dpdk-devargs="0000:00:09.0"}<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> Port "veth4-br"<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> Interface "veth4-br"<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> Port "veth1-br"<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> Interface "veth1-br"<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>$ sudo ip netns exec ns1 ifconfig veth1<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>veth1 Link encap:Ethernet HWaddr 26:32:e8:f3:1e:2a<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> inet addr:20.1.1.1 Bcast:20.1.1.255 Mask:255.255.255.0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> inet6 addr: fe80::2432:e8ff:fef3:1e2a/64 Scope:Link<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> RX packets:809 errors:0 dropped:0 overruns:0 frame:0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> TX packets:20 errors:0 dropped:0 overruns:0 carrier:0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> collisions:0 txqueuelen:1000<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> RX bytes:66050 (66.0 KB) TX bytes:1580 (1.5 KB)<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>$ sudo ip netns exec ns2 ifconfig veth2<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>veth2 Link encap:Ethernet HWaddr 82:71:3b:41:d1:ec<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> inet addr:20.1.1.2 Bcast:20.1.1.255 Mask:255.255.255.0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> inet6 addr: fe80::8071:3bff:fe41:d1ec/64 Scope:Link<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> RX packets:862 errors:0 dropped:0 overruns:0 frame:0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> TX packets:26 errors:0 dropped:0 overruns:0 carrier:0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> collisions:0 txqueuelen:1000<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> RX bytes:70436 (70.4 KB) TX bytes:2024 (2.0 KB)<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>$ sudo ip netns exec ns2 ping 20.1.1.1<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>PING 20.1.1.1 (20.1.1.1) 56(84) bytes of data.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>64 bytes from 20.1.1.1: icmp_seq=1 ttl=64 time=0.353 ms<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>64 bytes from 20.1.1.1: icmp_seq=2 ttl=64 time=0.322 ms<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>64 bytes from 20.1.1.1: icmp_seq=3 ttl=64 time=0.333 ms<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>64 bytes from 20.1.1.1: icmp_seq=4 ttl=64 time=0.329 ms<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>64 bytes from 20.1.1.1: icmp_seq=5 ttl=64 time=0.340 ms<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>^C<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>--- 20.1.1.1 ping statistics ---<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>5 packets transmitted, 5 received, 0% packet loss, time 4099ms<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>rtt min/avg/max/mdev = 0.322/0.335/0.353/0.019 ms<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>$ sudo ip netns exec ns1 iperf3 -s -i 10 &<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>[2] 2851<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>[1] Exit 1 sudo ip netns exec ns1 iperf3 -s -i 10<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>$ -----------------------------------------------------------<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>Server listening on 5201<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>-----------------------------------------------------------<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>$ sudo ip netns exec ns2 iperf3 -t 60 -i 10 -c 20.1.1.1<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>iperf3: error - unable to connect to server: Connection timed out<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>$<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>iperf3 has always been hanging there, then exit because of timeout, what's wrong here?<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>$ sudo ./my-ovs-ofctl -Oopenflow13 dump-flows br-int<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US> cookie=0x0, duration=1076.396s, table=0, n_packets=1522, n_bytes=124264, priority=0 actions=NORMAL<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>$<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>The below is Redhat OSP document for your reference.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US><a href="https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/12/html/network_functions_virtualization_planning_and_configuration_guide/part-dpdk-configure">https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/12/html/network_functions_virtualization_planning_and_configuration_guide/part-dpdk-configure</a><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><h2><span lang=EN-US>8.8. Known limitations<o:p></o:p></span></h2><p class=MsoNormal align=left style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-align:left'><span lang=EN-US style='font-size:12.0pt;font-family:宋体'>There are certain limitations when configuring OVS-DPDK with Red Hat OpenStack Platform for the NFV use case: <o:p></o:p></span></p><ul type=disc><li class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-align:left;mso-list:l0 level1 lfo1'><span lang=EN-US style='font-size:12.0pt;font-family:宋体'>Use Linux bonds for control plane networks. Ensure both PCI devices used in the bond are on the same NUMA node for optimum performance. Neutron Linux bridge configuration is not supported by Red Hat. <o:p></o:p></span></li><li class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-align:left;mso-list:l0 level1 lfo1'><span lang=EN-US style='font-size:12.0pt;font-family:宋体'>Huge pages are required for every instance running on the hosts with OVS-DPDK. If huge pages are not present in the guest, the interface appears but does not function. <o:p></o:p></span></li><li class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-align:left;mso-list:l0 level1 lfo1'><span lang=EN-US style='font-size:12.0pt;font-family:宋体'>There is a performance degradation of services that use tap devices, because these devices do not support DPDK. For example, services such as DVR, FWaaS, and LBaaS use tap devices. <o:p></o:p></span></li><ul type=circle><li class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-align:left;mso-list:l0 level2 lfo1'><span lang=EN-US style='font-size:12.0pt;font-family:宋体'>With OVS-DPDK, you can enable DVR with netdev datapath, but this has poor performance and is not suitable for a production environment. DVR uses kernel namespace and tap devices to perform the routing. <o:p></o:p></span></li><li class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-align:left;mso-list:l0 level2 lfo1'><span lang=EN-US style='font-size:12.0pt;font-family:宋体'>To ensure the DVR routing performs well with OVS-DPDK, you need to use a controller such as ODL which implements routing as OpenFlow rules. With OVS-DPDK, OpenFlow routing removes the bottleneck introduced by the Linux kernel interfaces so that the full performance of datapath is maintained. <o:p></o:p></span></li></ul></ul><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p></div></body></html>