[ovs-discuss] network bandwidth is too small in Openstack when using OVS+VLAN

Margin Hu jhu_com at 163.com
Tue Mar 27 12:31:44 UTC 2018


Hi Guys,

I met a strange issue about network bandwidth in Openstack when using 
OVS+VLAN.

version info:

openvswitch-2.6.1-4.1.git20161206.el7.x86_64

all server's os is centos7 , kernel is 3.10.0-514.36.5.el7.x86_64

In my test setup,  there are 3 servers.

1. physical server A  ;
2. compute node B  which run kvm as hypervisor, and  has a 10Gbits/s nic
3. instance on B is C

B-A  iperf output shows  7.8Gbits/s , C-A  iperf output shows only 
1.3Gbits/s  ,  the Instance has been enabled vhost-net .

I don't know where root cause is.   I notice cpu utilization is very 
low, only used 1%  on B and C  when iperf testing.

in the instance C ,

[root at perf2 ~]# ethtool -k eth0
Features for eth0:
rx-checksumming: on [fixed]
tx-checksumming: on
         tx-checksum-ipv4: off [fixed]
         tx-checksum-ip-generic: on
         tx-checksum-ipv6: off [fixed]
         tx-checksum-fcoe-crc: off [fixed]
         tx-checksum-sctp: off [fixed]
scatter-gather: on
         tx-scatter-gather: on
         tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
         tx-tcp-segmentation: on
         tx-tcp-ecn-segmentation: on
         tx-tcp6-segmentation: on
udp-fragmentation-offload: on
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: off [fixed]
tx-vlan-offload: off [fixed]
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: on [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-mpls-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
busy-poll: off [fixed]
tx-sctp-segmentation: off [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]

in the compute node B :

[root at cloud-sz-compute-b10-04 ~]# ethtool -k ens1f1
Features for ens1f1:
rx-checksumming: on
tx-checksumming: on
         tx-checksum-ipv4: on
         tx-checksum-ip-generic: off [fixed]
         tx-checksum-ipv6: on
         tx-checksum-fcoe-crc: off [fixed]
         tx-checksum-sctp: off [fixed]
scatter-gather: on
         tx-scatter-gather: on
         tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
         tx-tcp-segmentation: on
         tx-tcp-ecn-segmentation: off [fixed]
         tx-tcp6-segmentation: on
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [requested on]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: on
tx-mpls-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
busy-poll: off [fixed]
tx-sctp-segmentation: off [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]

[root at cloud-sz-compute-b10-04 boot]# ps uax |grep instance-00000036
root      14298  0.0  0.0 112644   968 pts/22   S+   18:32   0:00 grep 
--color=auto instance-00000036
42427     14747  6.2  2.6 316007880 13724096 ?  Sl   10:05  31:41 
/usr/libexec/qemu-kvm -name guest=instance-00000036,deb
ug-threads=on -S -object 
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-3-instance-00000036/master-key.aes 
-machine pc-i440fx-rhel7.3.0,accel=kvm,usb=off -cpu 
Broadwell,+vme,+ds,+acpi,+ss,+ht,+tm,+pbe,+dtes64,+ds_cpl,+vmx,+smx,+est,+tm2,+xtpr,+pdcm,+dca,+osxsave,+f16c,+rdrand,+arat,+tsc_adjust,+xsaveopt,+pdpe1gb,+abm,+rtm,+hle 
-m 307200 -realtime mlock=off -smp 64,sockets=64,cores=1,threads=1 -uuid 
d9cf97c9-a9c1-4d13-81b0-344eb061ffbf -smbios 
type=1,manufacturer=RDO,product=OpenStack 
Compute,version=15.0.4-1.el7,serial=c5e75cea-0425-9467-e711-ca25e82136af,uuid=d9cf97c9-a9c1-4d13-81b0-344eb061ffbf,family=Virtual 
Machine -no-user-config -nodefaults -chardev 
socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-3-instance-00000036/monitor.sock,server,nowait 
-mon chardev=charmonitor,id=monitor,mode=control -rtc 
base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet 
-no-shutdown -boot strict=on -device 
piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive 
file=/dev/disk/by-id/dm-uuid-mpath-3600a09803830434972244a78424b3479,format=raw,if=none,id=drive-virtio-disk0,serial=3a9cf4fd-47a8-4b0b-97fc-1b91b81f8770,cache=none,aio=native 
-device 
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 
-netdev tap,fd=29,id=hostnet0,vhost=on,vhostfd=30 -device 
virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:f1:61:dd,bus=pci.0,addr=0x3 
-chardev 
file,id=charserial0,path=/var/lib/nova/instances/d9cf97c9-a9c1-4d13-81b0-344eb061ffbf/console.log 
-device isa-serial,chardev=charserial0,id=serial0 -chardev 
pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 
-device usb-tablet,id=input0,bus=usb.0,port=1 -vnc 10.54.12.15:2 -k 
en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device 
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg timestamp=on











More information about the discuss mailing list