[ovs-discuss] network bandwidth is too small in Openstack when using OVS+VLAN
Margin Hu
jhu_com at 163.com
Tue Mar 27 12:31:44 UTC 2018
Hi Guys,
I met a strange issue about network bandwidth in Openstack when using
OVS+VLAN.
version info:
openvswitch-2.6.1-4.1.git20161206.el7.x86_64
all server's os is centos7 , kernel is 3.10.0-514.36.5.el7.x86_64
In my test setup, there are 3 servers.
1. physical server A ;
2. compute node B which run kvm as hypervisor, and has a 10Gbits/s nic
3. instance on B is C
B-A iperf output shows 7.8Gbits/s , C-A iperf output shows only
1.3Gbits/s , the Instance has been enabled vhost-net .
I don't know where root cause is. I notice cpu utilization is very
low, only used 1% on B and C when iperf testing.
in the instance C ,
[root at perf2 ~]# ethtool -k eth0
Features for eth0:
rx-checksumming: on [fixed]
tx-checksumming: on
tx-checksum-ipv4: off [fixed]
tx-checksum-ip-generic: on
tx-checksum-ipv6: off [fixed]
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
tx-tcp-segmentation: on
tx-tcp-ecn-segmentation: on
tx-tcp6-segmentation: on
udp-fragmentation-offload: on
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: off [fixed]
tx-vlan-offload: off [fixed]
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: on [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-mpls-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
busy-poll: off [fixed]
tx-sctp-segmentation: off [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
in the compute node B :
[root at cloud-sz-compute-b10-04 ~]# ethtool -k ens1f1
Features for ens1f1:
rx-checksumming: on
tx-checksumming: on
tx-checksum-ipv4: on
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: on
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
tx-tcp-segmentation: on
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp6-segmentation: on
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [requested on]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: on
tx-mpls-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
busy-poll: off [fixed]
tx-sctp-segmentation: off [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
[root at cloud-sz-compute-b10-04 boot]# ps uax |grep instance-00000036
root 14298 0.0 0.0 112644 968 pts/22 S+ 18:32 0:00 grep
--color=auto instance-00000036
42427 14747 6.2 2.6 316007880 13724096 ? Sl 10:05 31:41
/usr/libexec/qemu-kvm -name guest=instance-00000036,deb
ug-threads=on -S -object
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-3-instance-00000036/master-key.aes
-machine pc-i440fx-rhel7.3.0,accel=kvm,usb=off -cpu
Broadwell,+vme,+ds,+acpi,+ss,+ht,+tm,+pbe,+dtes64,+ds_cpl,+vmx,+smx,+est,+tm2,+xtpr,+pdcm,+dca,+osxsave,+f16c,+rdrand,+arat,+tsc_adjust,+xsaveopt,+pdpe1gb,+abm,+rtm,+hle
-m 307200 -realtime mlock=off -smp 64,sockets=64,cores=1,threads=1 -uuid
d9cf97c9-a9c1-4d13-81b0-344eb061ffbf -smbios
type=1,manufacturer=RDO,product=OpenStack
Compute,version=15.0.4-1.el7,serial=c5e75cea-0425-9467-e711-ca25e82136af,uuid=d9cf97c9-a9c1-4d13-81b0-344eb061ffbf,family=Virtual
Machine -no-user-config -nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-3-instance-00000036/monitor.sock,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control -rtc
base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet
-no-shutdown -boot strict=on -device
piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
file=/dev/disk/by-id/dm-uuid-mpath-3600a09803830434972244a78424b3479,format=raw,if=none,id=drive-virtio-disk0,serial=3a9cf4fd-47a8-4b0b-97fc-1b91b81f8770,cache=none,aio=native
-device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
-netdev tap,fd=29,id=hostnet0,vhost=on,vhostfd=30 -device
virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:f1:61:dd,bus=pci.0,addr=0x3
-chardev
file,id=charserial0,path=/var/lib/nova/instances/d9cf97c9-a9c1-4d13-81b0-344eb061ffbf/console.log
-device isa-serial,chardev=charserial0,id=serial0 -chardev
pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1
-device usb-tablet,id=input0,bus=usb.0,port=1 -vnc 10.54.12.15:2 -k
en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg timestamp=on
More information about the discuss
mailing list