[ovs-discuss] network bandwidth is too small in Openstack when using OVS+VLAN
Margin Hu
jhu_com at 163.com
Thu Mar 29 01:44:18 UTC 2018
I find an exception that the bridge br-int rx_dropped shows a lot of
packets were dropped.
# ifconfig br-int
br-int: flags=4098<BROADCAST,MULTICAST> mtu 1500
ether f2:60:24:57:a5:49 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 108368 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
# ovsdb-client -f list dump
Interface table
_uuid : b6013cf0-9ecf-40a3-a47b-117fe7d20213
admin_state : down
bfd : {}
bfd_status : {}
cfm_fault : []
cfm_fault_status : []
cfm_flap_count : []
cfm_health : []
cfm_mpid : []
cfm_remote_mpids : []
cfm_remote_opstate : []
duplex : []
error : []
external_ids : {}
ifindex : 20
ingress_policing_burst: 0
ingress_policing_rate: 0
lacp_current : []
link_resets : 0
link_speed : []
link_state : down
lldp : {}
mac : []
mac_in_use : "f2:60:24:57:a5:49"
mtu : 1500
mtu_request : []
name : br-int
ofport : 65534
ofport_request : []
options : {}
other_config : {}
statistics : {collisions=0, rx_bytes=0, rx_crc_err=0, rx_dropped=108260,
rx_errors=0, rx_frame_err=0, rx_over_err=0, rx_packets=0, tx_bytes=0,
tx_dropped=0, tx_errors=0, tx_packets=0}
status : {driver_name=openvswitch}
type : internal
If your environment has no this issue, can you help me show "ifconfig
br-int" ? I want to know it is culprit or not.
On 2018/3/28 15:34, Margin Hu wrote:
> Today, I took a compute node to reinstall centos7 , run a kvm vm with
> openvswitch ,
>
> traffic flow is : vm-> openvswitch -> physnic card -> another
> physcial server
>
> iperf3 results show bandwidth is 7.7Gbits/s , it is normal.
>
> On 2018/3/27 20:31, Margin Hu wrote:
>
>> Hi Guys,
>>
>> I met a strange issue about network bandwidth in Openstack when using
>> OVS+VLAN.
>>
>> version info:
>>
>> openvswitch-2.6.1-4.1.git20161206.el7.x86_64
>>
>> all server's os is centos7 , kernel is 3.10.0-514.36.5.el7.x86_64
>>
>> In my test setup, there are 3 servers.
>>
>> 1. physical server A ;
>> 2. compute node B which run kvm as hypervisor, and has a 10Gbits/s nic
>> 3. instance on B is C
>>
>> B-A iperf output shows 7.8Gbits/s , C-A iperf output shows only
>> 1.3Gbits/s , the Instance has been enabled vhost-net .
>>
>> I don't know where root cause is. I notice cpu utilization is very
>> low, only used 1% on B and C when iperf testing.
>>
>> in the instance C ,
>>
>> [root at perf2 ~]# ethtool -k eth0
>> Features for eth0:
>> rx-checksumming: on [fixed]
>> tx-checksumming: on
>> tx-checksum-ipv4: off [fixed]
>> tx-checksum-ip-generic: on
>> tx-checksum-ipv6: off [fixed]
>> tx-checksum-fcoe-crc: off [fixed]
>> tx-checksum-sctp: off [fixed]
>> scatter-gather: on
>> tx-scatter-gather: on
>> tx-scatter-gather-fraglist: off [fixed]
>> tcp-segmentation-offload: on
>> tx-tcp-segmentation: on
>> tx-tcp-ecn-segmentation: on
>> tx-tcp6-segmentation: on
>> udp-fragmentation-offload: on
>> generic-segmentation-offload: on
>> generic-receive-offload: on
>> large-receive-offload: off [fixed]
>> rx-vlan-offload: off [fixed]
>> tx-vlan-offload: off [fixed]
>> ntuple-filters: off [fixed]
>> receive-hashing: off [fixed]
>> highdma: on [fixed]
>> rx-vlan-filter: on [fixed]
>> vlan-challenged: off [fixed]
>> tx-lockless: off [fixed]
>> netns-local: off [fixed]
>> tx-gso-robust: off [fixed]
>> tx-fcoe-segmentation: off [fixed]
>> tx-gre-segmentation: off [fixed]
>> tx-ipip-segmentation: off [fixed]
>> tx-sit-segmentation: off [fixed]
>> tx-udp_tnl-segmentation: off [fixed]
>> tx-mpls-segmentation: off [fixed]
>> fcoe-mtu: off [fixed]
>> tx-nocache-copy: off
>> loopback: off [fixed]
>> rx-fcs: off [fixed]
>> rx-all: off [fixed]
>> tx-vlan-stag-hw-insert: off [fixed]
>> rx-vlan-stag-hw-parse: off [fixed]
>> rx-vlan-stag-filter: off [fixed]
>> busy-poll: off [fixed]
>> tx-sctp-segmentation: off [fixed]
>> l2-fwd-offload: off [fixed]
>> hw-tc-offload: off [fixed]
>>
>> in the compute node B :
>>
>> [root at cloud-sz-compute-b10-04 ~]# ethtool -k ens1f1
>> Features for ens1f1:
>> rx-checksumming: on
>> tx-checksumming: on
>> tx-checksum-ipv4: on
>> tx-checksum-ip-generic: off [fixed]
>> tx-checksum-ipv6: on
>> tx-checksum-fcoe-crc: off [fixed]
>> tx-checksum-sctp: off [fixed]
>> scatter-gather: on
>> tx-scatter-gather: on
>> tx-scatter-gather-fraglist: off [fixed]
>> tcp-segmentation-offload: on
>> tx-tcp-segmentation: on
>> tx-tcp-ecn-segmentation: off [fixed]
>> tx-tcp6-segmentation: on
>> udp-fragmentation-offload: off [fixed]
>> generic-segmentation-offload: on
>> generic-receive-offload: on
>> large-receive-offload: off [requested on]
>> rx-vlan-offload: on
>> tx-vlan-offload: on
>> ntuple-filters: off
>> receive-hashing: on
>> highdma: on [fixed]
>> rx-vlan-filter: on
>> vlan-challenged: off [fixed]
>> tx-lockless: off [fixed]
>> netns-local: off [fixed]
>> tx-gso-robust: off [fixed]
>> tx-fcoe-segmentation: off [fixed]
>> tx-gre-segmentation: off [fixed]
>> tx-ipip-segmentation: off [fixed]
>> tx-sit-segmentation: off [fixed]
>> tx-udp_tnl-segmentation: on
>> tx-mpls-segmentation: off [fixed]
>> fcoe-mtu: off [fixed]
>> tx-nocache-copy: off
>> loopback: off [fixed]
>> rx-fcs: off [fixed]
>> rx-all: off
>> tx-vlan-stag-hw-insert: off [fixed]
>> rx-vlan-stag-hw-parse: off [fixed]
>> rx-vlan-stag-filter: off [fixed]
>> busy-poll: off [fixed]
>> tx-sctp-segmentation: off [fixed]
>> l2-fwd-offload: off [fixed]
>> hw-tc-offload: off [fixed]
>>
>> [root at cloud-sz-compute-b10-04 boot]# ps uax |grep instance-00000036
>> root 14298 0.0 0.0 112644 968 pts/22 S+ 18:32 0:00 grep
>> --color=auto instance-00000036
>> 42427 14747 6.2 2.6 316007880 13724096 ? Sl 10:05 31:41
>> /usr/libexec/qemu-kvm -name guest=instance-00000036,deb
>> ug-threads=on -S -object
>> secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-3-instance-00000036/master-key.aes
>> -machine pc-i440fx-rhel7.3.0,accel=kvm,usb=off -cpu
>> Broadwell,+vme,+ds,+acpi,+ss,+ht,+tm,+pbe,+dtes64,+ds_cpl,+vmx,+smx,+est,+tm2,+xtpr,+pdcm,+dca,+osxsave,+f16c,+rdrand,+arat,+tsc_adjust,+xsaveopt,+pdpe1gb,+abm,+rtm,+hle
>> -m 307200 -realtime mlock=off -smp 64,sockets=64,cores=1,threads=1
>> -uuid d9cf97c9-a9c1-4d13-81b0-344eb061ffbf -smbios
>> type=1,manufacturer=RDO,product=OpenStack
>> Compute,version=15.0.4-1.el7,serial=c5e75cea-0425-9467-e711-ca25e82136af,uuid=d9cf97c9-a9c1-4d13-81b0-344eb061ffbf,family=Virtual
>> Machine -no-user-config -nodefaults -chardev
>> socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-3-instance-00000036/monitor.sock,server,nowait
>> -mon chardev=charmonitor,id=monitor,mode=control -rtc
>> base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard
>> -no-hpet -no-shutdown -boot strict=on -device
>> piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
>> file=/dev/disk/by-id/dm-uuid-mpath-3600a09803830434972244a78424b3479,format=raw,if=none,id=drive-virtio-disk0,serial=3a9cf4fd-47a8-4b0b-97fc-1b91b81f8770,cache=none,aio=native
>> -device
>> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
>> -netdev tap,fd=29,id=hostnet0,vhost=on,vhostfd=30 -device
>> virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:f1:61:dd,bus=pci.0,addr=0x3
>> -chardev
>> file,id=charserial0,path=/var/lib/nova/instances/d9cf97c9-a9c1-4d13-81b0-344eb061ffbf/console.log
>> -device isa-serial,chardev=charserial0,id=serial0 -chardev
>> pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1
>> -device usb-tablet,id=input0,bus=usb.0,port=1 -vnc 10.54.12.15:2 -k
>> en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device
>> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg timestamp=on
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> discuss mailing list
>> discuss at openvswitch.org
>> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20180329/e27661bb/attachment-0001.html>
More information about the discuss
mailing list