[ovs-discuss] network bandwidth is too small in Openstack when using OVS+VLAN

Margin Hu jhu_com at 163.com
Thu Mar 29 01:44:18 UTC 2018


I find an exception that the bridge br-int rx_dropped shows a lot of 
packets were dropped.

# ifconfig br-int

br-int: flags=4098<BROADCAST,MULTICAST>  mtu 1500
         ether f2:60:24:57:a5:49  txqueuelen 1000  (Ethernet)
         RX packets 0  bytes 0 (0.0 B)
         RX errors 0  dropped 108368  overruns 0  frame 0
         TX packets 0  bytes 0 (0.0 B)
         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0


# ovsdb-client -f list dump

Interface table
_uuid : b6013cf0-9ecf-40a3-a47b-117fe7d20213
admin_state : down
bfd : {}
bfd_status : {}
cfm_fault : []
cfm_fault_status : []
cfm_flap_count : []
cfm_health : []
cfm_mpid : []
cfm_remote_mpids : []
cfm_remote_opstate : []
duplex : []
error : []
external_ids : {}
ifindex : 20
ingress_policing_burst: 0
ingress_policing_rate: 0
lacp_current : []
link_resets : 0
link_speed : []
link_state : down
lldp : {}
mac : []
mac_in_use : "f2:60:24:57:a5:49"
mtu : 1500
mtu_request : []
name : br-int
ofport : 65534
ofport_request : []
options : {}
other_config : {}
statistics : {collisions=0, rx_bytes=0, rx_crc_err=0, rx_dropped=108260, 
rx_errors=0, rx_frame_err=0, rx_over_err=0, rx_packets=0, tx_bytes=0, 
tx_dropped=0, tx_errors=0, tx_packets=0}
status : {driver_name=openvswitch}
type : internal

If your environment has no this issue, can you help me show "ifconfig 
br-int" ? I want to know it is culprit or not.


On 2018/3/28 15:34, Margin Hu wrote:
> Today, I took a compute node to reinstall centos7 , run a kvm vm with 
> openvswitch ,
>
> traffic flow is  :  vm-> openvswitch -> physnic card -> another 
> physcial server
>
> iperf3  results show bandwidth is 7.7Gbits/s , it is normal.
>
> On 2018/3/27 20:31, Margin Hu wrote:
>
>> Hi Guys,
>>
>> I met a strange issue about network bandwidth in Openstack when using 
>> OVS+VLAN.
>>
>> version info:
>>
>> openvswitch-2.6.1-4.1.git20161206.el7.x86_64
>>
>> all server's os is centos7 , kernel is 3.10.0-514.36.5.el7.x86_64
>>
>> In my test setup,  there are 3 servers.
>>
>> 1. physical server A  ;
>> 2. compute node B  which run kvm as hypervisor, and  has a 10Gbits/s nic
>> 3. instance on B is C
>>
>> B-A  iperf output shows  7.8Gbits/s , C-A  iperf output shows only 
>> 1.3Gbits/s  ,  the Instance has been enabled vhost-net .
>>
>> I don't know where root cause is.   I notice cpu utilization is very 
>> low, only used 1%  on B and C  when iperf testing.
>>
>> in the instance C ,
>>
>> [root at perf2 ~]# ethtool -k eth0
>> Features for eth0:
>> rx-checksumming: on [fixed]
>> tx-checksumming: on
>>         tx-checksum-ipv4: off [fixed]
>>         tx-checksum-ip-generic: on
>>         tx-checksum-ipv6: off [fixed]
>>         tx-checksum-fcoe-crc: off [fixed]
>>         tx-checksum-sctp: off [fixed]
>> scatter-gather: on
>>         tx-scatter-gather: on
>>         tx-scatter-gather-fraglist: off [fixed]
>> tcp-segmentation-offload: on
>>         tx-tcp-segmentation: on
>>         tx-tcp-ecn-segmentation: on
>>         tx-tcp6-segmentation: on
>> udp-fragmentation-offload: on
>> generic-segmentation-offload: on
>> generic-receive-offload: on
>> large-receive-offload: off [fixed]
>> rx-vlan-offload: off [fixed]
>> tx-vlan-offload: off [fixed]
>> ntuple-filters: off [fixed]
>> receive-hashing: off [fixed]
>> highdma: on [fixed]
>> rx-vlan-filter: on [fixed]
>> vlan-challenged: off [fixed]
>> tx-lockless: off [fixed]
>> netns-local: off [fixed]
>> tx-gso-robust: off [fixed]
>> tx-fcoe-segmentation: off [fixed]
>> tx-gre-segmentation: off [fixed]
>> tx-ipip-segmentation: off [fixed]
>> tx-sit-segmentation: off [fixed]
>> tx-udp_tnl-segmentation: off [fixed]
>> tx-mpls-segmentation: off [fixed]
>> fcoe-mtu: off [fixed]
>> tx-nocache-copy: off
>> loopback: off [fixed]
>> rx-fcs: off [fixed]
>> rx-all: off [fixed]
>> tx-vlan-stag-hw-insert: off [fixed]
>> rx-vlan-stag-hw-parse: off [fixed]
>> rx-vlan-stag-filter: off [fixed]
>> busy-poll: off [fixed]
>> tx-sctp-segmentation: off [fixed]
>> l2-fwd-offload: off [fixed]
>> hw-tc-offload: off [fixed]
>>
>> in the compute node B :
>>
>> [root at cloud-sz-compute-b10-04 ~]# ethtool -k ens1f1
>> Features for ens1f1:
>> rx-checksumming: on
>> tx-checksumming: on
>>         tx-checksum-ipv4: on
>>         tx-checksum-ip-generic: off [fixed]
>>         tx-checksum-ipv6: on
>>         tx-checksum-fcoe-crc: off [fixed]
>>         tx-checksum-sctp: off [fixed]
>> scatter-gather: on
>>         tx-scatter-gather: on
>>         tx-scatter-gather-fraglist: off [fixed]
>> tcp-segmentation-offload: on
>>         tx-tcp-segmentation: on
>>         tx-tcp-ecn-segmentation: off [fixed]
>>         tx-tcp6-segmentation: on
>> udp-fragmentation-offload: off [fixed]
>> generic-segmentation-offload: on
>> generic-receive-offload: on
>> large-receive-offload: off [requested on]
>> rx-vlan-offload: on
>> tx-vlan-offload: on
>> ntuple-filters: off
>> receive-hashing: on
>> highdma: on [fixed]
>> rx-vlan-filter: on
>> vlan-challenged: off [fixed]
>> tx-lockless: off [fixed]
>> netns-local: off [fixed]
>> tx-gso-robust: off [fixed]
>> tx-fcoe-segmentation: off [fixed]
>> tx-gre-segmentation: off [fixed]
>> tx-ipip-segmentation: off [fixed]
>> tx-sit-segmentation: off [fixed]
>> tx-udp_tnl-segmentation: on
>> tx-mpls-segmentation: off [fixed]
>> fcoe-mtu: off [fixed]
>> tx-nocache-copy: off
>> loopback: off [fixed]
>> rx-fcs: off [fixed]
>> rx-all: off
>> tx-vlan-stag-hw-insert: off [fixed]
>> rx-vlan-stag-hw-parse: off [fixed]
>> rx-vlan-stag-filter: off [fixed]
>> busy-poll: off [fixed]
>> tx-sctp-segmentation: off [fixed]
>> l2-fwd-offload: off [fixed]
>> hw-tc-offload: off [fixed]
>>
>> [root at cloud-sz-compute-b10-04 boot]# ps uax |grep instance-00000036
>> root      14298  0.0  0.0 112644   968 pts/22   S+   18:32 0:00 grep 
>> --color=auto instance-00000036
>> 42427     14747  6.2  2.6 316007880 13724096 ?  Sl   10:05 31:41 
>> /usr/libexec/qemu-kvm -name guest=instance-00000036,deb
>> ug-threads=on -S -object 
>> secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-3-instance-00000036/master-key.aes 
>> -machine pc-i440fx-rhel7.3.0,accel=kvm,usb=off -cpu 
>> Broadwell,+vme,+ds,+acpi,+ss,+ht,+tm,+pbe,+dtes64,+ds_cpl,+vmx,+smx,+est,+tm2,+xtpr,+pdcm,+dca,+osxsave,+f16c,+rdrand,+arat,+tsc_adjust,+xsaveopt,+pdpe1gb,+abm,+rtm,+hle 
>> -m 307200 -realtime mlock=off -smp 64,sockets=64,cores=1,threads=1 
>> -uuid d9cf97c9-a9c1-4d13-81b0-344eb061ffbf -smbios 
>> type=1,manufacturer=RDO,product=OpenStack 
>> Compute,version=15.0.4-1.el7,serial=c5e75cea-0425-9467-e711-ca25e82136af,uuid=d9cf97c9-a9c1-4d13-81b0-344eb061ffbf,family=Virtual 
>> Machine -no-user-config -nodefaults -chardev 
>> socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-3-instance-00000036/monitor.sock,server,nowait 
>> -mon chardev=charmonitor,id=monitor,mode=control -rtc 
>> base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard 
>> -no-hpet -no-shutdown -boot strict=on -device 
>> piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive 
>> file=/dev/disk/by-id/dm-uuid-mpath-3600a09803830434972244a78424b3479,format=raw,if=none,id=drive-virtio-disk0,serial=3a9cf4fd-47a8-4b0b-97fc-1b91b81f8770,cache=none,aio=native 
>> -device 
>> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 
>> -netdev tap,fd=29,id=hostnet0,vhost=on,vhostfd=30 -device 
>> virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:f1:61:dd,bus=pci.0,addr=0x3 
>> -chardev 
>> file,id=charserial0,path=/var/lib/nova/instances/d9cf97c9-a9c1-4d13-81b0-344eb061ffbf/console.log 
>> -device isa-serial,chardev=charserial0,id=serial0 -chardev 
>> pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 
>> -device usb-tablet,id=input0,bus=usb.0,port=1 -vnc 10.54.12.15:2 -k 
>> en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device 
>> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg timestamp=on
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> discuss mailing list
>> discuss at openvswitch.org
>> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20180329/e27661bb/attachment-0001.html>


More information about the discuss mailing list