[ovs-dev] ovs segments frames when transmitting to veth

Ramu Ramamurthy sramamur at linux.vnet.ibm.com
Wed Jan 13 23:51:10 UTC 2016


Problem:
---------

We are running centos 7.2 (ovs-kernel module from distribution) with OVS 
2.3.1.

Large tcp frames (> 30Kbytes gro'd) are coming into a vxlan-tunnel port 
and switched by an ovs-bridge onto a veth port

vxlan-tunnel--(gro'd tcp frames sized > 30K)-->OVS-bridge--(frames sized 
1448)-->veth port

As the ovs transmits onto the veth port, the frames are getting 
segmented into 1448 byte sized packets. All the offload settings are at 
default on the veth port (gso on, gro on, tso on)

We expect that the OVS can transmit the frame without segmentation onto 
the veth port. But, due to segmentation, the performance suffers.

Root Cause
-----------

The following trace with debugs shows the root cause that the gso_type 
is not correctly set on
the skb as the packet comes in, and as a result, the dev_xmit_skb is 
segmenting the frame as ovs transmits it.




     ksoftirqd/25-231   [025] ..s.   319.964120: vxlan_rcv: 
gso_size=1448,gso_type=1001

         The above is debug in vport-vxlan.c:vxlan_rcv:        
trace_printk("gso_size=%d,gso_type=%x\n",skb_shinfo(skb)->gso_size,skb_shinfo(skb)->gso_type);
         Here, gso_type seem to have an invalid value.

     ksoftirqd/25-231   [025] ..s.   319.964120: ovs_vport_receive 
<-vxlan_rcv
     ksoftirqd/25-231   [025] ..s.   319.964121: ovs_flow_key_extract 
<-ovs_vport_receive
     ksoftirqd/25-231   [025] ..s.   319.964121: key_extract 
<-ovs_flow_key_extract
     ksoftirqd/25-231   [025] ..s.   319.964121: check_header 
<-key_extract
     ksoftirqd/25-231   [025] ..s.   319.964121: tcphdr_ok <-key_extract
     ksoftirqd/25-231   [025] ..s.   319.964121: ovs_dp_process_packet 
<-ovs_vport_receive
     ksoftirqd/25-231   [025] ..s.   319.964121: 
ovs_flow_tbl_lookup_stats <-ovs_dp_process_packet
     ksoftirqd/25-231   [025] ..s.   319.964121: masked_flow_lookup 
<-ovs_flow_tbl_lookup_stats
     ksoftirqd/25-231   [025] ..s.   319.964121: ovs_flow_mask_key 
<-masked_flow_lookup
     ksoftirqd/25-231   [025] ..s.   319.964122: find_bucket.isra.2 
<-masked_flow_lookup
     ksoftirqd/25-231   [025] ..s.   319.964122: masked_flow_lookup 
<-ovs_flow_tbl_lookup_stats
     ksoftirqd/25-231   [025] ..s.   319.964122: ovs_flow_mask_key 
<-masked_flow_lookup
     ksoftirqd/25-231   [025] ..s.   319.964122: find_bucket.isra.2 
<-masked_flow_lookup
     ksoftirqd/25-231   [025] ..s.   319.964122: ovs_flow_stats_update 
<-ovs_dp_process_packet
     ksoftirqd/25-231   [025] ..s.   319.964122: _raw_spin_loca 
<-ovs_flow_stats_update
     ksoftirqd/25-231   [025] ..s.   319.964122: _raw_spin_unlock 
<-ovs_flow_stats_update
     ksoftirqd/25-231   [025] ..s.   319.964122: ovs_execute_actions 
<-ovs_dp_process_packet
     ksoftirqd/25-231   [025] ..s.   319.964123: do_execute_actions 
<-ovs_execute_actions
     ksoftirqd/25-231   [025] ..s.   319.964123: do_output 
<-do_execute_actions
     ksoftirqd/25-231   [025] ..s.   319.964123: ovs_lookup_vport 
<-do_output
     ksoftirqd/25-231   [025] ..s.   319.964123: ovs_vport_send 
<-do_output
     ksoftirqd/25-231   [025] ..s.   319.964123: netdev_send 
<-ovs_vport_send



     ksoftirqd/25-231   [025] ..s.   319.964123: netdev_send: devmtu=8950
     ksoftirqd/25-231   [025] ..s.   319.964123: netdev_send: 
devname=ovs-veth0
     ksoftirqd/25-231   [025] ..s.   319.964124: netdev_send: gso_segs=45
     ksoftirqd/25-231   [025] ..s.   319.964124: netif_skb_features 
<-netdev_send
     ksoftirqd/25-231   [025] ..s.   319.964124: skb_network_protocol 
<-netif_skb_features
     ksoftirqd/25-231   [025] ..s.   319.964124: netdev_send: ip_summed=3
     ksoftirqd/25-231   [025] ..s.   319.964124: netdev_send: 
gso_segs=45,len=65226,feat=6103db59e9
     ksoftirqd/25-231   [025] ..s.   319.964124: netdev_send: 
net_gso_ok=0,has_frag_list=1,frag_feat=1
     ksoftirqd/25-231   [025] ..s.   319.964125: netdev_send: 
gso_size=1448,gso_type=4097,ok=0

       The above are debugs in vport-netdev.c:netdev_send:
         trace_printk("devmtu=%d\n",mtu);
         trace_printk("devname=%s\n",netdev_vport->dev->name);
         trace_printk("gso_segs=%d\n",skb_shinfo(skb)->gso_segs);
         features = netif_skb_features(skb);
         feature = skb_shinfo(skb)->gso_type << NETIF_F_GSO_SHIFT;
         net_gso_ok = ((features & feature) == feature);
         gso_ok = skb_gso_ok(skb, features);
         has_frag_list = skb_has_frag_list(skb);
         frag_feat = features & NETIF_F_FRAGLIST ? 1 : 0;
         trace_printk("ip_summed=%d\n",skb->ip_summed);
         
trace_printk("gso_segs=%d,len=%d,feat=%llx\n",skb_shinfo(skb)->gso_segs,len,features);
            
trace_printk("net_gso_ok=%d,has_frag_list=%d,frag_feat=%d\n",net_gso_ok,has_frag_list,frag_feat);
         
trace_printk("gso_size=%d,gso_type=%d,ok=%d\n",skb_shinfo(skb)->gso_size,skb_shinfo(skb)->gso_type,gso_ok);




     ksoftirqd/25-231   [025] ..s.   319.964125: dev_queue_xmit 
<-netdev_send
     ksoftirqd/25-231   [025] ..s.   319.964125: local_bh_disable 
<-dev_queue_xmit
     ksoftirqd/25-231   [025] ..s.   319.964125: netdev_pick_tx 
<-dev_queue_xmit
     ksoftirqd/25-231   [025] ..s.   319.964125: _raw_spin_lock 
<-dev_queue_xmit
     ksoftirqd/25-231   [025] ..s.   319.964125: 
validate_xmit_skb.part.86 <-dev_queue_xmit
     ksoftirqd/25-231   [025] ..s.   319.964125: netif_skb_features 
<-validate_xmit_skb.part.86
     ksoftirqd/25-231   [025] ..s.   319.964125: skb_network_protocol 
<-netif_skb_features
                  validate_xmit_skb() calls netif_needs_gso()
                  netif_needs_gso() is true because:
                  skb_gso_ok is FALSE !! because net_gso_ok() is FALSE 
because
                  skb_shinfo(skb)->gso_type has a bad value of 0x1001 !!


     ksoftirqd/25-231   [025] ..s.   319.964125: __skb_gso_segment 
<-validate_xmit_skb.part.86
     ksoftirqd/25-231   [025] ..s.   319.964126: skb_mac_gso_segment 
<-__skb_gso_segment
     ksoftirqd/25-231   [025] ..s.   319.964126: skb_network_protocol 
<-skb_mac_gso_segment
     ksoftirqd/25-231   [025] ..s.   319.964126: inet_gso_segment 
<-skb_mac_gso_segment
     ksoftirqd/25-231   [025] ..s.   319.964126: tcp4_gso_segment 
<-inet_gso_segment
     ksoftirqd/25-231   [025] ..s.   319.964126: tcp_gso_segment 
<-tcp4_gso_segment
     ksoftirqd/25-231   [025] ..s.   319.964126: skb_segment 
<-tcp_gso_segment
                   Frame is getting segmented


I tried the following experiment: in vport-vxlan.c:vxlan_rcv() set the 
value of gso_type as:
   skb_shinfo(skb)->gso_type = skb_shinfo(skb)->gso_type&0xff;
With this change, frames dont segment onto the veth.

Has this sort of problem been fixed already in more recent versions ?




More information about the dev mailing list