[ovs-dev] same tcp session encapsulated with different udp src port in kernel mode if packet has do ip_forward

[ychen]

we can easily reproduce this phenomenon by using tcp socket stream sending from ovs internal port.




At 2019-10-30 19:49:16, "ychen" <ychen103103 at 163.com> wrote:

Hi, 
   when we use docker to establish tcp session, we found that the packet which must do upcall to userspace has different encapsulated udp source port 
   with packet that only needs do datapath flow forwarding.


   After some code research and kprobe debug,  we found the following:
   1.  use udp_flow_src_port() to get the port
        so when both skb->l4_hash==0 and skb->sw_hash==0, 5 tuple data will be used to calculate the skb->hash
    2. when first packet of tcp session coming,  packet needs do upcall to userspace, and then ovs_packet_cmd_execute() called
        new skb is allocated with both l4_hash and sw_hash set to 0
    3. when none first packet of tcp sesion coming, function ovs_dp_process_packet()->ovs_execute_actions() called,
        and this time original skb is reserved. 
        when packet has do ip_forward(), kprobe debug prints skb->l4_hash=1, sw_hash=0
    4. we searched kernel code, and found such code:
         skb_set_hash_from_sk(struct sk_buff *skb, struct sock *sk)
        {  if (sk->sk_txhash) {
skb->l4_hash = 1;
skb->hash = sk->sk_txhash;
    }
       }
      static inline void sk_set_txhash(struct sock *sk)
      {sk->sk_txhash = net_tx_rndhash();  ==============>it is a random value!!}
   5. so let's have a summary:
       when packet is processing only in datapath flow, skb->hash is random value for the same tcp session?
       when packet needs processing first to userspace, than kernel space, skb->hash is calculated by 5 tuple?
    
   Our testing enviroment:
   debian 9, kernel 4.9.65
   ovs version: 2.8.2


   Simple topo is like this:
   docker_eth0<-------+
                                  | veth                 ip_forward
                                 +host_veth0<----------------->port-eth(ovs-ineternal)
    host_veth0 and port-eth device stay in physical host.


   So can we treat skb->hash as a attribute, when send packet to userspace, encode this attribute; 
   and then do ovs_packet_cmd_execute(), retrieve the same hash value from userspace?


  another important tips:
 if we send packets from qemu based tap device, vxlan source port is always same for the same tcp session;
 only when send packets from docker in which packets will do ip_forward, vxlan source port may different for same tcp session.