<div dir="rtl"><div dir="rtl"><div dir="rtl"><div dir="ltr">Calling it from net_dev_init doesn't look clean.</div><div dir="ltr">If you will see who is calling 'dev_add_offload' you will find out that there are couple of modules that doing that except of the 8021q. so either all modules should add offload from net_dev_init or stay with the current model where each module adds its offload.</div><div dir="ltr"><br></div><div dir="ltr">When adding a vlan interface using 'ip link add', the module 8021q is loaded. What is the flow that causing ip link to load this module?</div><div dir="ltr">Why can't we do the same thing when a vlan interface is created from ovs?</div><div dir="ltr"><br></div><div dir="ltr">Jiri,</div><div dir="ltr">Any other idea how we can solve this issue?</div><div dir="ltr"><br></div><div dir="ltr">Michael </div><div dir="ltr"><br></div></div></div></div><br><div class="gmail_quote"><div dir="rtl">בתאריך יום א׳, 28 באוק׳ 2018 ב-18:54 מאת Jiri Pirko <<a href="mailto:jiri@resnulli.us">jiri@resnulli.us</a>>:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Sun, Oct 28, 2018 at 03:03:42PM CET, <a href="mailto:michaelsh86@gmail.com" target="_blank">michaelsh86@gmail.com</a> wrote:<br>
>Jiri,<br>
>I am not sure it would be simple to move the add_offload to vlan_Core.c as<br>
>the add_offload should happen once.<br>
<br>
Just call it from net_dev_init()<br>
<br>
>in vlan.c it's done as part of module init but in vlan_core.c we are not<br>
>initializing any model and making some logic for doing that once looks<br>
>awkward to me.<br>
>Why can't we just load the module (8021q) once a vlan interface is added<br>
>via OVS?<br>
<br>
I have no clue how exactly would you like to do it...<br>
<br>
<br>
><br>
><br>
>בתאריך יום ה׳, 25 באוק׳ 2018 ב-15:15 מאת Jiri Pirko <<a href="mailto:jiri@resnulli.us" target="_blank">jiri@resnulli.us</a><br>
>>:<br>
><br>
>> Thu, Oct 25, 2018 at 01:34:04PM CEST, <a href="mailto:michaelsh86@gmail.com" target="_blank">michaelsh86@gmail.com</a> wrote:<br>
>> >I'v used ftrace to see which paths we take in the kernel.<br>
>> >I remind you that the packet (in my scenario) looks like the following:<br>
>> >ETH | IP | UDP | VXLAN | L2 | VLAN | IP | TCP | payload<br>
>> ><br>
>> >When 8021q is not loaded:<br>
>> >dev_gro_receive <-napi_gro_receive<br>
>> >inet_gro_receive <-dev_gro_receive<br>
>> >udp4_gro_receive <-inet_gro_receive<br>
>> >udp_gro_receive <-udp4_gro_receive<br>
>> >udp4_lib_lookup_skb <-udp_gro_receive<br>
>> >__udp4_lib_lookup <-udp4_lib_lookup_skb<br>
>> >compute_score <-__udp4_lib_lookup<br>
>> >vxlan_gro_receive <-udp_gro_receive<br>
>> >__pskb_pull_tail <-vxlan_gro_receive<br>
>> >skb_copy_bits <-__pskb_pull_tail<br>
>> >eth_gro_receive <-vxlan_gro_receive<br>
>> >__pskb_pull_tail <-eth_gro_receive<br>
>> >skb_copy_bits <-__pskb_pull_tail<br>
>> >gro_find_receive_by_type <-eth_gro_receive<br>
>> >netif_receive_skb_internal <-napi_gro_receive<br>
>> >ktime_get_real <-netif_receive_skb_internal<br>
>> >getnstimeofday64 <-ktime_get_real<br>
>> >__getnstimeofday64 <-getnstimeofday64<br>
>> >skb_defer_rx_timestamp <-netif_receive_skb_internal<br>
>> >classify <-skb_defer_rx_timestamp<br>
>> >__netif_receive_skb <-netif_receive_skb_internal<br>
>> >__netif_receive_skb_core <-__netif_receive_skb<br>
>> >ip_rcv <-__netif_receive_skb_core<br>
>> >nf_hook_slow <-ip_rcv<br>
>> >nf_iterate <-nf_hook_slow<br>
>> >ipv4_conntrack_defrag <-nf_iterate<br>
>> >ipv4_conntrack_in <-nf_iterate<br>
>> ><br>
>> >When 8021q was loaded manually:<br>
>> ><br>
>> ><br>
>> >inet_gro_receive <-dev_gro_receive<br>
>> >classify <-skb_clone_tx_timestamp<br>
>> >udp4_gro_receive <-inet_gro_receive<br>
>> >_raw_spin_lock <-sch_direct_xmit<br>
>> >udp_gro_receive <-udp4_gro_receive<br>
>> >udp4_lib_lookup_skb <-udp_gro_receive<br>
>> >local_bh_enable <-__dev_queue_xmit<br>
>> >__udp4_lib_lookup <-udp4_lib_lookup_skb<br>
>> >__local_bh_enable_ip <-local_bh_enable<br>
>> >compute_score <-__udp4_lib_lookup<br>
>> >local_bh_enable <-ip_finish_output<br>
>> >vxlan_gro_receive <-udp_gro_receive<br>
>> >__local_bh_enable_ip <-local_bh_enable<br>
>> >__pskb_pull_tail <-vxlan_gro_receive<br>
>> >skb_copy_bits <-__pskb_pull_tail<br>
>> >local_bh_enable <-__dev_queue_xmit<br>
>> >eth_gro_receive <-vxlan_gro_receive<br>
>> >__local_bh_enable_ip <-local_bh_enable<br>
>> >__pskb_pull_tail <-eth_gro_receive<br>
>> >skb_copy_bits <-__pskb_pull_tail<br>
>> >gro_find_receive_by_type <-eth_gro_receive<br>
>> >local_bh_enable <-__dev_queue_xmit<br>
>> >__local_bh_enable_ip <-local_bh_enable<br>
>> >vlan_gro_receive <-eth_gro_receive<br>
>> >__pskb_pull_tail <-vlan_gro_receive<br>
>> >local_bh_enable <-ip_finish_output<br>
>> >skb_copy_bits <-__pskb_pull_tail<br>
>> >__local_bh_enable_ip <-local_bh_enable<br>
>> >gro_find_receive_by_type <-vlan_gro_receive<br>
>> >inet_gro_receive <-vlan_gro_receive<br>
>> >kfree_skb_partial <-tcp_rcv_established<br>
>> >__pskb_pull_tail <-inet_gro_receive<br>
>> >skb_release_all <-kfree_skb_partial<br>
>> >skb_copy_bits <-__pskb_pull_tail<br>
>> >skb_release_head_state <-skb_release_all<br>
>> >skb_release_data <-skb_release_all<br>
>> >tcp4_gro_receive <-inet_gro_receive<br>
>> >tcp_gro_receive <-tcp4_gro_receive<br>
>> >put_page <-skb_release_data<br>
>> ><br>
>> >You can see that eth_gro_receive calls vlan_gro_recieve when 8021q module<br>
>> >is loaded and then performs ding gro_receive for the next layers as well<br>
>> >(inet, tcp), whereas on default scenario where 8021q is not loaded the<br>
>> vlan<br>
>> >device ndo cannot be used.<br>
>><br>
>> Ah, got it.<br>
>><br>
>> What happens is that during the module load, vlan_proto_init() registers<br>
>> vlan_packet_offloads:<br>
>><br>
>> for (i = 0; i < ARRAY_SIZE(vlan_packet_offloads); i++)<br>
>> dev_add_offload(&vlan_packet_offloads[i]);<br>
>><br>
>> I think that it would make sense to move this and the related code<br>
>> to net/8021q/vlan_core.c<br>
>> Then the offloads will be there regardless if the vlan module is loaded<br>
>> or not.<br>
>><br>
>> Please feel free to spin-up a patch, will be more than happy to review<br>
>> it. Thanks!<br>
>><br>
>> Jiri<br>
>><br>
>><br>
>><br>
>> ><br>
>> >בתאריך יום ה׳, 25 באוק׳ 2018 ב-9:40 מאת Jiri Pirko <<a href="mailto:jiri@resnulli.us" target="_blank">jiri@resnulli.us</a><br>
>> >>:<br>
>> ><br>
>> >> Wed, Oct 24, 2018 at 10:12:54PM CEST, <a href="mailto:michaelsh86@gmail.com" target="_blank">michaelsh86@gmail.com</a> wrote:<br>
>> >> >Hi,<br>
>> >> >I noticed that there is a performance issue when running traffic on a<br>
>> vlan<br>
>> >> >interface that was created by OVS.<br>
>> >> >If we create a bridge with a vlan interface, the 8021q module is not<br>
>> >> loaded.<br>
>> >> >Then when packets with a 8021q tag arrives, the linux stack can't use<br>
>> the<br>
>> >> >vlan ndos (such as gro_recieve) since there is no such vlan device.<br>
>> >> >If I perform the same test after loading the 8021q module, I get 2x-5x<br>
>> >> >better performance.<br>
>> >><br>
>> >> Could you please describe why exacly do you think you see this increase?<br>
>> >><br>
>> >><br>
>> >> >I personally test that using tunnels (vxlan) but I think the issue<br>
>> exists<br>
>> >> >even without the tunnel.<br>
>> >> ><br>
>> >> >*Creating a bridge and vlan interface:*<br>
>> >> >ovs-vsctl --no-wait add-br br_0<br>
>> >> >ip link set br_0 up<br>
>> >> >ovs-vsctl --no-wait add-port br_0 ens5f0<br>
>> >> >ovs-vsctl --no-wait add-port br_0 vlan10 tag=10 -- set interface vlan10<br>
>> >> >type=internal<br>
>> >> >ip link set vlan10 up<br>
>> >> >ip addr add <ip> dev vlan10<br>
>> >> ><br>
>> >> >run traffic (netperf on the vlan10 interface) and see the result.<br>
>> >> >Then do 'modprobe 8021q' and perform the same test -> you will see much<br>
>> >> >better numbers.<br>
>> >> ><br>
>> >> >I am suggesting to load 80021q module once vlan interface is added<br>
>> through<br>
>> >> >ovs.<br>
>> >> >similarly as a vlan interface would be created using 'ip link'.<br>
>> >> ><br>
>> >> >Michael<br>
>> >><br>
>><br>
</blockquote></div>