<div dir="ltr"><div><div><div><div><div><div><div>Hi Joe,<br><br></div>In my first message I forgot to say that the interface on which to run the command must be an existing interface, not a dummy one as in my example. <br><br></div>Anyways, I recently tried on a 4.8 kernel and the problem disappeared. For easiness I used the version of openvswitch that I already had in the system, that is 2.4. I strongly suspect that it's a problem with kernel 4.6, independently of the version of openvswitch.<br><br></div><div></div><div>So in the setup which worked:<br></div>$ uname -a<br>Linux OVP800 4.8.0-0.bpo.2-amd64 #1 SMP Debian 4.8.11-1~bpo8+1 (2016-12-14) x86_64 GNU/Linux<br>$ ovs-vsctl --version <br>ovs-vsctl (Open vSwitch) 2.4.90<br><br></div>And the setup wich didn't work:<br>$ uname -a<br>Linux OVPi7RD2 4.6.0-1-amd64 #1 SMP Debian 4.6.1-1 (2016-06-06) x86_64 GNU/Linux<br>$ ovs-vsctl --version <br>ovs-vsctl (Open vSwitch) 2.4.90<br><br></div>I use Debian machines running Jessie. I can't install kernels 4.4 or 4.9 right away, but here is the log from the kernel 4.6 when it crashed:<br><br>Debian GNU/Linux 8 mymachine ttyS0 - 1.0.4.4:N<br><br>mymachine login: [ 433.265229] BUG: unable to handle kernel NULL pointer dereference at 000000000000000c<br>[ 433.273210] IP: [<ffffffff815c6237>] _raw_spin_lock_bh+0x17/0x30<br>[ 433.279317] PGD 0 <br>[ 433.281425] Oops: 0002 [#1] SMP <br>[ 433.284803] Modules linked in: act_police(E) cls_basic(E) sch_ingress(E) vhost_net(E) vhost(E) macvtap(E) macvlan(E) openvswitch(E) nf_conntrack_ipv6(E) nf_nat_ipv6(E) nf_conntrack_ipv4(E) nf_defrag_ipv4(E) nf_nat_ipv4(E) nf_defrag_ipv6(E) nf_nat(E) nf_conntrack(E) libcrc32c(E) veth(E) tun(E) bridge(E) stp(E) llc(E) cpufreq_conservative(E) cpufreq_userspace(E) cpufreq_powersave(E) cpufreq_stats(E) w83627ehf(E) hwmon_vid(E) igb_uio(OE) uio_pci_generic(E) uio(E) intel_rapl(E) x86_pkg_temp_thermal(E) intel_powerclamp(E) coretemp(E) kvm_intel(E) kvm(E) irqbypass(E) crct10dif_pclmul(E) crc32_pclmul(E) ghash_clmulni_intel(E) iTCO_wdt(E) jitterentropy_rng(E) iTCO_vendor_support(E) i915(E) hmac(E) evdev(E) drbg(E) ansi_cprng(E) aesni_intel(E) aes_x86_64(E) lrw(E) gf128mul(E) glue_helper(E) ablk_helper(E) cryptd(E) snd_hda_intel(E) snd_hda_codec(E) pcspkr(E) snd_hda_core(E) snd_hwdep(E) drm_kms_helper(E) snd_pcm(E) drm(E) i2c_i801(E) snd_timer(E) 8250_fintek(E) snd(E) tpm_tis(E) soundcore(E) lpc_ich(E) tpm(E) mfd_core(E) shpchp(E) battery(E) video(E) processor(E) button(E) ext4(E) crc16(E) jbd2(E) crc32c_generic(E) mbcache(E) sg(E) sd_mod(E) ahci(E) libahci(E) crc32c_intel(E) libata(E) xhci_pci(E) ehci_pci(E) scsi_mod(E) ehci_hcd(E) xhci_hcd(E) igb(E) i2c_algo_bit(E) dca(E) ptp(E) usbcore(E) pps_core(E) usb_common(E) fan(E) thermal(E) fjes(E)<br>[ 433.408872] CPU: 0 PID: 4244 Comm: qemu-system-x86 Tainted: G OE 4.6.0-1-amd64 #1 Debian 4.6.1-1<br>[ 433.418763] Hardware name: To be filled by O.E.M. To be filled by O.E.M./SHARKBAY, BIOS 4.6.5 10/08/2014<br>[ 433.428303] task: ffff8800d6674fc0 ti: ffff88040b7f0000 task.ti: ffff88040b7f0000<br>[ 433.435850] RIP: 0010:[<ffffffff815c6237>] [<ffffffff815c6237>] _raw_spin_lock_bh+0x17/0x30<br>[ 433.444417] RSP: 0018:ffff88041fa03e70 EFLAGS: 00010246<br>[ 433.449779] RAX: 0000000000000000 RBX: 000000000000000c RCX: ffff88040a3c2200<br>[ 433.456966] RDX: 0000000000000001 RSI: 0000000000000001 RDI: 000000000000000c<br>[ 433.464151] RBP: ffff88040a3c2200 R08: ffff88040a0e12e8 R09: 0000000000000100<br>[ 433.471335] R10: 00000000000001da R11: 000000000000303a R12: ffff88040a02b200<br>[ 433.478519] R13: ffff880023fda110 R14: 0000000000000001 R15: ffff88041fa16940<br>[ 433.485705] FS: 00007fcfcf7ed700(0000) GS:ffff88041fa00000(0000) knlGS:0000000000000000<br>[ 433.493856] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033<br>[ 433.499657] CR2: 000000000000000c CR3: 000000040b745000 CR4: 00000000001426f0<br>[ 433.506844] Stack:<br>[ 433.508906] ffffffff814f5042 ffff880023fda0f8 ffff88040a02b218 ffffffff814f573c<br>[ 433.516590] ffff880023fda108 ffff880023fda110 ffff88040a02b258 000000000000000a<br>[ 433.524270] 0000000000000008 ffff88041fa16940 ffffffff814f388b ffff880023fda100<br>[ 433.531956] Call Trace:<br>[ 433.534458] <IRQ> <br>[ 433.536422] [<ffffffff814f5042>] ? __tcf_hash_release+0x72/0xf0<br>[ 433.542724] [<ffffffff814f573c>] ? tcf_action_destroy+0x6c/0xa0<br>[ 433.548779] [<ffffffff814f388b>] ? tcf_exts_destroy+0x1b/0x30<br>[ 433.554666] [<ffffffffc0580213>] ? basic_delete_filter+0x13/0x30 [cls_basic]<br>[ 433.561855] [<ffffffff810dba59>] ? rcu_process_callbacks+0x1f9/0x5e0<br>[ 433.568354] [<ffffffff815c90b8>] ? __do_softirq+0xf8/0x28e<br>[ 433.573982] [<ffffffff8107fe3b>] ? irq_exit+0x9b/0xa0<br>[ 433.579173] [<ffffffff815c8ece>] ? smp_apic_timer_interrupt+0x3e/0x50<br>[ 433.585753] [<ffffffff815c71e2>] ? apic_timer_interrupt+0x82/0x90<br>[ 433.591985] <EOI> <br>[ 433.593952] [<ffffffffc0c02240>] ? vmx_invpcid_supported+0x20/0x20 [kvm_intel]<br>[ 433.601568] [<ffffffffc0c058d5>] ? vmx_interrupt_allowed+0x25/0x30 [kvm_intel]<br>[ 433.608958] [<ffffffffc0691140>] ? kvm_arch_vcpu_runnable+0xa0/0xd0 [kvm]<br>[ 433.615896] [<ffffffffc0675a8e>] ? kvm_vcpu_check_block+0xe/0x50 [kvm]<br>[ 433.622569] [<ffffffffc06761de>] ? kvm_vcpu_block+0x1ee/0x2b0 [kvm]<br>[ 433.628982] [<ffffffffc06918cc>] ? kvm_arch_vcpu_ioctl_run+0x75c/0x1520 [kvm]<br>[ 433.636274] [<ffffffffc0678c36>] ? kvm_vcpu_ioctl+0x316/0x5d0 [kvm]<br>[ 433.642678] [<ffffffff810ba0be>] ? __wake_up_common+0x4e/0x90<br>[ 433.648560] [<ffffffff81204a6d>] ? do_vfs_ioctl+0x9d/0x5c0<br>[ 433.654193] [<ffffffffc068390e>] ? kvm_on_user_return+0x3e/0x70 [kvm]<br>[ 433.660772] [<ffffffff81205004>] ? SyS_ioctl+0x74/0x80<br>[ 433.666048] [<ffffffff815c65b6>] ? system_call_fast_compare_end+0xc/0x96<br>[ 433.672888] Code: 01 00 00 00 c3 0f 1f 44 00 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 65 81 05 20 71 a4 7e 00 02 00 00 31 c0 ba 01 00 00 00 <f0> 0f b1 17 85 c0 75 02 f3 c3 89 c6 e8 18 a0 af ff 66 90 c3 0f <br>[ 433.695297] RIP [<ffffffff815c6237>] _raw_spin_lock_bh+0x17/0x30<br>[ 433.701501] RSP <ffff88041fa03e70><br>[ 433.705047] CR2: 000000000000000c<br>[ 433.708417] ---[ end trace befdabb552866e5d ]---<br>[ 433.713082] Kernel panic - not syncing: Fatal exception in interrupt<br>[ 433.719503] Kernel Offset: disabled<br>[ 433.723046] ---[ end Kernel panic - not syncing: Fatal exception in interrupt<br>[ 433.730244] ------------[ cut here ]------------<br>[ 433.734915] WARNING: CPU: 0 PID: 4244 at /build/linux-Jaz5I6/linux-4.6.1/arch/x86/kernel/smp.c:125 update_process_times+0x4a/0x60<br>[ 433.746629] Modules linked in: act_police(E) cls_basic(E) sch_ingress(E) vhost_net(E) vhost(E) macvtap(E) macvlan(E) openvswitch(E) nf_conntrack_ipv6(E) nf_nat_ipv6(E) nf_conntrack_ipv4(E) nf_defrag_ipv4(E) nf_nat_ipv4(E) nf_defrag_ipv6(E) nf_nat(E) nf_conntrack(E) libcrc32c(E) veth(E) tun(E) bridge(E) stp(E) llc(E) cpufreq_conservative(E) cpufreq_userspace(E) cpufreq_powersave(E) cpufreq_stats(E) w83627ehf(E) hwmon_vid(E) igb_uio(OE) uio_pci_generic(E) uio(E) intel_rapl(E) x86_pkg_temp_thermal(E) intel_powerclamp(E) coretemp(E) kvm_intel(E) kvm(E) irqbypass(E) crct10dif_pclmul(E) crc32_pclmul(E) ghash_clmulni_intel(E) iTCO_wdt(E) jitterentropy_rng(E) iTCO_vendor_support(E) i915(E) hmac(E) evdev(E) drbg(E) ansi_cprng(E) aesni_intel(E) aes_x86_64(E) lrw(E) gf128mul(E) glue_helper(E) ablk_helper(E) cryptd(E) snd_hda_intel(E) snd_hda_codec(E) pcspkr(E) snd_hda_core(E) snd_hwdep(E) drm_kms_helper(E) snd_pcm(E) drm(E) i2c_i801(E) snd_timer(E) 8250_fintek(E) snd(E) tpm_tis(E) soundcore(E) lpc_ich(E) tpm(E) mfd_core(E) shpchp(E) battery(E) video(E) processor(E) button(E) ext4(E) crc16(E) jbd2(E) crc32c_generic(E) mbcache(E) sg(E) sd_mod(E) ahci(E) libahci(E) crc32c_intel(E) libata(E) xhci_pci(E) ehci_pci(E) scsi_mod(E) ehci_hcd(E) xhci_hcd(E) igb(E) i2c_algo_bit(E) dca(E) ptp(E) usbcore(E) pps_core(E) usb_common(E) fan(E) thermal(E) fjes(E)<br>[ 433.870854] CPU: 0 PID: 4244 Comm: qemu-system-x86 Tainted: G D OE 4.6.0-1-amd64 #1 Debian 4.6.1-1<br>[ 433.880743] Hardware name: To be filled by O.E.M. To be filled by O.E.M./SHARKBAY, BIOS 4.6.5 10/08/2014<br>[ 433.890283] 0000000000000086 000000003b22f910 ffffffff81311425 0000000000000000<br>[ 433.897969] 0000000000000000 ffffffff8107a50e ffff8800d6674fc0 0000000000000000<br>[ 433.905655] ffff88041fa03c18 ffffffff810f1dd0 0000000000000003 ffff88041fa0ee40<br>[ 433.913344] Call Trace:<br>[ 433.915843] <IRQ> [<ffffffff81311425>] ? dump_stack+0x5c/0x77<br>[ 433.921872] [<ffffffff8107a50e>] ? __warn+0xbe/0xe0<br>[ 433.926890] [<ffffffff810f1dd0>] ? tick_sched_handle.isra.13+0x50/0x50<br>[ 433.933558] [<ffffffff810e2e5a>] ? update_process_times+0x4a/0x60<br>[ 433.939789] [<ffffffff810f1da0>] ? tick_sched_handle.isra.13+0x20/0x50<br>[ 433.946456] [<ffffffff810f1e08>] ? tick_sched_timer+0x38/0x70<br>[ 433.952336] [<ffffffff810e36ea>] ? __hrtimer_run_queues+0xea/0x280<br>[ 433.958657] [<ffffffff810e3e69>] ? hrtimer_interrupt+0x99/0x190<br>[ 433.964716] [<ffffffff815c8ec9>] ? smp_apic_timer_interrupt+0x39/0x50<br>[ 433.971294] [<ffffffff815c71e2>] ? apic_timer_interrupt+0x82/0x90<br>[ 433.977526] [<ffffffff8116f9fe>] ? panic+0x1e0/0x220<br>[ 433.982626] [<ffffffff8116f9f7>] ? panic+0x1d9/0x220<br>[ 433.987733] [<ffffffff8102faa3>] ? oops_end+0xc3/0xd0<br>[ 433.992925] [<ffffffff81064011>] ? no_context+0x131/0x370<br>[ 433.998460] [<ffffffff815c85f8>] ? page_fault+0x28/0x30<br>[ 434.003827] [<ffffffff815c6237>] ? _raw_spin_lock_bh+0x17/0x30<br>[ 434.009798] [<ffffffff814f5042>] ? __tcf_hash_release+0x72/0xf0<br>[ 434.015857] [<ffffffff814f573c>] ? tcf_action_destroy+0x6c/0xa0<br>[ 434.021915] [<ffffffff814f388b>] ? tcf_exts_destroy+0x1b/0x30<br>[ 434.027798] [<ffffffffc0580213>] ? basic_delete_filter+0x13/0x30 [cls_basic]<br>[ 434.034987] [<ffffffff810dba59>] ? rcu_process_callbacks+0x1f9/0x5e0<br>[ 434.041475] [<ffffffff815c90b8>] ? __do_softirq+0xf8/0x28e<br>[ 434.047099] [<ffffffff8107fe3b>] ? irq_exit+0x9b/0xa0<br>[ 434.052289] [<ffffffff815c8ece>] ? smp_apic_timer_interrupt+0x3e/0x50<br>[ 434.058871] [<ffffffff815c71e2>] ? apic_timer_interrupt+0x82/0x90<br>[ 434.065099] <EOI> [<ffffffffc0c02240>] ? vmx_invpcid_supported+0x20/0x20 [kvm_intel]<br>[ 434.073139] [<ffffffffc0c058d5>] ? vmx_interrupt_allowed+0x25/0x30 [kvm_intel]<br>[ 434.080516] [<ffffffffc0691140>] ? kvm_arch_vcpu_runnable+0xa0/0xd0 [kvm]<br>[ 434.087450] [<ffffffffc0675a8e>] ? kvm_vcpu_check_block+0xe/0x50 [kvm]<br>[ 434.094123] [<ffffffffc06761de>] ? kvm_vcpu_block+0x1ee/0x2b0 [kvm]<br>[ 434.100538] [<ffffffffc06918cc>] ? kvm_arch_vcpu_ioctl_run+0x75c/0x1520 [kvm]<br>[ 434.107831] [<ffffffffc0678c36>] ? kvm_vcpu_ioctl+0x316/0x5d0 [kvm]<br>[ 434.114235] [<ffffffff810ba0be>] ? __wake_up_common+0x4e/0x90<br>[ 434.120117] [<ffffffff81204a6d>] ? do_vfs_ioctl+0x9d/0x5c0<br>[ 434.125747] [<ffffffffc068390e>] ? kvm_on_user_return+0x3e/0x70 [kvm]<br>[ 434.132330] [<ffffffff81205004>] ? SyS_ioctl+0x74/0x80<br>[ 434.137605] [<ffffffff815c65b6>] ? system_call_fast_compare_end+0xc/0x96<br>[ 434.144445] ---[ end trace befdabb552866e5e ]---<br><br><br></div><div>Let me know if more details are needed.<br></div><div><br></div>Thanks!<br><br></div>Riccardo<br><div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jan 11, 2017 at 11:30 PM, Joe Stringer <span dir="ltr"><<a href="mailto:joe@ovn.org" target="_blank">joe@ovn.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="gmail-">On 10 January 2017 at 05:46, Riccardo R. <<a href="mailto:riccardoravaioli@gmail.com">riccardoravaioli@gmail.com</a>> wrote:<br>
> Hi,<br>
><br>
> By setting QoS parameters more than once on an OpenvSwitch bridge, we caused<br>
> kernel panic on our machine. This is<br>
> reproducible with the following commands:<br>
><br>
> $ ovs-vsctl add-br mybridge<br>
> $ ovs-vsctl add-port mybridge myiface<br>
> $ ovs-vsctl set interface myiface ingress_policing_rate=100<br>
> # with the command below the kernel will crash, regardless of the value<br>
> given as input:<br>
> $ ovs-vsctl set interface myiface ingress_policing_rate=1000<br>
><br>
> We tried this with openvswitch 2.4 and 2.6.1 on a 4.6 Linux kernel.<br>
> Interestingly, it works correctly on openvswitch 2.3.0 installed on a 3.16<br>
> kernel.<br>
<br>
</span>Are you able to get the kernel backtrace from the console when it crashes?<br>
<br>
Most likely this is a regression in upstream Linux for the kernel that<br>
you are seeing the problem. Can you provide more details? Eg, where<br>
did you get your 4.6 kernel from? What is the full "uname -r" version<br>
for it? What .config was used when compiling it? Are you able to also<br>
try other versions, eg 4.4 or 4.9?<br>
</blockquote></div><br></div></div></div></div></div>