[ovs-discuss] TC flow offload using OVS - failed to offload flow: Invalid argument: enp0s25
satish dhote
sdhote926 at gmail.com
Wed Feb 26 06:39:28 UTC 2020
Hi Moshe,
Sorry for the confusion. My recent response has new setup details, so
please ignore the instructions and setup details which I sent as part of
very first email.
Here is the fresh setup details and logs, and yes, ens11 is a physical link.
*Openvswitch version - *"2.12.0"
*Host machine - *
OS - Centos7.5
mellanox card - ConnectX-4 Lx
Physical interface ens11
*Virtual machines - *
2 VM's
OS - Minimal Centos7.5
Attached each VF to each virtual machine
*Steps followed to enable sriov and configuring ovs offload*
*# lspci | grep -i mell*
01:00.0 Ethernet controller: Mellanox Technologies MT27710 Family
[ConnectX-4 Lx]
*# ethtool -i ens11 | head -5*
driver: mlx5_core
version: 4.7-3.2.9
firmware-version: 14.26.4012 (MT_2410110034)
expansion-rom-version:
bus-info: 0000:01:00.0
*# cat /sys/class/net/ens11/device/sriov_totalvfs*
4
*# echo 2 > /sys/class/net/ens11/device/sriov_numvfs*
*# lspci | grep -i mell*
01:00.0 Ethernet controller: Mellanox Technologies MT27710 Family
[ConnectX-4 Lx]
01:00.1 Ethernet controller: Mellanox Technologies MT27710 Family
[ConnectX-4 Lx Virtual Function]
01:00.2 Ethernet controller: Mellanox Technologies MT27710 Family
[ConnectX-4 Lx Virtual Function]
*#cat /sys/class/net/ens11/device/sriov_numvfs*
2
*# echo nic_netdev > /sys/class/net/ens11/compat/devlink/uplink_rep_mode*
*# echo 0000:01:00.1 > /sys/bus/pci/drivers/mlx5_core/unbind*
*# echo 0000:01:00.2 > /sys/bus/pci/drivers/mlx5_core/unbind*
*# echo switchdev > /sys/class/net/ens11/compat/devlink/mode*
*# systemctl start openvswitch*
*# ovs-vsctl add-br ovs-sriov*
*# ovs-vsctl set Open_vSwitch . other_config:hw-offload=true*
*# systemctl restart openvswitch*
*# ovs-vsctl add-port ovs-sriov ens11*
*# ovs-vsctl add-port ovs-sriov eth0*
*# ovs-vsctl add-port ovs-sriov eth1*
# ovs-vsctl show
8bcbcf4b-e1df-4e79-b218-075d9d77568b
Bridge ovs-sriov
Port "ens11"
Interface "ens11"
Port "eth1"
Interface "eth1"
Port ovs-sriov
Interface ovs-sriov
type: internal
Port "eth0"
Interface "eth0"
*Traffic type from one from one VM to another VM:*
ICMP(command: ping), TCP(command: ssh)
*Logs:*
dpif_netlink(handler6)|ERR|Dropped 5 log messages in last 7 seconds (most
recently, 3 seconds ago) due to excessive rate
dpif_netlink(handler6)|ERR|failed to offload flow: Invalid argument: ens11
dpif_netlink(handler6)|ERR|Dropped 3 log messages in last 8 seconds (most
recently, 1 seconds ago) due to excessive rate
dpif_netlink(handler6)|ERR|failed to offload flow: Invalid argument: eth1
Thanks
Satish
On Wed, Feb 26, 2020 at 11:38 AM Moshe Levi <moshele at mellanox.com> wrote:
> You uplink (or physical link) is ens11 so why did you add enp0s25 to the
> bridge?
>
>
>
> *From:* discuss <ovs-discuss-bounces at openvswitch.org> * On Behalf Of *satish
> dhote
> *Sent:* Wednesday, February 26, 2020 8:01 AM
> *To:* Roni Bar Yanai <roniba at mellanox.com>
> *Cc:* ovs-discuss at openvswitch.org
> *Subject:* Re: [ovs-discuss] TC flow offload using OVS - failed to
> offload flow: Invalid argument: enp0s25
>
>
>
> Hi Roni,
>
>
>
> Thanks for your response.
>
> Below are the setup details.
>
>
>
> *Openvswitch version - *"2.12.0"
>
>
>
> *Host machine - *
>
> OS - Centos7.5
>
> mellanox card - ConnectX-4 Lx
>
> Physical interface ens11
>
>
>
> *Virtual machines - *
>
> 2 VM's
>
> OS - Minimal Centos7.5
>
> Attached each VF to each virtual machine
>
>
>
> *Steps followed to enable sriov and configuring ovs offload*
>
> *# lspci | grep -i mell*
>
> 01:00.0 Ethernet controller: Mellanox Technologies MT27710 Family
> [ConnectX-4 Lx]
>
>
>
> *# ethtool -i ens11 | head -5*
>
> driver: mlx5_core
>
> version: 4.7-3.2.9
>
> firmware-version: 14.26.4012 (MT_2410110034)
>
> expansion-rom-version:
>
> bus-info: 0000:01:00.0
>
>
>
> *# cat /sys/class/net/ens11/device/sriov_totalvfs*
>
> 4
>
>
>
> *# echo 2 > /sys/class/net/ens11/device/sriov_numvfs*
>
>
>
> *# lspci | grep -i mell*
>
> 01:00.0 Ethernet controller: Mellanox Technologies MT27710 Family
> [ConnectX-4 Lx]
>
> 01:00.1 Ethernet controller: Mellanox Technologies MT27710 Family
> [ConnectX-4 Lx Virtual Function]
>
> 01:00.2 Ethernet controller: Mellanox Technologies MT27710 Family
> [ConnectX-4 Lx Virtual Function]
>
>
>
> *#cat /sys/class/net/ens11/device/sriov_numvfs*
>
> 2
>
>
>
> *# echo nic_netdev > /sys/class/net/ens11/compat/devlink/uplink_rep_mode*
>
> *# echo 0000:01:00.1 > /sys/bus/pci/drivers/mlx5_core/unbind*
>
> *# echo 0000:01:00.2 > /sys/bus/pci/drivers/mlx5_core/unbind*
>
>
>
> *# echo switchdev > /sys/class/net/ens11/compat/devlink/mode*
>
>
>
> *# systemctl start openvswitch*
>
> *# ovs-vsctl add-br ovs-sriov*
>
> *# ovs-vsctl set Open_vSwitch . other_config:hw-offload=true*
>
> *# systemctl restart openvswitch*
>
> *# ovs-vsctl add-port ovs-sriov ens11*
>
> *# ovs-vsctl add-port ovs-sriov eth0*
>
> *# ovs-vsctl add-port ovs-sriov eth1*
>
>
>
> *Traffic type from one from one VM to another VM:* ICMP(command: ping),
> TCP(command: ssh)
>
>
>
> let me know if I'm missing anything.
>
>
>
> Thanks
>
> Satish
>
>
>
> On Mon, Feb 24, 2020 at 2:50 PM Roni Bar Yanai <roniba at mellanox.com>
> wrote:
>
> Hi Satish,
>
> Can you share more details of your setup. How it is connected and what is
> the traffic you run?
>
> Are you in switchdev mode?
>
> See
> https://www.mellanox.com/related-docs/prod_software/ASAP2_Hardware_Offloading_for_vSwitches_User_Manual_v4.4.pdf
>
> BR,
>
> Roni
>
> Hi Team,
>
>
>
> I'm using OVS to understand its feature and workflow and came across the
> topic of "offloading flow in hardware" through TC.
>
> I installed OVS following official guide
> http://docs.openvswitch.org/en/latest/intro/install/general/#bootstrapping
> <https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdocs.openvswitch.org%2Fen%2Flatest%2Fintro%2Finstall%2Fgeneral%2F%23bootstrapping&data=02%7C01%7Cmoshele%40mellanox.com%7Cb6420a5f97a444dabba508d7ba813f34%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C1%7C637182936596641894&sdata=AgpKBngwt9S%2FohUDyVUgXlVnNnJ3ASDjnIKk5MR2fpc%3D&reserved=0>
>
>
>
> https://docs.mellanox.com/display/MLNXOFEDv473290/OVS+Offload+Using+ASAP2+Direct#OVSOffloadUsingASAP2Direct-Overview
>
>
>
>
> I tried to add flows after enabling hw_offload and then added few basic
> flow using ovs-ofctl utilities. Flows are getting added into ovs-datapath
> but not getting offloaded into hardware.
>
>
>
> I'm using mellanox card connectx4, centos 7.5, ovs 2.1.12.
>
>
>
> *Please let me know whether am I missing something? OR this could be a
> bug? Below are the various command output for the references, let me know
> for anything else. *
>
>
>
> *ovs-vsctl set Open_vSwitch . other_config:hw-offload=true*
>
>
>
> *root at satish:~# ovs-vsctl show*
> 966880f9-7702-44b9-be38-3dfc2d3cfb44
> Bridge test-br
> Port "enp0s25"
> Interface "enp0s25"
> Port test-br
> Interface test-br
> type: internal
> ovs_version: "2.12.0"
>
>
>
> *root at satish:~# ovs-ofctl dump-flows test-br*
> cookie=0x0, duration=26540.500s, table=0, n_packets=0, n_bytes=0,
> in_port=1 actions=output:enp0s25
> cookie=0x0, duration=31399.514s, table=0, n_packets=18537,
> n_bytes=2420553, priority=0 actions=NORMAL
>
>
>
> *root at satish:~# tc filter show dev test-br ingress*
> root at satish:~#
> *root at satish:~# tc filter show dev enp0s25 ingress*
>
>
>
> *satish at satish:~$ sudo tail -f
> /usr/local/var/log/openvswitch/ovs-vswitchd.log*
>
> *..................*
>
> 2020-02-20T05:31:20.155Z|00255|dpif_netlink(handler6)|ERR|Dropped 5 log
> messages in last 7 seconds (most recently, 3 seconds ago) due to excessive
> rate
> 2020-02-20T05:31:20.155Z|00256|dpif_netlink(handler6)|ERR|failed to
> offload flow: Invalid argument: enp0s25
> 2020-02-20T05:31:30.516Z|00257|dpif_netlink(handler6)|ERR|Dropped 3 log
> messages in last 8 seconds (most recently, 1 seconds ago) due to excessive
> rate
> *2020-02-20T05:31:30.516Z|00258|dpif_netlink(handler6)|ERR|failed to
> offload flow: Invalid argument: enp0s25*
>
>
>
>
>
> Thanks
>
> Satish
>
> _______________________________________________
> discuss mailing list
> discuss at openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
> <https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.openvswitch.org%2Fmailman%2Flistinfo%2Fovs-discuss&data=02%7C01%7Cmoshele%40mellanox.com%7Cb6420a5f97a444dabba508d7ba813f34%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C1%7C637182936596651885&sdata=nwf63laNCKFyJTsWOO9vf82WLxg4fievqlAanqj5dns%3D&reserved=0>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20200226/c9f5f711/attachment-0001.html>
More information about the discuss
mailing list