[ovs-discuss] TC flow offload using OVS - failed to offload flow: Invalid argument: enp0s25

Roni Bar Yanai roniba at mellanox.com
Wed Feb 26 07:40:54 UTC 2020


Hi,

eth1 is it the port representor of VF1?
eth0 is the up-link representor?

Can you share dpif/dump-flows so we can see what are the flows
you are trying to offload? Not all flows are supported.
Is traffic running? Ping replay and ssh?

Thanks.

From: satish dhote <sdhote926 at gmail.com>
Sent: Wednesday, February 26, 2020 8:39 AM
To: Moshe Levi <moshele at mellanox.com>
Cc: Roni Bar Yanai <roniba at mellanox.com>; ovs-discuss at openvswitch.org
Subject: Re: [ovs-discuss] TC flow offload using OVS - failed to offload flow: Invalid argument: enp0s25

Hi Moshe,

Sorry for the confusion. My recent response has new setup details, so please ignore the instructions and setup details which I sent as part of very first email.

Here is the fresh setup details and logs, and yes, ens11 is a physical link.

Openvswitch version - "2.12.0"

Host machine -
OS - Centos7.5
mellanox card - ConnectX-4 Lx
Physical interface ens11

Virtual machines -
2 VM's
OS - Minimal Centos7.5
Attached each VF to each virtual machine

Steps followed to enable sriov and configuring ovs offload
# lspci | grep -i mell
01:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx]

# ethtool -i ens11 | head -5
driver: mlx5_core
version: 4.7-3.2.9
firmware-version: 14.26.4012 (MT_2410110034)
expansion-rom-version:
bus-info: 0000:01:00.0

# cat /sys/class/net/ens11/device/sriov_totalvfs
4

# echo 2 > /sys/class/net/ens11/device/sriov_numvfs

# lspci | grep -i mell
01:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx]
01:00.1 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function]
01:00.2 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function]

#cat /sys/class/net/ens11/device/sriov_numvfs
2

# echo nic_netdev > /sys/class/net/ens11/compat/devlink/uplink_rep_mode
# echo 0000:01:00.1 > /sys/bus/pci/drivers/mlx5_core/unbind
# echo 0000:01:00.2 > /sys/bus/pci/drivers/mlx5_core/unbind

# echo switchdev > /sys/class/net/ens11/compat/devlink/mode

# systemctl start openvswitch
# ovs-vsctl add-br ovs-sriov
# ovs-vsctl set Open_vSwitch . other_config:hw-offload=true
# systemctl restart openvswitch
# ovs-vsctl add-port ovs-sriov ens11
# ovs-vsctl add-port ovs-sriov eth0
# ovs-vsctl add-port ovs-sriov eth1

# ovs-vsctl show
8bcbcf4b-e1df-4e79-b218-075d9d77568b
   Bridge ovs-sriov
       Port "ens11"
           Interface "ens11"
       Port "eth1"
           Interface "eth1"
       Port ovs-sriov
           Interface ovs-sriov
               type: internal
       Port "eth0"
           Interface "eth0"
Traffic type from one from one VM to another VM:
ICMP(command: ping), TCP(command: ssh)

Logs:
dpif_netlink(handler6)|ERR|Dropped 5 log messages in last 7 seconds (most recently, 3 seconds ago) due to excessive rate
dpif_netlink(handler6)|ERR|failed to offload flow: Invalid argument: ens11
dpif_netlink(handler6)|ERR|Dropped 3 log messages in last 8 seconds (most recently, 1 seconds ago) due to excessive rate
dpif_netlink(handler6)|ERR|failed to offload flow: Invalid argument: eth1


Thanks
Satish


On Wed, Feb 26, 2020 at 11:38 AM Moshe Levi <moshele at mellanox.com<mailto:moshele at mellanox.com>> wrote:
You uplink (or physical link)  is ens11 so why did you add enp0s25 to the bridge?

From: discuss <ovs-discuss-bounces at openvswitch.org<mailto:ovs-discuss-bounces at openvswitch.org>> On Behalf Of satish dhote
Sent: Wednesday, February 26, 2020 8:01 AM
To: Roni Bar Yanai <roniba at mellanox.com<mailto:roniba at mellanox.com>>
Cc: ovs-discuss at openvswitch.org<mailto:ovs-discuss at openvswitch.org>
Subject: Re: [ovs-discuss] TC flow offload using OVS - failed to offload flow: Invalid argument: enp0s25

Hi Roni,

Thanks for your response.
Below are the setup details.

Openvswitch version - "2.12.0"

Host machine -
OS - Centos7.5
mellanox card - ConnectX-4 Lx
Physical interface ens11

Virtual machines -
2 VM's
OS - Minimal Centos7.5
Attached each VF to each virtual machine

Steps followed to enable sriov and configuring ovs offload
# lspci | grep -i mell
01:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx]

# ethtool -i ens11 | head -5
driver: mlx5_core
version: 4.7-3.2.9
firmware-version: 14.26.4012 (MT_2410110034)
expansion-rom-version:
bus-info: 0000:01:00.0

# cat /sys/class/net/ens11/device/sriov_totalvfs
4

# echo 2 > /sys/class/net/ens11/device/sriov_numvfs

# lspci | grep -i mell
01:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx]
01:00.1 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function]
01:00.2 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function]

#cat /sys/class/net/ens11/device/sriov_numvfs
2

# echo nic_netdev > /sys/class/net/ens11/compat/devlink/uplink_rep_mode
# echo 0000:01:00.1 > /sys/bus/pci/drivers/mlx5_core/unbind
# echo 0000:01:00.2 > /sys/bus/pci/drivers/mlx5_core/unbind

# echo switchdev > /sys/class/net/ens11/compat/devlink/mode

# systemctl start openvswitch
# ovs-vsctl add-br ovs-sriov
# ovs-vsctl set Open_vSwitch . other_config:hw-offload=true
# systemctl restart openvswitch
# ovs-vsctl add-port ovs-sriov ens11
# ovs-vsctl add-port ovs-sriov eth0
# ovs-vsctl add-port ovs-sriov eth1

Traffic type from one from one VM to another VM: ICMP(command: ping), TCP(command: ssh)

let me know if I'm missing anything.

Thanks
Satish

On Mon, Feb 24, 2020 at 2:50 PM Roni Bar Yanai <roniba at mellanox.com<mailto:roniba at mellanox.com>> wrote:
Hi Satish,
Can you share more details of your setup. How it is connected and what is the traffic you run?
Are you in switchdev mode?
See https://www.mellanox.com/related-docs/prod_software/ASAP2_Hardware_Offloading_for_vSwitches_User_Manual_v4.4.pdf
BR,
Roni
Hi Team,

I'm using OVS to understand its feature and workflow and came across the topic of "offloading flow in hardware" through TC.
I installed OVS following official guide http://docs.openvswitch.org/en/latest/intro/install/general/#bootstrapping<https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdocs.openvswitch.org%2Fen%2Flatest%2Fintro%2Finstall%2Fgeneral%2F%23bootstrapping&data=02%7C01%7Croniba%40mellanox.com%7Ce65cb87e91eb48e75d0608d7ba86a899%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C1%7C637182959842403245&sdata=0f9ONwrE1PZpzXEbbC1oBE5rig1gHqiL50IC8%2BH55o0%3D&reserved=0>
https://docs.mellanox.com/display/MLNXOFEDv473290/OVS+Offload+Using+ASAP2+Direct#OVSOffloadUsingASAP2Direct-Overview

I tried to add flows after enabling hw_offload and then added few basic flow using ovs-ofctl utilities. Flows are getting added into ovs-datapath but not getting offloaded into hardware.

I'm using mellanox card connectx4, centos 7.5, ovs 2.1.12.

Please let me know whether am I missing something? OR this could be a bug? Below are the various command output for the references, let me know for anything else.

ovs-vsctl set Open_vSwitch . other_config:hw-offload=true

root at satish:~# ovs-vsctl show
966880f9-7702-44b9-be38-3dfc2d3cfb44
    Bridge test-br
        Port "enp0s25"
            Interface "enp0s25"
        Port test-br
            Interface test-br
                type: internal
    ovs_version: "2.12.0"

root at satish:~# ovs-ofctl dump-flows test-br
 cookie=0x0, duration=26540.500s, table=0, n_packets=0, n_bytes=0, in_port=1 actions=output:enp0s25
 cookie=0x0, duration=31399.514s, table=0, n_packets=18537, n_bytes=2420553, priority=0 actions=NORMAL

root at satish:~# tc filter show dev test-br ingress
root at satish:~#
root at satish:~# tc filter show dev enp0s25 ingress

satish at satish:~$ sudo tail -f /usr/local/var/log/openvswitch/ovs-vswitchd.log
..................
2020-02-20T05:31:20.155Z|00255|dpif_netlink(handler6)|ERR|Dropped 5 log messages in last 7 seconds (most recently, 3 seconds ago) due to excessive rate
2020-02-20T05:31:20.155Z|00256|dpif_netlink(handler6)|ERR|failed to offload flow: Invalid argument: enp0s25
2020-02-20T05:31:30.516Z|00257|dpif_netlink(handler6)|ERR|Dropped 3 log messages in last 8 seconds (most recently, 1 seconds ago) due to excessive rate
2020-02-20T05:31:30.516Z|00258|dpif_netlink(handler6)|ERR|failed to offload flow: Invalid argument: enp0s25


Thanks
Satish
_______________________________________________
discuss mailing list
discuss at openvswitch.org<mailto:discuss at openvswitch.org>
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.openvswitch.org%2Fmailman%2Flistinfo%2Fovs-discuss&data=02%7C01%7Croniba%40mellanox.com%7Ce65cb87e91eb48e75d0608d7ba86a899%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C1%7C637182959842403245&sdata=Z6asuaAFysunoF42WlCzV0LpCV2vDcvgSdZaIiwg%2BUk%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20200226/9d344299/attachment-0001.html>


More information about the discuss mailing list