[ovs-discuss] incoming packets truncated on I225 interface in OVS-DPDK
Riccardo Ravaioli
riccardoravaioli at gmail.com
Thu Apr 15 13:31:10 UTC 2021
Hi all,
I noticed something weird when using an I225 interface in OVS-DPDK:
outgoing packets are processed normally, while incoming packets get
truncated. In particular, starting from a certain IP packet length, the
last 4 Bytes are always truncated.
For instance here are an ICMP and a UDP packet that got truncated:
08:36:06.482278 IP truncated-ip - 4 bytes missing! (tos 0x0, ttl 64, id
28665, offset 0, flags [DF], proto ICMP (1), length 84)
10.88.1.12 > 10.88.1.22: ICMP echo request, id 8967, seq 10, length 64
15:02:14.512439 IP truncated-ip - 4 bytes missing! (tos 0x0, ttl 64, id
36650, offset 0, flags [DF], proto UDP (17), length 75)
1.1.1.2.37599 > 1.1.1.1.9000: UDP, length 47
If I play a bit with the IP packet size, I see that:
- IP packets of up to 42 bytes are not altered,
- 43-byte packets have their last byte truncated,
- 44-byte packets their last 2 bytes
- 45-bytes their last 3 bytes,
- starting from 46 bytes it's always the last 4 bytes that get truncated.
My setup is all on the same device, it includes 2 Debian VMs with a virtio
interface, exposing a TAP interface on the host (just for simplicity in
this particular test, in order to run tcpdump on the host) and two physical
interfaces, connected one to another with an ethernet cable:
[ Debian VM1, virtio] ------ [tap, OVS switch, eth1] === cable=== [eth2 in
DPDK, OVS switch, tap] ------ [virtio, Debian VM1]
eth2 is I225, the NIC corresponding to eth1 is not relevant, it might as
well be I225 in non-dpdk mode or any other NIC.
I run a ping to VM2 from VM1 and if I compare the same packet on the two
VMs it is clear that the bytes that get dropped are at the end of the IP
packet:
At the source:
08:36:06.479357 IP (tos 0x0, ttl 64, id 28665, offset 0, flags [DF], proto
ICMP (1), length 84)
10.88.1.12 > 10.88.1.22: ICMP echo request, id 8967, seq 10, length 64
0x0000: 4500 0054 6ff9 4000 4001 b3de 0a58 010c E..To. at .@....X..
0x0010: 0a58 0116 0800 cac1 2307 000a 76a9 7660 .X......#...v.v`
0x0020: 0000 0000 5750 0700 0000 0000 1011 1213 ....WP..........
0x0030: 1415 1617 1819 1a1b 1c1d 1e1f 2021 2223 .............!"#
0x0040: 2425 2627 2829 2a2b 2c2d 2e2f 3031 3233 $%&'()*+,-./0123
0x0050: 3435 3637
At the destination:
08:36:06.482278 IP truncated-ip - 4 bytes missing! (tos 0x0, ttl 64, id
28665, offset 0, flags [DF], proto ICMP (1), length 84)
10.88.1.12 > 10.88.1.22: ICMP echo request, id 8967, seq 10, length 64
0x0000: 4500 0054 6ff9 4000 4001 b3de 0a58 010c E..To. at .@....X..
0x0010: 0a58 0116 0800 cac1 2307 000a 76a9 7660 .X......#...v.v`
0x0020: 0000 0000 5750 0700 0000 0000 1011 1213 ....WP..........
0x0030: 1415 1617 1819 1a1b 1c1d 1e1f 2021 2223 .............!"#
0x0040: 2425 2627 2829 2a2b 2c2d 2e2f 3031 3233 $%&'()*+,-./0123
The MTU of the I225 physical interface in OVS-DPDK is 1600. Decreasing it
to 1500 won't fix this issue.
Here are all the values configured on this interface:
root at OVP:~# ovs-vsctl list interface 1.myswitch
_uuid : b475c7c3-aec5-44ec-9cef-869079f8badf
admin_state : up
bfd : {}
bfd_status : {}
cfm_fault : []
cfm_fault_status : []
cfm_flap_count : []
cfm_health : []
cfm_mpid : []
cfm_remote_mpids : []
cfm_remote_opstate : []
duplex : []
error : []
external_ids : {}
ifindex : 11962937
ingress_policing_burst: 0
ingress_policing_rate: 0
lacp_current : []
link_resets : 1
link_speed : []
link_state : up
lldp : {}
mac : []
mac_in_use : "00:08:a2:12:15:e6"
mtu : 1600
mtu_request : 1600
name : "1.myswitch"
ofport : 1
ofport_request : []
options : {dpdk-devargs="0000:04:00.0"}
other_config : {}
statistics : {ovs_rx_qos_drops=0, ovs_tx_failure_drops=0,
ovs_tx_invalid_hwol_drops=0, ovs_tx_mtu_exceeded_drops=0,
ovs_tx_qos_drops=0, rx_128_to_255_packets=0, rx_1_to_64_packets=0,
rx_256_to_511_packets=0, rx_512_to_1023_packets=0, rx_65_to_127_packets=0,
rx_align_errors=0, rx_broadcast_packets=0, rx_bytes=0, rx_crc_errors=0,
rx_dropped=0, rx_errors=0, rx_fragment_errors=0, rx_jabber_errors=0,
rx_length_errors=0, rx_management_dropped=0, rx_management_packets=0,
rx_mbuf_allocation_errors=0, rx_missed_errors=0, rx_oversize_errors=0,
rx_packets=0, rx_q0_errors=0, rx_undersize_errors=0,
tx_128_to_255_packets=0, tx_1_to_64_packets=0, tx_256_to_511_packets=0,
tx_512_to_1023_packets=0, tx_65_to_127_packets=0, tx_broadcast_packets=0,
tx_bytes=0, tx_dropped=0, tx_errors=0, tx_management_packets=0,
tx_multicast_packets=0, tx_packets=0}
status : {driver_name=net_igc, if_descr="DPDK 20.11.0
net_igc", if_type="6", link_speed="2.5Gbps", max_hash_mac_addrs="0",
max_mac_addrs="16", max_rx_pktlen="1618", max_rx_queues="4",
max_tx_queues="4", max_vfs="0", max_vmdq_pools="0", min_rx_bufsize="256",
numa_id="0", pci-device_id="0x15f3", pci-vendor_id="0x8086", port_no="0"}
type : dpdk
I'm running OVS 2.15.0 with DPDK 20.11.0 on linux kernel 5.4.89.
It is important to point out that this does NOT happen when the same
interface is bound to its kernel driver (igc), or is in PCI passthrough to
the VM or, most importantly, when in use by another DPDK application (a
router dataplane) compiled with the same DPDK version.
Any ideas on the root cause of this and how to fix it?
Thanks!
Best,
Riccardo Ravaioli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20210415/4a720a8b/attachment.html>
More information about the discuss
mailing list