[ovs-discuss] OF rules debuging
blp at ovn.org
Tue Jul 23 16:19:45 UTC 2019
On Tue, Jul 23, 2019 at 01:43:54PM +0300, bogun.dmitriy at gmail.com wrote:
> Is there any analogue of '-j TRACE` from iptables for OVS? I.e. what is the
> correct way to debug complex OF rules setup?
Usually I reach for "ofproto/trace" (see ovs-vswitchd(8)) first. The
FAQ has a more complete answer:
Q: I have a sophisticated network setup involving Open vSwitch, VMs or multiple
hosts, and other components. The behavior isn't what I expect. Help!
A: To debug network behavior problems, trace the path of a packet,
hop-by-hop, from its origin in one host to a remote host. If that's
correct, then trace the path of the response packet back to the origin.
The open source tool called ``plotnetcfg`` can help to understand the
relationship between the networking devices on a single host.
Usually a simple ICMP echo request and reply (``ping``) packet is good
enough. Start by initiating an ongoing ``ping`` from the origin host to a
remote host. If you are tracking down a connectivity problem, the "ping"
will not display any successful output, but packets are still being sent.
(In this case the packets being sent are likely ARP rather than ICMP.)
Tools available for tracing include the following:
- ``tcpdump`` and ``wireshark`` for observing hops across network devices,
such as Open vSwitch internal devices and physical wires.
- ``ovs-appctl dpif/dump-flows <br>`` in Open vSwitch 1.10 and later or
``ovs-dpctl dump-flows <br>`` in earlier versions. These tools allow one
to observe the actions being taken on packets in ongoing flows.
See ovs-vswitchd(8) for ``ovs-appctl dpif/dump-flows`` documentation,
ovs-dpctl(8) for ``ovs-dpctl dump-flows`` documentation, and "Why are
there so many different ways to dump flows?" above for some background.
- ``ovs-appctl ofproto/trace`` to observe the logic behind how ovs-vswitchd
treats packets. See ovs-vswitchd(8) for documentation. You can out more
details about a given flow that ``ovs-dpctl dump-flows`` displays, by
cutting and pasting a flow from the output into an ``ovs-appctl
- SPAN, RSPAN, and ERSPAN features of physical switches, to observe what
goes on at these physical hops.
Starting at the origin of a given packet, observe the packet at each hop in
turn. For example, in one plausible scenario, you might:
1. ``tcpdump`` the ``eth`` interface through which an ARP egresses a VM,
from inside the VM.
2. ``tcpdump`` the ``vif`` or ``tap`` interface through which the ARP
ingresses the host machine.
3. Use ``ovs-dpctl dump-flows`` to spot the ARP flow and observe the host
interface through which the ARP egresses the physical machine. You may
need to use ``ovs-dpctl show`` to interpret the port numbers. If the
output seems surprising, you can use ``ovs-appctl ofproto/trace`` to
observe details of how ovs-vswitchd determined the actions in the
``ovs-dpctl dump-flows`` output.
4. ``tcpdump`` the ``eth`` interface through which the ARP egresses the
5. ``tcpdump`` the ``eth`` interface through which the ARP ingresses the
physical machine, at the remote host that receives the ARP.
6. Use ``ovs-dpctl dump-flows`` to spot the ARP flow on the remote host
remote host that receives the ARP and observe the VM ``vif`` or ``tap``
interface to which the flow is directed. Again, ``ovs-dpctl show`` and
``ovs-appctl ofproto/trace`` might help.
7. ``tcpdump`` the ``vif`` or ``tap`` interface to which the ARP is
8. ``tcpdump`` the ``eth`` interface through which the ARP ingresses a VM,
from inside the VM.
It is likely that during one of these steps you will figure out the
problem. If not, then follow the ARP reply back to the origin, in reverse.
More information about the discuss