[ovs-dev] [PATCH 3/4] system-ovn.at: Add a OVN NAT test using OVN gateway.

Guru Shetty guru at ovn.org
Wed Jul 13 16:37:54 UTC 2016


On 12 July 2016 at 17:56, Joe Stringer <joe at ovn.org> wrote:

> On 11 July 2016 at 23:35, Gurucharan Shetty <guru at ovn.org> wrote:
> > This unit test adds a basic OVN NAT test that tests north-south
> > DNAT, south-north SNAT and east-west DNAT and SNAT. It uses network
> > namespaces connected to br-int using veth pairs to act as logical
> > ports. This test does not cover multi-host scenarios, so there is
> > a gap. But userspace OVN tests do multi-host scenarios (without NAT
> > testing), so it should still be a decent coverage.
> >
> > Signed-off-by: Gurucharan Shetty <guru at ovn.org>
>
> Awesome, more users of the kernel tests:) A few comments below.
>
> > ---
> > Please note that there are a couple of unit tests around fragmentation
> > (unrelated to OVN) that can cause kernel crashes when you run OVN kernel
> > tests. So, if you intend to run these, run it via:
> > make check-kmod TESTSUITEFLAGS="-k ovn"
> > ---
> >  tests/automake.mk              |   3 +-
> >  tests/system-kmod-testsuite.at |   1 +
> >  tests/system-ovn.at            | 141
> +++++++++++++++++++++++++++++++++++++++++
> >  3 files changed, 144 insertions(+), 1 deletion(-)
> >  create mode 100644 tests/system-ovn.at
> >
> > diff --git a/tests/automake.mk b/tests/automake.mk
> > index bdf6828..0a4e9e6 100644
> > --- a/tests/automake.mk
> > +++ b/tests/automake.mk
> > @@ -106,7 +106,8 @@ SYSTEM_USERSPACE_TESTSUITE_AT = \
> >
> >  SYSTEM_TESTSUITE_AT = \
> >         tests/system-common-macros.at \
> > -       tests/system-traffic.at
> > +       tests/system-traffic.at \
> > +       tests/system-ovn.at
> >
> >  TESTSUITE = $(srcdir)/tests/testsuite
> >  TESTSUITE_PATCH = $(srcdir)/tests/testsuite.patch
> > diff --git a/tests/system-kmod-testsuite.at b/tests/
> system-kmod-testsuite.at
> > index fc71a48..bdf57c8 100644
> > --- a/tests/system-kmod-testsuite.at
> > +++ b/tests/system-kmod-testsuite.at
> > @@ -23,3 +23,4 @@ m4_include([tests/system-common-macros.at])
> >  m4_include([tests/system-kmod-macros.at])
> >
> >  m4_include([tests/system-traffic.at])
> > +m4_include([tests/system-ovn.at])
> >
> > diff --git a/tests/system-ovn.at b/tests/system-ovn.at
> > new file mode 100644
> > index 0000000..b58a5b7
> > --- /dev/null
> > +++ b/tests/system-ovn.at
> > @@ -0,0 +1,141 @@
> > +AT_SETUP([ovn -- 2 LRs connected via LS, gateway router, NAT])
>
> In most of the existing tests, we also put CHECK_CONNTRACK() to ensure
> that modules are loaded, and to skip the tests where conntrack is
> unsupported (eg, check-system-userspace target).


Makes sense.


> Currently this patch
> doesn't add system-ovn.at to the check-system-userspace target, but
> perhaps it should (even if all tests would be skipped currently due to
> a lack of NAT support).
>
 I will do that.


>
> > +AT_KEYWORDS([ovnnat])
> > +ovn_start
> > +
> > +OVS_TRAFFIC_VSWITCHD_START()
> > +ADD_BR([br-int])
> > +
> > +# Set external-ids in br-int needed for ovn-controller
> > +ovs-vsctl \
> > +        -- set Open_vSwitch . external-ids:system-id=hv1 \
> > +        -- set Open_vSwitch .
> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
> > +        -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
> > +        -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
> > +        -- set bridge br-int fail-mode=secure
> other-config:disable-in-band=true
> > +
> > +# Start ovn-controller
> > +start_daemon ovn-controller
> > +
> > +# Logical network:
> > +# Two LRs - R1 and R2 that are connected to each other via LS "join"
> > +# in 20.0.0.0/24 network. R1 has switchess foo (192.168.1.0/24) and
> > +# bar (192.168.2.0/24) connected to it. R2 has alice (172.16.1.0/24)
> connected
> > +# to it.  R2 is a gateway router on which we add NAT rules.
>

I will add a small topology diagram.


> > +
> > +ovn-nbctl create Logical_Router name=R1
> > +ovn-nbctl create Logical_Router name=R2 options:chassis=hv1
> > +
> > +ovn-nbctl ls-add foo
> > +ovn-nbctl ls-add bar
> > +ovn-nbctl ls-add alice
> > +ovn-nbctl ls-add join
> > +
> > +# Connect foo to R1
> > +ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 192.168.1.1/24
> > +ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo \
> > +    type=router options:router-port=foo addresses=\"00:00:01:01:02:03\"
> > +
> > +# Connect bar to R1
> > +ovn-nbctl lrp-add R1 bar 00:00:01:01:02:04 192.168.2.1/24
> > +ovn-nbctl lsp-add bar rp-bar -- set Logical_Switch_Port rp-bar \
> > +    type=router options:router-port=bar addresses=\"00:00:01:01:02:04\"
> > +
> > +# Connect alice to R2
> > +ovn-nbctl lrp-add R2 alice 00:00:02:01:02:03 172.16.1.1/24
> > +ovn-nbctl lsp-add alice rp-alice -- set Logical_Switch_Port rp-alice \
> > +    type=router options:router-port=alice
> addresses=\"00:00:02:01:02:03\"
> > +
> > +# Connect R1 to join
> > +ovn-nbctl lrp-add R1 R1_join 00:00:04:01:02:03 20.0.0.1/24
> > +ovn-nbctl lsp-add join r1-join -- set Logical_Switch_Port r1-join \
> > +    type=router options:router-port=R1_join
> addresses='"00:00:04:01:02:03"'
> > +
> > +# Connect R2 to join
> > +ovn-nbctl lrp-add R2 R2_join 00:00:04:01:02:04 20.0.0.2/24
> > +ovn-nbctl lsp-add join r2-join -- set Logical_Switch_Port r2-join \
> > +    type=router options:router-port=R2_join
> addresses='"00:00:04:01:02:04"'
> > +
> > +# Static routes.
> > +ovn-nbctl lr-route-add R1 172.16.1.0/24 20.0.0.2
> > +ovn-nbctl lr-route-add R2 192.168.0.0/16 20.0.0.1
> > +
> > +# Logical port 'foo1' in switch 'foo'.
> > +ADD_NAMESPACES(foo1)
> > +ADD_VETH(foo1, foo1, br-int, "192.168.1.2/24", "f0:00:00:01:02:03", \
> > +         "192.168.1.1")
> > +ovn-nbctl lsp-add foo foo1 \
> > +-- lsp-set-addresses foo1 "f0:00:00:01:02:03 192.168.1.2"
> > +
> > +# Logical port 'alice1' in switch 'alice'.
> > +ADD_NAMESPACES(alice1)
> > +ADD_VETH(alice1, alice1, br-int, "172.16.1.2/24", "f0:00:00:01:02:04",
> \
> > +         "172.16.1.1")
> > +ovn-nbctl lsp-add alice alice1 \
> > +-- lsp-set-addresses alice1 "f0:00:00:01:02:04 172.16.1.2"
> > +
> > +# Logical port 'bar1' in switch 'bar'.
> > +ADD_NAMESPACES(bar1)
> > +ADD_VETH(bar1, bar1, br-int, "192.168.2.2/24", "f0:00:00:01:02:05", \
> > +"192.168.2.1")
> > +ovn-nbctl lsp-add bar bar1 \
> > +-- lsp-set-addresses bar1 "f0:00:00:01:02:05 192.168.2.2"
> > +
> > +# Add a DNAT rule.
> > +ovn-nbctl -- --id=@nat create nat type="dnat" logical_ip=192.168.1.2 \
> > +    external_ip=30.0.0.2 -- add logical_router R2 nat @nat
> >
> > +# Add a SNAT rule
> > +ovn-nbctl -- --id=@nat create nat type="snat" logical_ip=192.168.2.2 \
> > +    external_ip=30.0.0.1 -- add logical_router R2 nat @nat
>
> OK, so for DNAT the external IP is replaced with the logical IP (ie,
> the destination is an IP that doesn't exist within the OVN network);
> then, for SNAT the logical IP is replaced with the external IP - ie,
> the OVN IP gets hidden from the external network. This wasn't
> immediately obvious to me, but it makes sense.
>
Right. 'man ovn-nb' does try to explain it, but it is always confusing.


>
> > +# wait for ovn-controller to catch up.
> > +sleep 2
>
> I suppose we don't have a mechanism to wait and ensure the
> configuration has propagated?
>
Since this is a very specific NAT test, I think we can do better here as we
know the flows to expect. So I replaced 'sleep 2' with:
OVS_WAIT_UNTIL([ovs-ofctl dump-flows br-int | grep ct\( | grep nat])



>
> > +# 'alice1' should be able to ping 'foo1' directly.
> > +NS_CHECK_EXEC([alice1], [ping -q -c 3 -i 0.3 -w 2 192.168.1.2 |
> FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +# North-South DNAT: 'alice1' should also be able to ping 'foo1' via
> 30.0.0.2
> > +NS_CHECK_EXEC([alice1], [ping -q -c 3 -i 0.3 -w 2 192.168.1.2 |
> FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
>
> Maybe you meant to use 30.0.0.2 in the ping command in the latter test
> here.
>
Yeah. That was sloppy.


>
> > +# South-North SNAT: 'bar1' pings 'alice1'. But 'alice1' receives traffic
> > +# from 30.0.0.1
> > +NS_CHECK_EXEC([bar1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.2 |
> FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +# We verify that SNAT indeed happened via 'dump-conntrack' command.
> > +AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(30.0.0.1) | \
> > +sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
> >
> +icmp,orig=(src=192.168.2.2,dst=172.16.1.2,id=<cleared>),reply=(src=172.16.1.2,dst=30.0.0.1,id=<cleared>),zone=<cleared>
> > +])
>
> I realise you're filtering for 30.0.0.1 here, but should we also be
> able to find the connections from alice in the conntrack table?
>

I suppose you meant that I should also verify connections for the previous
DNAT test. I suppose one could argue that it wouldn't have worked without
conntrack working as there are no 30.0.0.2 as real IPs assigned. But it is
easy enough to add the verification, so I will add it.


>
> > +# Add static routes to handle east-west NAT.
> > +ovn-nbctl lr-route-add R1 30.0.0.0/24 20.0.0.2
> > +
> > +# East-west DNAT and SNAT: 'bar1' pings 30.0.0.2. 'foo1' receives it.
> > +NS_CHECK_EXEC([bar1], [ping -q -c 3 -i 0.3 -w 2 30.0.0.2 |
> FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
>
> Why does foo1 receive it and how are you proving that? (something
> responds, and as far as I follow, foo1 should be that something, but
> it's not clear that this is proven here)
>
That is fair (though I don't see how it would work without correct DNAT and
SNATting). I will add the connection tracking verification.



>
>
> > +OVS_APP_EXIT_AND_WAIT([ovn-controller])
>
> If one of the checks above fails, this is not hit. Are the OVN
> components cleaned up in that case? (I guess the answer is yes, though
> I'm not sure exactly why)
>
The ovn_start macro uses 'start_daemon' to start all daemons.
'start_daemon' in turn has a:
on_exit "kill $pid"



>
> Thanks,
> Joe
>



More information about the dev mailing list