[ovs-dev] [PATCH ovn v6 2/4] northd, utils: support for RouteTables in LRs

Vladislav Odintsov odivlad at gmail.com
Fri Oct 15 09:36:41 UTC 2021



Regards,
Vladislav Odintsov

> On 15 Oct 2021, at 08:42, Han Zhou <hzhou at ovn.org> wrote:
> 
> On Thu, Oct 14, 2021 at 12:58 AM Vladislav Odintsov <odivlad at gmail.com>
> wrote:
>> 
>> Hi Han,
>> 
>> Thanks for the review.
>> 
>> Regards,
>> Vladislav Odintsov
>> 
>> On 14 Oct 2021, at 08:13, Han Zhou <hzhou at ovn.org> wrote:
>> 
>> 
>> 
>> On Tue, Oct 5, 2021 at 1:26 PM Vladislav Odintsov <odivlad at gmail.com>
> wrote:
>>>> 
>>>> This patch extends Logical Router's routing functionality.
>>>> Now user may create multiple routing tables within a Logical Router
>>>> and assign them to Logical Router Ports.
>>>> 
>>>> Traffic coming from Logical Router Port with assigned route_table
>>>> is checked against global routes if any (Logical_Router_Static_Routes
>>>> whith empty route_table field), next against directly connected routes
>>> 
>>> This is not accurate. The "directly connected routes" is NOT after the
>>> global routes. Their priority only depends on the prefix length.
>> 
>>>> and then Logical_Router_Static_Routes with same route_table value as
>>>> in Logical_Router_Port options:route_table field.
>>>> 
>>>> A new Logical Router ingress table #10 is added - IN_IP_ROUTING_PRE.
>>>> In this table packets which come from LRPs with configured
>>>> options:route_table field are checked against inport and in OVS
>>>> register 7 unique non-zero value identifying route table is written.
>>>> 
>>>> Then in 11th table IN_IP_ROUTING routes which have non-empty
>>>> `route_table` field are added with additional match on reg7 value
>>>> associated with appropriate route_table.
>>>> 
>>> 
>>> Hi Vladislav,
>>> 
>>> First of all, sorry for the delayed review, and thanks for implementing
>>> this new feature.
>>> 
>>> I have some questions regarding the feature itself. I remember that we
>>> had some discussion earlier for this feature, but it seems I misunderstood
>>> the feature you are implementing here. I thought by multiple routing tables
>>> you were trying to support something like VRF in physical routers, but this
>>> seems to be something different. According to your implementation, instead
>>> of assigning LRPs to different routing tables, a LRP can actually
>>> participate to both the global routing table and the table with a specific
>>> ID. For ingress, global routes are prefered over other routes; for egress
>>> (i.e. forwarding to the next hop), it doesn't even enforce any table-id
>>> check, so packet routed by a entry of table-X can go out of a LRP with
>>> table-id Y. Is my understanding correct about the desired behavior of this
>>> feature?
>>> 
>>> 
>> Yes, your understanding is correct.
>> This is not VRF. At first glance VRF can be done just by using LR-per-VRF
>> without any code modifications (maybe there are corner cases, but it’s not
>> something I’m currently working on).
>> 
> I agree VRF can be achieved by just separate LRs. I am trying to understand
> why multiple LRs wouldn't satisfy your use case here.
> 
>> LRP can be optionally assigned to specific routing table name. This means
>> that for LR ingress pipeline packets coming from this specific LRP would be
>> checked against routes with same route_table value within appropriate LR.
>> This is some kind of PBR, analog of "ip rule add iif <interface name>
>> lookup <id>".
>> There is one specific use-case, which requires special handling:
>> directly-connected routes (subnet CIDRs from connected to this LR LRPs).
>> These routes can’t be added manually by user, though routing between LRPs
>> belonging to different routing tables is still needed. So, these routes
>> should be added to global routing table to override routing for LRPs with
>> configured routing tables. If for some reason user wants to prohibit IP
>> connectivity to any LRP (honestly, I can’t imagine, why), it can be done by
>> utilising OVN PBR with drop action or a route with "discard" nexthop. But
>> I’d think in this case whether this LRP is needed in this LR.
>> Also, if user wants to create routes, which apply for all LRPs from all
>> routing tables, it can be done by adding new entries to global routing
>> table.
>> And the third reason to have global routing table - a fully
>> backward-compatible solution. All users, who don’t need multiple routing
>> tables support just continue using static routing in the same manner
>> without any changes.
>> 
>> 
>>> If this is correct, it doesn't seem to be a common/standard requirement
>>> (or please educate me if I am wrong). Could you explain a little more about
>>> the actual use cases: what kind of real world problems need to be solved by
>>> this feature or how are you going to use this. For example, why would a
>>> port need to participate in both routing tables? It looks like what you
>>> really need is policy routing instead of multiple isolated routing tables.
>>> I understand that you already use policy routing to implement ACLs, so it
>>> is not convenient to combine other policies (e.g. inport based routing)
>>> into the policy routing stage. If that's the case, would it be more generic
>>> to support multiple policy routing stages? My concern to the current
>>> approach is that it is implemented for a very special use case. It makes
>>> the code more complex but when there is a slightly different requirement in
>>> the future it becomes insufficient. I am thinking that policy routing seems
>>> more flexible and has more potential to be made more generic. Maybe I will
>>> have a better understanding when I hear more detailed use cases and
>>> considerations from you.
>> 
>> 
>> I can't agree here in it’s uncommon requirement.
>> This implementation was inspired by AWS Route Tables feature [1]: within
>> a VPC (LR in terms of OVN) user may create multiple routing tables and
>> assign them to different subnets (LRPs) in multiple availability zones.
>> Auto-generated directly-connected routes from LRPs CIDRs are working in the
>> same manner as they do in AWS - apply to all subnets regardless of their
>> association to routing table. GCP has similar behaviour: [2], Azure, I
>> guess, too.
>> Our public cloud (CROC Cloud Platform) supports AWS behaviour [3], so I
>> primarily was oriented on it. Internally we already use this feature and
>> currently it fits our use-case, but again I can't say it is specific.
>> 
> If it is for AWS/GCP alike routing features, then from what I understand
> what's implemented in this patch is a little different. To implement a VPC
> model in OVN, I think it is ok to have multiple LRs instead of only one LR
> for each VPC. So naturally I would use a LR to map to AWS's routing table.
> In AWS's document (the link you provided), it says:
> 
> "A subnet can only be associated with one route table at a time"
> 
> So basically in AWS a subnet is either attached to the default/main route
> table or a custom table, but not both, right? However, in your use case, a
> LRP (maps to a subnet) attaches to both "main" and a custom table, which
> seems not common to me. Or did I miss something?

That’s true about AWS, but there is still a bit not accurate about OVN. Global routing table in OVN terms is not that AWS main route table is.
Main route table is just a configuration hint for users for implicit route tables association with subnets.
Implicitly-associated via main routing table subnets routing functions the same manner as a normal explicit route_table-subnet association.

Global routing table in OVN is just a list of routes with higher priority than routes with configured "route_table".

I do not offer to configure both tables at the same time. But it is _possible_ to do if required for some reason (for instance to configure some service chaining or just internal VPC services like metadata/another internal APIs, access to another services).
Normally, if we talk about AWS Route Table to OVN, it is mostly one-to-one mapping, except "Local route":

Example:
AWS Route Table rtb-xxx:
172.31.0.0/16: local route (VPC CIDR for subnets)
0.0.0.0/0: igw-XXX (internet gateway)

AWS Route Table rtb-yyy:
172.31.0.0/16: local route (VPC CIDR for subnets)
5.5.5.5/32: instance-xxx

This maps to OVN configuration (for one LR with one subnet 172.31.0.0/24):

implicit route (not present in logical router static routes table):
172.31.0.0/24: LRP-subnet-xxx - has highest priority over other route table-oriented routes and can be threat as placed in global routing table

Normal static routes:
ip_prefix: 0.0.0.0/0, nexthop: <IP for edge LR’s LRP>, route_table: rtb-xxx
ip_prefix: 5.5.5.5/32, nexthop: <IP of some LSP, which belongs to VM/container via which route is built>, route_table: rtb-yyy

I guess, I understood the reason for misunderstanding: the global routing table, which I referred earlier is a routing table which has no value in "route_table" field and directly-connected routes at the same time. Latter have no records in logical router static routes, but I still referred them as a routes from "global routing table". I can think about terminology here, if it’s a root cause for misunderstanding. What do you think?

> 
>> 
>> In my opinion having this feature to be implemented using PBR is less
>> convenient and native for users, who are familiar with behaviour for
>> mentioned above public cloud platforms, because configuring routes should
>> be done in routes section. And adding route table property seems native in
>> this route, not in PBR. Moreover, I _think_ using
>> Logical_Router_Static_Route to extend this feature for OVN-Interconnection
>> becomes quite easy comparing to PBR (though, I didn’t try the latter).
>> 
> I agree if it is just AWS -like requirement, PBR is less convenient.
> 
> I am trying to understand if it can be achieved with separate LRs. If not,
> what's special about the requirement, and is the current approach providing
> a solution common enough so that more use cases can also benefit from?
> Could you clarify a little more? Thanks again.
> 
That was our initial approach - to use separate LRs for each Route Table.
We rejected that solution because met some difficulties and blocker. See below:

Brief topology description if using LRs per Route Table:
Imagine 2 subnets in VPC in 1st AZ, one in another AZ. Each subnet in it’s own Route Table (LR).
All subnets must have IP connectivity, so we have to somehow interconnect these Route Table LRs.

[BLOCKER] It is impossible to place route in route table 1 via VM from subnet assiciated to route table 2 if using per-RTB LR. Because in RTB-2 LR we have to add route from RTB 1 and this breaks route table isolation. Route Table 2 LR will start looking up routes and there could be routes from another route tables. This breaks the idea of having LR per Route Table completely. Here we rejected this solution and moved to adding support for route tables in OVN.

But some more cons:

1. complex network topology. Interconnecting all LRs even with some transit switch is harder than having one LR and all VPC-related configuration is done in one place (in AZ).
2. Directly-connected routes. In case we have multiple Route Table LRs, we have to add route for each subnet from another LR. In case one LR per VPC all such routes are installed automatically and learnt via ovn-ic.
3. PBR, LBs. It’s much easier to implement PBR and configuring Load Balancers in one LR, than in multiple.
4. There could be very many LRs, LRPs, LSPs (datapaths in SB) - excess database records, huge database growth.
5. Extra client code complexity (updating routes required configuring routes in many LRs on different availibility zones);
5. We couldn’t use ovn-ic routes learning, because VPC requires out-of-box IP connectivity between subnets, and if connect Route Table LRs between AZs, because connecting multiple LRs from different Route Tables would learn routes without binding to route table. This requires additional isolation at the transit switches level.
6. There were some more problems. Here are listed some, which I could refresh in my mind.

From our half-of-a-year experience using LR per VPC is very comfortable and it looks quite extendable it terms of network features.

Let me know if this makes sense.

> Han
> 
>> 
>> 
>>> I haven't finished reviewing the code yet, but I have one comment
>>> regarding adding 100 to the priority of the global routes. For IPv6, the
>>> priority range from 0 to 120x2=240, so adding 100 is not enough. It would
>>> create overlapping priority ranges, and some table-id specific route
>>> entries may override the global routes.
>> 
>> 
>> Thanks, I’ll dig into this when you finish review.
>> 
>> 
>> Let me know if I answered your questions or if you have new ones.
>> Again many thanks for your time and digging into this patch series.
>> 
>> 1: https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Route_Tables.html
>> 2: https://cloud.google.com/vpc/docs/routes#subnet-routes
>> 3: https://docs.cloud.croc.ru/en/services/networks/routetables.html
>> 
>> Thanks,
>> Han
>>> 
>>>> Signed-off-by: Vladislav Odintsov <odivlad at gmail.com>
>>>> Acked-by: Numan Siddique <numans at ovn.org>
>>>> ---
>>>> northd/northd.c         | 159 ++++++++++++---
>>>> northd/ovn-northd.8.xml |  63 ++++--
>>>> ovn-nb.ovsschema        |   5 +-
>>>> ovn-nb.xml              |  30 +++
>>>> tests/ovn-ic.at         |   4 +
>>>> tests/ovn-nbctl.at      | 196 +++++++++++++++++-
>>>> tests/ovn-northd.at     |  76 ++++++-
>>>> tests/ovn.at            | 441 +++++++++++++++++++++++++++++++++++++++-
>>>> utilities/ovn-nbctl.c   | 134 +++++++++++-
>>>> 9 files changed, 1041 insertions(+), 67 deletions(-)
>>>> 
>>>> diff --git a/northd/northd.c b/northd/northd.c
>>>> index 092eca829..6a020cb2e 100644
>>>> --- a/northd/northd.c
>>>> +++ b/northd/northd.c
>>>> @@ -148,15 +148,16 @@ enum ovn_stage {
>>>>     PIPELINE_STAGE(ROUTER, IN,  ECMP_STATEFUL,   7,
>> "lr_in_ecmp_stateful") \
>>>>     PIPELINE_STAGE(ROUTER, IN,  ND_RA_OPTIONS,   8,
>> "lr_in_nd_ra_options") \
>>>>     PIPELINE_STAGE(ROUTER, IN,  ND_RA_RESPONSE,  9,
>> "lr_in_nd_ra_response") \
>>>> -    PIPELINE_STAGE(ROUTER, IN,  IP_ROUTING,      10,
>> "lr_in_ip_routing")   \
>>>> -    PIPELINE_STAGE(ROUTER, IN,  IP_ROUTING_ECMP, 11,
>> "lr_in_ip_routing_ecmp") \
>>>> -    PIPELINE_STAGE(ROUTER, IN,  POLICY,          12, "lr_in_policy")
>>    \
>>>> -    PIPELINE_STAGE(ROUTER, IN,  POLICY_ECMP,     13,
>> "lr_in_policy_ecmp")  \
>>>> -    PIPELINE_STAGE(ROUTER, IN,  ARP_RESOLVE,     14,
>> "lr_in_arp_resolve")  \
>>>> -    PIPELINE_STAGE(ROUTER, IN,  CHK_PKT_LEN   ,  15,
>> "lr_in_chk_pkt_len")  \
>>>> -    PIPELINE_STAGE(ROUTER, IN,  LARGER_PKTS,     16,
>> "lr_in_larger_pkts")  \
>>>> -    PIPELINE_STAGE(ROUTER, IN,  GW_REDIRECT,     17,
>> "lr_in_gw_redirect")  \
>>>> -    PIPELINE_STAGE(ROUTER, IN,  ARP_REQUEST,     18,
>> "lr_in_arp_request")  \
>>>> +    PIPELINE_STAGE(ROUTER, IN,  IP_ROUTING_PRE,  10,
>> "lr_in_ip_routing_pre")  \
>>>> +    PIPELINE_STAGE(ROUTER, IN,  IP_ROUTING,      11,
>> "lr_in_ip_routing")      \
>>>> +    PIPELINE_STAGE(ROUTER, IN,  IP_ROUTING_ECMP, 12,
>> "lr_in_ip_routing_ecmp") \
>>>> +    PIPELINE_STAGE(ROUTER, IN,  POLICY,          13, "lr_in_policy")
>>       \
>>>> +    PIPELINE_STAGE(ROUTER, IN,  POLICY_ECMP,     14,
>> "lr_in_policy_ecmp")     \
>>>> +    PIPELINE_STAGE(ROUTER, IN,  ARP_RESOLVE,     15,
>> "lr_in_arp_resolve")     \
>>>> +    PIPELINE_STAGE(ROUTER, IN,  CHK_PKT_LEN,     16,
>> "lr_in_chk_pkt_len")     \
>>>> +    PIPELINE_STAGE(ROUTER, IN,  LARGER_PKTS,     17,
>> "lr_in_larger_pkts")     \
>>>> +    PIPELINE_STAGE(ROUTER, IN,  GW_REDIRECT,     18,
>> "lr_in_gw_redirect")     \
>>>> +    PIPELINE_STAGE(ROUTER, IN,  ARP_REQUEST,     19,
>> "lr_in_arp_request")     \
>>>>                                                                       \
>>>>     /* Logical router egress stages. */                               \
>>>>     PIPELINE_STAGE(ROUTER, OUT, UNDNAT,      0, "lr_out_undnat")
>> \
>>>> @@ -225,6 +226,7 @@ enum ovn_stage {
>>>> #define REG_NEXT_HOP_IPV6 "xxreg0"
>>>> #define REG_SRC_IPV4 "reg1"
>>>> #define REG_SRC_IPV6 "xxreg1"
>>>> +#define REG_ROUTE_TABLE_ID "reg7"
>>>> 
>>>> #define REG_ORIG_TP_DPORT_ROUTER   "reg9[16..31]"
>>>> 
>>>> @@ -287,8 +289,9 @@ enum ovn_stage {
>>>>  * | R6  |        UNUSED            | X |                 | G |
>> IN_IP_ROUTING)|
>>>>  * |     |                          | R |                 | 1 |
>>        |
>>>>  * +-----+--------------------------+ E |     UNUSED      |   |
>>        |
>>>> - * | R7  |        UNUSED            | G |                 |   |
>>        |
>>>> - * |     |                          | 3 |                 |   |
>>        |
>>>> + * | R7  |      ROUTE_TABLE_ID      | G |                 |   |
>>        |
>>>> + * |     | (>= IN_IP_ROUTING_PRE && | 3 |                 |   |
>>        |
>>>> + * |     |  <= IN_IP_ROUTING)       |   |                 |   |
>>        |
>>>>  *
>> +-----+--------------------------+---+-----------------+---+---------------+
>>>>  * | R8  |     ECMP_GROUP_ID        |   |                 |
>>>>  * |     |     ECMP_MEMBER_ID       | X |                 |
>>>> @@ -8511,11 +8514,72 @@ cleanup:
>>>>     ds_destroy(&actions);
>>>> }
>>>> 
>>>> +static uint32_t
>>>> +route_table_add(struct simap *route_tables, const char
>> *route_table_name)
>>>> +{
>>>> +    /* route table ids start from 1 */
>>>> +    uint32_t rtb_id = simap_count(route_tables) + 1;
>>>> +
>>>> +    if (rtb_id == UINT16_MAX) {
>>>> +        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
>>>> +        VLOG_WARN_RL(&rl, "too many route tables for Logical Router.");
>>>> +        return 0;
>>>> +    }
>>>> +
>>>> +    if (!simap_put(route_tables, route_table_name, rtb_id)) {
>>>> +        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
>>>> +        VLOG_WARN_RL(&rl, "Route table id unexpectedly appeared");
>>>> +    }
>>>> +
>>>> +    return rtb_id;
>>>> +}
>>>> +
>>>> +static uint32_t
>>>> +get_route_table_id(struct simap *route_tables, const char
>> *route_table_name)
>>>> +{
>>>> +    if (!route_table_name || !strlen(route_table_name)) {
>>>> +        return 0;
>>>> +    }
>>>> +
>>>> +    uint32_t rtb_id = simap_get(route_tables, route_table_name);
>>>> +    if (!rtb_id) {
>>>> +        rtb_id = route_table_add(route_tables, route_table_name);
>>>> +    }
>>>> +
>>>> +    return rtb_id;
>>>> +}
>>>> +
>>>> +static void
>>>> +build_route_table_lflow(struct ovn_datapath *od, struct hmap *lflows,
>>>> +                        struct nbrec_logical_router_port *lrp,
>>>> +                        struct simap *route_tables)
>>>> +{
>>>> +    struct ds match = DS_EMPTY_INITIALIZER;
>>>> +    struct ds actions = DS_EMPTY_INITIALIZER;
>>>> +
>>>> +    const char *route_table_name = smap_get(&lrp->options,
>> "route_table");
>>>> +    uint32_t rtb_id = get_route_table_id(route_tables,
>> route_table_name);
>>>> +    if (!rtb_id) {
>>>> +        return;
>>>> +    }
>>>> +
>>>> +    ds_put_format(&match, "inport == \"%s\"", lrp->name);
>>>> +    ds_put_format(&actions, "%s = %d; next;",
>>>> +                  REG_ROUTE_TABLE_ID, rtb_id);
>>>> +
>>>> +    ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING_PRE, 100,
>>>> +                  ds_cstr(&match), ds_cstr(&actions));
>>>> +
>>>> +    ds_destroy(&match);
>>>> +    ds_destroy(&actions);
>>>> +}
>>>> +
>>>> struct parsed_route {
>>>>     struct ovs_list list_node;
>>>>     struct in6_addr prefix;
>>>>     unsigned int plen;
>>>>     bool is_src_route;
>>>> +    uint32_t route_table_id;
>>>>     uint32_t hash;
>>>>     const struct nbrec_logical_router_static_route *route;
>>>>     bool ecmp_symmetric_reply;
>>>> @@ -8540,7 +8604,7 @@ find_static_route_outport(struct ovn_datapath
>> *od, struct hmap *ports,
>>>>  * Otherwise return NULL. */
>>>> static struct parsed_route *
>>>> parsed_routes_add(struct ovn_datapath *od, struct hmap *ports,
>>>> -                  struct ovs_list *routes,
>>>> +                  struct ovs_list *routes, struct simap *route_tables,
>>>>                   const struct nbrec_logical_router_static_route
>> *route,
>>>>                   struct hmap *bfd_connections)
>>>> {
>>>> @@ -8622,6 +8686,7 @@ parsed_routes_add(struct ovn_datapath *od, struct
>> hmap *ports,
>>>>     struct parsed_route *pr = xzalloc(sizeof *pr);
>>>>     pr->prefix = prefix;
>>>>     pr->plen = plen;
>>>> +    pr->route_table_id = get_route_table_id(route_tables,
>> route->route_table);
>>>>     pr->is_src_route = (route->policy && !strcmp(route->policy,
>>>>                                                  "src-ip"));
>>>>     pr->hash = route_hash(pr);
>>>> @@ -8655,6 +8720,7 @@ struct ecmp_groups_node {
>>>>     struct in6_addr prefix;
>>>>     unsigned int plen;
>>>>     bool is_src_route;
>>>> +    uint32_t route_table_id;
>>>>     uint16_t route_count;
>>>>     struct ovs_list route_list; /* Contains ecmp_route_list_node */
>>>> };
>>>> @@ -8663,7 +8729,7 @@ static void
>>>> ecmp_groups_add_route(struct ecmp_groups_node *group,
>>>>                       const struct parsed_route *route)
>>>> {
>>>> -   if (group->route_count == UINT16_MAX) {
>>>> +    if (group->route_count == UINT16_MAX) {
>>>>         static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
>>>>         VLOG_WARN_RL(&rl, "too many routes in a single ecmp group.");
>>>>         return;
>>>> @@ -8692,6 +8758,7 @@ ecmp_groups_add(struct hmap *ecmp_groups,
>>>>     eg->prefix = route->prefix;
>>>>     eg->plen = route->plen;
>>>>     eg->is_src_route = route->is_src_route;
>>>> +    eg->route_table_id = route->route_table_id;
>>>>     ovs_list_init(&eg->route_list);
>>>>     ecmp_groups_add_route(eg, route);
>>>> 
>>>> @@ -8705,7 +8772,8 @@ ecmp_groups_find(struct hmap *ecmp_groups, struct
>> parsed_route *route)
>>>>     HMAP_FOR_EACH_WITH_HASH (eg, hmap_node, route->hash, ecmp_groups) {
>>>>         if (ipv6_addr_equals(&eg->prefix, &route->prefix) &&
>>>>             eg->plen == route->plen &&
>>>> -            eg->is_src_route == route->is_src_route) {
>>>> +            eg->is_src_route == route->is_src_route &&
>>>> +            eg->route_table_id == route->route_table_id) {
>>>>             return eg;
>>>>         }
>>>>     }
>>>> @@ -8752,7 +8820,8 @@ unique_routes_remove(struct hmap *unique_routes,
>>>>     HMAP_FOR_EACH_WITH_HASH (ur, hmap_node, route->hash,
>> unique_routes) {
>>>>         if (ipv6_addr_equals(&route->prefix, &ur->route->prefix) &&
>>>>             route->plen == ur->route->plen &&
>>>> -            route->is_src_route == ur->route->is_src_route) {
>>>> +            route->is_src_route == ur->route->is_src_route &&
>>>> +            route->route_table_id == ur->route->route_table_id) {
>>>>             hmap_remove(unique_routes, &ur->hmap_node);
>>>>             const struct parsed_route *existed_route = ur->route;
>>>>             free(ur);
>>>> @@ -8790,9 +8859,9 @@ build_route_prefix_s(const struct in6_addr
>> *prefix, unsigned int plen)
>>>> }
>>>> 
>>>> static void
>>>> -build_route_match(const struct ovn_port *op_inport, const char
>> *network_s,
>>>> -                  int plen, bool is_src_route, bool is_ipv4, struct ds
>> *match,
>>>> -                  uint16_t *priority)
>>>> +build_route_match(const struct ovn_port *op_inport, uint32_t rtb_id,
>>>> +                  const char *network_s, int plen, bool is_src_route,
>>>> +                  bool is_ipv4, struct ds *match, uint16_t *priority)
>>>> {
>>>>     const char *dir;
>>>>     /* The priority here is calculated to implement
>> longest-prefix-match
>>>> @@ -8808,6 +8877,15 @@ build_route_match(const struct ovn_port
>> *op_inport, const char *network_s,
>>>>     if (op_inport) {
>>>>         ds_put_format(match, "inport == %s && ", op_inport->json_key);
>>>>     }
>>>> +    if (rtb_id) {
>>>> +        ds_put_format(match, "%s == %d && ", REG_ROUTE_TABLE_ID,
>> rtb_id);
>>>> +    } else {
>>>> +        /* Route-table assigned LRPs' routes should have lower priority
>>>> +         * in order not to affect directly-connected global routes.
>>>> +         * So, enlarge non-route-table routes priority by 100.
>>>> +         */
>>>> +        *priority += 100;
>>>> +    }
>>>>     ds_put_format(match, "ip%s.%s == %s/%d", is_ipv4 ? "4" : "6", dir,
>>>>                   network_s, plen);
>>>> }
>>>> @@ -8946,7 +9024,7 @@ add_ecmp_symmetric_reply_flows(struct hmap
>> *lflows,
>>>>                   out_port->lrp_networks.ea_s,
>>>>                   IN6_IS_ADDR_V4MAPPED(&route->prefix) ? "" : "xx",
>>>>                   port_ip, out_port->json_key);
>>>> -    ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, 300,
>>>> +    ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, 400,
>>>>                            ds_cstr(&match), ds_cstr(&actions),
>>>>                            &st_route->header_);
>>>> 
>>>> @@ -8976,8 +9054,8 @@ build_ecmp_route_flow(struct hmap *lflows, struct
>> ovn_datapath *od,
>>>>     struct ds route_match = DS_EMPTY_INITIALIZER;
>>>> 
>>>>     char *prefix_s = build_route_prefix_s(&eg->prefix, eg->plen);
>>>> -    build_route_match(NULL, prefix_s, eg->plen, eg->is_src_route,
>> is_ipv4,
>>>> -                      &route_match, &priority);
>>>> +    build_route_match(NULL, eg->route_table_id, prefix_s, eg->plen,
>>>> +                      eg->is_src_route, is_ipv4, &route_match,
>> &priority);
>>>>     free(prefix_s);
>>>> 
>>>>     struct ds actions = DS_EMPTY_INITIALIZER;
>>>> @@ -9052,8 +9130,8 @@ static void
>>>> add_route(struct hmap *lflows, struct ovn_datapath *od,
>>>>           const struct ovn_port *op, const char *lrp_addr_s,
>>>>           const char *network_s, int plen, const char *gateway,
>>>> -          bool is_src_route, const struct ovsdb_idl_row *stage_hint,
>>>> -          bool is_discard_route)
>>>> +          bool is_src_route, const uint32_t rtb_id,
>>>> +          const struct ovsdb_idl_row *stage_hint, bool
>> is_discard_route)
>>>> {
>>>>     bool is_ipv4 = strchr(network_s, '.') ? true : false;
>>>>     struct ds match = DS_EMPTY_INITIALIZER;
>>>> @@ -9068,8 +9146,8 @@ add_route(struct hmap *lflows, struct
>> ovn_datapath *od,
>>>>             op_inport = op;
>>>>         }
>>>>     }
>>>> -    build_route_match(op_inport, network_s, plen, is_src_route,
>> is_ipv4,
>>>> -                      &match, &priority);
>>>> +    build_route_match(op_inport, rtb_id, network_s, plen, is_src_route,
>>>> +                      is_ipv4, &match, &priority);
>>>> 
>>>>     struct ds common_actions = DS_EMPTY_INITIALIZER;
>>>>     struct ds actions = DS_EMPTY_INITIALIZER;
>>>> @@ -9132,7 +9210,8 @@ build_static_route_flow(struct hmap *lflows,
>> struct ovn_datapath *od,
>>>>     char *prefix_s = build_route_prefix_s(&route_->prefix,
>> route_->plen);
>>>>     add_route(lflows, route_->is_discard_route ? od : out_port->od,
>> out_port,
>>>>               lrp_addr_s, prefix_s, route_->plen, route->nexthop,
>>>> -              route_->is_src_route, &route->header_,
>> route_->is_discard_route);
>>>> +              route_->is_src_route, route_->route_table_id,
>> &route->header_,
>>>> +              route_->is_discard_route);
>>>> 
>>>>     free(prefix_s);
>>>> }
>>>> @@ -10584,6 +10663,17 @@ build_ND_RA_flows_for_lrouter(struct
>> ovn_datapath *od, struct hmap *lflows)
>>>>     }
>>>> }
>>>> 
>>>> +/* Logical router ingress table IP_ROUTING_PRE:
>>>> + * by default goto next. (priority 0). */
>>>> +static void
>>>> +build_ip_routing_pre_flows_for_lrouter(struct ovn_datapath *od,
>>>> +                                       struct hmap *lflows)
>>>> +{
>>>> +    if (od->nbr) {
>>>> +        ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING_PRE, 0, "1",
>> "next;");
>>>> +    }
>>>> +}
>>>> +
>>>> /* Logical router ingress table IP_ROUTING : IP Routing.
>>>>  *
>>>>  * A packet that arrives at this table is an IP packet that should be
>>>> @@ -10609,14 +10699,14 @@ build_ip_routing_flows_for_lrouter_port(
>>>>         for (int i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) {
>>>>             add_route(lflows, op->od, op,
>> op->lrp_networks.ipv4_addrs[i].addr_s,
>>>>                       op->lrp_networks.ipv4_addrs[i].network_s,
>>>> -                      op->lrp_networks.ipv4_addrs[i].plen, NULL, false,
>>>> +                      op->lrp_networks.ipv4_addrs[i].plen, NULL,
>> false, 0,
>>>>                       &op->nbrp->header_, false);
>>>>         }
>>>> 
>>>>         for (int i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) {
>>>>             add_route(lflows, op->od, op,
>> op->lrp_networks.ipv6_addrs[i].addr_s,
>>>>                       op->lrp_networks.ipv6_addrs[i].network_s,
>>>> -                      op->lrp_networks.ipv6_addrs[i].plen, NULL, false,
>>>> +                      op->lrp_networks.ipv6_addrs[i].plen, NULL,
>> false, 0,
>>>>                       &op->nbrp->header_, false);
>>>>         }
>>>>     } else if (lsp_is_router(op->nbsp)) {
>>>> @@ -10639,7 +10729,7 @@ build_ip_routing_flows_for_lrouter_port(
>>>>                     add_route(lflows, peer->od, peer,
>>>>                               peer->lrp_networks.ipv4_addrs[0].addr_s,
>>>>                               laddrs->ipv4_addrs[k].network_s,
>>>> -                              laddrs->ipv4_addrs[k].plen, NULL, false,
>>>> +                              laddrs->ipv4_addrs[k].plen, NULL, false,
>> 0,
>>>>                               &peer->nbrp->header_, false);
>>>>                 }
>>>>             }
>>>> @@ -10659,10 +10749,17 @@ build_static_route_flows_for_lrouter(
>>>>         struct hmap ecmp_groups = HMAP_INITIALIZER(&ecmp_groups);
>>>>         struct hmap unique_routes = HMAP_INITIALIZER(&unique_routes);
>>>>         struct ovs_list parsed_routes =
>> OVS_LIST_INITIALIZER(&parsed_routes);
>>>> +        struct simap route_tables = SIMAP_INITIALIZER(&route_tables);
>>>>         struct ecmp_groups_node *group;
>>>> +
>>>> +        for (int i = 0; i < od->nbr->n_ports; i++) {
>>>> +            build_route_table_lflow(od, lflows, od->nbr->ports[i],
>>>> +                                    &route_tables);
>>>> +        }
>>>> +
>>>>         for (int i = 0; i < od->nbr->n_static_routes; i++) {
>>>>             struct parsed_route *route =
>>>> -                parsed_routes_add(od, ports, &parsed_routes,
>>>> +                parsed_routes_add(od, ports, &parsed_routes,
>> &route_tables,
>>>>                                   od->nbr->static_routes[i],
>> bfd_connections);
>>>>             if (!route) {
>>>>                 continue;
>>>> @@ -10695,6 +10792,7 @@ build_static_route_flows_for_lrouter(
>>>>         ecmp_groups_destroy(&ecmp_groups);
>>>>         unique_routes_destroy(&unique_routes);
>>>>         parsed_routes_destroy(&parsed_routes);
>>>> +        simap_destroy(&route_tables);
>>>>     }
>>>> }
>>>> 
>>>> @@ -12800,6 +12898,7 @@ build_lswitch_and_lrouter_iterate_by_od(struct
>> ovn_datapath *od,
>>>>     build_neigh_learning_flows_for_lrouter(od, lsi->lflows,
>> &lsi->match,
>>>>                                            &lsi->actions,
>> lsi->meter_groups);
>>>>     build_ND_RA_flows_for_lrouter(od, lsi->lflows);
>>>> +    build_ip_routing_pre_flows_for_lrouter(od, lsi->lflows);
>>>>     build_static_route_flows_for_lrouter(od, lsi->lflows, lsi->ports,
>>>>                                          lsi->bfd_connections);
>>>>     build_mcast_lookup_flows_for_lrouter(od, lsi->lflows, &lsi->match,
>>>> diff --git a/northd/ovn-northd.8.xml b/northd/ovn-northd.8.xml
>>>> index 39f4eaa0c..cc2e25367 100644
>>>> --- a/northd/ovn-northd.8.xml
>>>> +++ b/northd/ovn-northd.8.xml
>>>> @@ -2899,7 +2899,7 @@ icmp6 {
>>>> 
>>>>     <p>
>>>>       If ECMP routes with symmetric reply are configured in the
>>>> -      <code>OVN_Northbound</code> database for a gateway router, a
>> priority-300
>>>> +      <code>OVN_Northbound</code> database for a gateway router, a
>> priority-400
>>>>       flow is added for each router port on which symmetric replies are
>>>>       configured. The matching logic for these ports essentially
>> reverses the
>>>>       configured logic of the ECMP route. So for instance, a route
>> with a
>>>> @@ -3245,7 +3245,35 @@ output;
>>>>       </li>
>>>>     </ul>
>>>> 
>>>> -    <h3>Ingress Table 10: IP Routing</h3>
>>>> +    <h3>Ingress Table 10: IP Routing Pre</h3>
>>>> +
>>>> +    <p>
>>>> +      If a packet arrived at this table from Logical Router Port
>> <var>P</var>
>>>> +      which has <code>options:route_table</code> value set, a logical
>> flow with
>>>> +      match <code>inport == "<var>P</var>"</code> with priority 100
>> and action,
>>>> +      setting unique-generated per-datapath 32-bit value (non-zero) in
>> OVS
>>>> +      register 7.  This register is checked in next table.
>>>> +    </p>
>>>> +
>>>> +    <p>
>>>> +      This table contains the following logical flows:
>>>> +    </p>
>>>> +
>>>> +    <ul>
>>>> +      <li>
>>>> +        <p>
>>>> +          Priority-100 flow with match <code>inport ==
>> "LRP_NAME"</code> value
>>>> +          and action, which set route table identifier in reg7.
>>>> +        </p>
>>>> +
>>>> +        <p>
>>>> +          A priority-0 logical flow with match <code>1</code> has
>> actions
>>>> +          <code>next;</code>.
>>>> +        </p>
>>>> +      </li>
>>>> +    </ul>
>>>> +
>>>> +    <h3>Ingress Table 11: IP Routing</h3>
>>>> 
>>>>     <p>
>>>>       A packet that arrives at this table is an IP packet that should
>> be
>>>> @@ -3316,10 +3344,10 @@ output;
>>>>         <p>
>>>>           IPv4 routing table.  For each route to IPv4 network
>> <var>N</var> with
>>>>           netmask <var>M</var>, on router port <var>P</var> with IP
>> address
>>>> -          <var>A</var> and Ethernet
>>>> -          address <var>E</var>, a logical flow with match
>> <code>ip4.dst ==
>>>> -          <var>N</var>/<var>M</var></code>, whose priority is the
>> number of
>>>> -          1-bits in <var>M</var>, has the following actions:
>>>> +          <var>A</var> and Ethernet address <var>E</var>, a logical
>> flow with
>>>> +          match <code>ip4.dst == <var>N</var>/<var>M</var></code>,
>> whose
>>>> +          priority is 100 + the number of 1-bits in <var>M</var>, has
>> the
>>>> +          following actions:
>>>>         </p>
>>>> 
>>>>         <pre>
>>>> @@ -3382,6 +3410,13 @@ next;
>>>>           If the address <var>A</var> is in the link-local scope, the
>>>>           route will be limited to sending on the ingress port.
>>>>         </p>
>>>> +
>>>> +        <p>
>>>> +          For routes with <code>route_table</code> value set
>>>> +          <code>reg7 == id</code> is prefixed in logical flow match
>> portion.
>>>> +          Priority for routes with <code>route_table</code> value set
>> is
>>>> +          the number of 1-bits in <var>M</var>.
>>>> +        </p>
>>>>       </li>
>>>> 
>>>>       <li>
>>>> @@ -3408,7 +3443,7 @@ select(reg8[16..31], <var>MID1</var>,
>> <var>MID2</var>, ...);
>>>>       </li>
>>>>     </ul>
>>>> 
>>>> -    <h3>Ingress Table 11: IP_ROUTING_ECMP</h3>
>>>> +    <h3>Ingress Table 12: IP_ROUTING_ECMP</h3>
>>>> 
>>>>     <p>
>>>>       This table implements the second part of IP routing for ECMP
>> routes
>>>> @@ -3460,7 +3495,7 @@ outport = <var>P</var>;
>>>>       </li>
>>>>     </ul>
>>>> 
>>>> -    <h3>Ingress Table 12: Router policies</h3>
>>>> +    <h3>Ingress Table 13: Router policies</h3>
>>>>     <p>
>>>>       This table adds flows for the logical router policies configured
>>>>       on the logical router. Please see the
>>>> @@ -3532,7 +3567,7 @@ next;
>>>>       </li>
>>>>     </ul>
>>>> 
>>>> -    <h3>Ingress Table 13: ECMP handling for router policies</h3>
>>>> +    <h3>Ingress Table 14: ECMP handling for router policies</h3>
>>>>     <p>
>>>>       This table handles the ECMP for the router policies configured
>>>>       with multiple nexthops.
>>>> @@ -3576,7 +3611,7 @@ outport = <var>P</var>
>>>>       </li>
>>>>     </ul>
>>>> 
>>>> -    <h3>Ingress Table 14: ARP/ND Resolution</h3>
>>>> +    <h3>Ingress Table 15: ARP/ND Resolution</h3>
>>>> 
>>>>     <p>
>>>>       Any packet that reaches this table is an IP packet whose next-hop
>>>> @@ -3767,7 +3802,7 @@ outport = <var>P</var>
>>>> 
>>>>     </ul>
>>>> 
>>>> -    <h3>Ingress Table 15: Check packet length</h3>
>>>> +    <h3>Ingress Table 16: Check packet length</h3>
>>>> 
>>>>     <p>
>>>>       For distributed logical routers or gateway routers with gateway
>>>> @@ -3797,7 +3832,7 @@ REGBIT_PKT_LARGER =
>> check_pkt_larger(<var>L</var>); next;
>>>>       and advances to the next table.
>>>>     </p>
>>>> 
>>>> -    <h3>Ingress Table 16: Handle larger packets</h3>
>>>> +    <h3>Ingress Table 17: Handle larger packets</h3>
>>>> 
>>>>     <p>
>>>>       For distributed logical routers or gateway routers with gateway
>> port
>>>> @@ -3860,7 +3895,7 @@ icmp6 {
>>>>       and advances to the next table.
>>>>     </p>
>>>> 
>>>> -    <h3>Ingress Table 17: Gateway Redirect</h3>
>>>> +    <h3>Ingress Table 18: Gateway Redirect</h3>
>>>> 
>>>>     <p>
>>>>       For distributed logical routers where one or more of the logical
>> router
>>>> @@ -3907,7 +3942,7 @@ icmp6 {
>>>>       </li>
>>>>     </ul>
>>>> 
>>>> -    <h3>Ingress Table 18: ARP Request</h3>
>>>> +    <h3>Ingress Table 19: ARP Request</h3>
>>>> 
>>>>     <p>
>>>>       In the common case where the Ethernet destination has been
>> resolved, this
>>>> diff --git a/ovn-nb.ovsschema b/ovn-nb.ovsschema
>>>> index 2ac8ef3ea..a0a171e19 100644
>>>> --- a/ovn-nb.ovsschema
>>>> +++ b/ovn-nb.ovsschema
>>>> @@ -1,7 +1,7 @@
>>>> {
>>>>     "name": "OVN_Northbound",
>>>> -    "version": "5.32.1",
>>>> -    "cksum": "2805328215 29734",
>>>> +    "version": "5.33.1",
>>>> +    "cksum": "3874993350 29785",
>>>>     "tables": {
>>>>         "NB_Global": {
>>>>             "columns": {
>>>> @@ -387,6 +387,7 @@
>>>>             "isRoot": false},
>>>>         "Logical_Router_Static_Route": {
>>>>             "columns": {
>>>> +                "route_table": {"type": "string"},
>>>>                 "ip_prefix": {"type": "string"},
>>>>                 "policy": {"type": {"key": {"type": "string",
>>>>                                             "enum": ["set", ["src-ip",
>>>> diff --git a/ovn-nb.xml b/ovn-nb.xml
>>>> index d8266ed4d..b2917c363 100644
>>>> --- a/ovn-nb.xml
>>>> +++ b/ovn-nb.xml
>>>> @@ -2772,6 +2772,14 @@
>>>>           prefix according to RFC3663
>>>>         </p>
>>>>       </column>
>>>> +
>>>> +      <column name="options" key="route_table">
>>>> +        Designates lookup Logical_Router_Static_Routes with specified
>>>> +        <code>route_table</code> value. Routes to directly connected
>> networks
>>>> +        from same Logical Router and routes without
>> <code>route_table</code>
>>>> +        option set have higher priority than routes with
>>>> +        <code>route_table</code> option set.
>>>> +      </column>
>>>>     </group>
>>>> 
>>>>     <group title="Attachment">
>>>> @@ -2891,6 +2899,28 @@
>>>>       </p>
>>>>     </column>
>>>> 
>>>> +    <column name="route_table">
>>>> +      <p>
>>>> +        Any string to place route to separate routing table. If
>> Logical Router
>>>> +        Port has configured value in <ref table="Logical_Router_Port"
>>>> +        column="options" key="route_table"/> other than empty string,
>> OVN
>>>> +        performs route lookup for all packets entering Logical Router
>> ingress
>>>> +        pipeline from this port in the following manner:
>>>> +      </p>
>>>> +
>>>> +      <ul>
>>>> +        <li>
>>>> +          1. First lookup among "global" routes: routes without
>>>> +          <code>route_table</code> value set and routes to directly
>> connected
>>>> +          networks.
>>>> +        </li>
>>>> +        <li>
>>>> +          2. Next lookup among routes with same
>> <code>route_table</code> value
>>>> +          as specified in LRP's options:route_table field.
>>>> +        </li>
>>>> +      </ul>
>>>> +    </column>
>>>> +
>>>>     <column name="external_ids" key="ic-learned-route">
>>>>       <code>ovn-ic</code> populates this key if the route is learned
>> from the
>>>>       global <ref db="OVN_IC_Southbound"/> database.  In this case the
>> value
>>>> diff --git a/tests/ovn-ic.at b/tests/ovn-ic.at
>>>> index 32f4e9d02..3aab54362 100644
>>>> --- a/tests/ovn-ic.at
>>>> +++ b/tests/ovn-ic.at
>>>> @@ -281,6 +281,7 @@ done
>>>> 
>>>> AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl
>>>> IPv4 Routes
>>>> +Route Table global:
>>>>              10.11.1.0/24               169.254.0.1 dst-ip
>>>>              10.11.2.0/24             169.254.100.2 dst-ip (learned)
>>>>              10.22.1.0/24               169.254.0.2 src-ip
>>>> @@ -299,6 +300,7 @@ ovn_as az1 ovn-nbctl set nb_global .
>> options:ic-route-learn=false
>>>> OVS_WAIT_WHILE([ovn_as az1 ovn-nbctl lr-route-list lr1 | grep learned])
>>>> AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl
>>>> IPv4 Routes
>>>> +Route Table global:
>>>>              10.11.1.0/24               169.254.0.1 dst-ip
>>>>              10.22.1.0/24               169.254.0.2 src-ip
>>>> ])
>>>> @@ -314,6 +316,7 @@ ovn_as az1 ovn-nbctl set nb_global .
>> options:ic-route-adv=false
>>>> OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned])
>>>> AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr2], [0], [dnl
>>>> IPv4 Routes
>>>> +Route Table global:
>>>>              10.11.2.0/24               169.254.0.1 dst-ip
>>>>              10.22.2.0/24               169.254.0.2 src-ip
>>>> ])
>>>> @@ -332,6 +335,7 @@ done
>>>> # Default route should NOT get advertised or learned, by default.
>>>> AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr2], [0], [dnl
>>>> IPv4 Routes
>>>> +Route Table global:
>>>>              10.11.1.0/24             169.254.100.1 dst-ip (learned)
>>>>              10.11.2.0/24               169.254.0.1 dst-ip
>>>>              10.22.2.0/24               169.254.0.2 src-ip
>>>> diff --git a/tests/ovn-nbctl.at b/tests/ovn-nbctl.at
>>>> index 9b80ae410..ddb5536ce 100644
>>>> --- a/tests/ovn-nbctl.at
>>>> +++ b/tests/ovn-nbctl.at
>>>> @@ -1520,6 +1520,7 @@ AT_CHECK([ovn-nbctl --ecmp --policy=src-ip
>> lr-route-add lr0 20.0.0.0/24 11.0.0.1
>>>> 
>>>> AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
>>>> IPv4 Routes
>>>> +Route Table global:
>>>>               10.0.0.0/24                  11.0.0.1 dst-ip
>>>>               10.0.1.0/24                  11.0.1.1 dst-ip lp0
>>>>              10.0.10.0/24                           dst-ip lp0
>>>> @@ -1534,6 +1535,7 @@ AT_CHECK([ovn-nbctl lrp-add lr0 lp1
>> f0:00:00:00:00:02 11.0.0.254/24])
>>>> AT_CHECK([ovn-nbctl --may-exist lr-route-add lr0 10.0.0.111/24
>> 11.0.0.1 lp1])
>>>> AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
>>>> IPv4 Routes
>>>> +Route Table global:
>>>>               10.0.0.0/24                  11.0.0.1 dst-ip lp1
>>>>               10.0.1.0/24                  11.0.1.1 dst-ip lp0
>>>>              10.0.10.0/24                           dst-ip lp0
>>>> @@ -1564,6 +1566,7 @@ AT_CHECK([ovn-nbctl --policy=src-ip lr-route-del
>> lr0 9.16.1.0/24])
>>>> 
>>>> AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
>>>> IPv4 Routes
>>>> +Route Table global:
>>>>               10.0.0.0/24                  11.0.0.1 dst-ip lp1
>>>>              10.0.10.0/24                           dst-ip lp0
>>>>               10.0.0.0/24                  11.0.0.2 src-ip
>>>> @@ -1575,6 +1578,7 @@ AT_CHECK([ovn-nbctl --policy=dst-ip lr-route-del
>> lr0 10.0.0.0/24])
>>>> AT_CHECK([ovn-nbctl --policy=src-ip lr-route-del lr0 10.0.0.0/24])
>>>> AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
>>>> IPv4 Routes
>>>> +Route Table global:
>>>>              10.0.10.0/24                           dst-ip lp0
>>>>                 0.0.0.0/0               192.168.0.1 dst-ip
>>>> ])
>>>> @@ -1585,6 +1589,7 @@ AT_CHECK([ovn-nbctl --policy=src-ip lr-route-add
>> lr0 10.0.0.0/24 11.0.0.2])
>>>> AT_CHECK([ovn-nbctl lr-route-del lr0 10.0.0.0/24])
>>>> AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
>>>> IPv4 Routes
>>>> +Route Table global:
>>>>              10.0.10.0/24                           dst-ip lp0
>>>>                 0.0.0.0/0               192.168.0.1 dst-ip
>>>> ])
>>>> @@ -1601,6 +1606,7 @@ AT_CHECK([ovn-nbctl --ecmp lr-route-add lr0
>> 10.0.0.0/24 11.0.0.3])
>>>> AT_CHECK([ovn-nbctl --ecmp lr-route-add lr0 10.0.0.0/24 11.0.0.4 lp0])
>>>> AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
>>>> IPv4 Routes
>>>> +Route Table global:
>>>>               10.0.0.0/24                  11.0.0.1 dst-ip ecmp
>>>>               10.0.0.0/24                  11.0.0.2 dst-ip ecmp
>>>>               10.0.0.0/24                  11.0.0.3 dst-ip ecmp
>>>> @@ -1615,6 +1621,7 @@ dnl Delete ecmp routes
>>>> AT_CHECK([ovn-nbctl lr-route-del lr0 10.0.0.0/24 11.0.0.1])
>>>> AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
>>>> IPv4 Routes
>>>> +Route Table global:
>>>>               10.0.0.0/24                  11.0.0.2 dst-ip ecmp
>>>>               10.0.0.0/24                  11.0.0.3 dst-ip ecmp
>>>>               10.0.0.0/24                  11.0.0.4 dst-ip lp0 ecmp
>>>> @@ -1622,12 +1629,14 @@ IPv4 Routes
>>>> AT_CHECK([ovn-nbctl lr-route-del lr0 10.0.0.0/24 11.0.0.2])
>>>> AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
>>>> IPv4 Routes
>>>> +Route Table global:
>>>>               10.0.0.0/24                  11.0.0.3 dst-ip ecmp
>>>>               10.0.0.0/24                  11.0.0.4 dst-ip lp0 ecmp
>>>> ])
>>>> AT_CHECK([ovn-nbctl lr-route-del lr0 10.0.0.0/24 11.0.0.4 lp0])
>>>> AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
>>>> IPv4 Routes
>>>> +Route Table global:
>>>>               10.0.0.0/24                  11.0.0.3 dst-ip
>>>> ])
>>>> AT_CHECK([ovn-nbctl lr-route-del lr0 10.0.0.0/24 11.0.0.3])
>>>> @@ -1641,6 +1650,7 @@ AT_CHECK([ovn-nbctl lr-route-add lr0
>> 2001:0db8:1::/64 2001:0db8:0:f103::1])
>>>> 
>>>> AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
>>>> IPv6 Routes
>>>> +Route Table global:
>>>>             2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
>>>>           2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
>>>>                      ::/0        2001:db8:0:f101::1 dst-ip
>>>> @@ -1650,6 +1660,7 @@ AT_CHECK([ovn-nbctl lr-route-del lr0
>> 2001:0db8:0::/64])
>>>> 
>>>> AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
>>>> IPv6 Routes
>>>> +Route Table global:
>>>>           2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
>>>>                      ::/0        2001:db8:0:f101::1 dst-ip
>>>> ])
>>>> @@ -1677,11 +1688,13 @@ AT_CHECK([ovn-nbctl --may-exist
>> --ecmp-symmetric-reply lr-route-add lr0 2003:0db
>>>> 
>>>> AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
>>>> IPv4 Routes
>>>> +Route Table global:
>>>>               10.0.0.0/24                  11.0.0.1 dst-ip
>>>>               10.0.1.0/24                  11.0.1.1 dst-ip lp0
>>>>                 0.0.0.0/0               192.168.0.1 dst-ip
>>>> 
>>>> IPv6 Routes
>>>> +Route Table global:
>>>>             2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
>>>>           2001:db8:1::/64        2001:db8:0:f103::1 dst-ip ecmp
>>>>           2001:db8:1::/64        2001:db8:0:f103::2 dst-ip ecmp
>>>> @@ -1696,7 +1709,188 @@ AT_CHECK([ovn-nbctl lrp-add lr0 lr0-p0
>> 00:00:01:01:02:03 192.168.10.1/24])
>>>> bfd_uuid=$(ovn-nbctl create bfd logical_port=lr0-p0 dst_ip=100.0.0.50
>> status=down min_tx=250 min_rx=250 detect_mult=10)
>>>> AT_CHECK([ovn-nbctl lr-route-add lr0 100.0.0.0/24 192.168.0.1])
>>>> route_uuid=$(fetch_column nb:logical_router_static_route _uuid
>> ip_prefix="100.0.0.0/24")
>>>> -AT_CHECK([ovn-nbctl set logical_router_static_route $route_uuid
>> bfd=$bfd_uuid])])
>>>> +AT_CHECK([ovn-nbctl set logical_router_static_route $route_uuid
>> bfd=$bfd_uuid])
>>>> +
>>>> +check ovn-nbctl lr-route-del lr0
>>>> +AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
>>>> +])
>>>> +
>>>> +dnl Check IPv4 routes in route table
>>>> +check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 0.0.0.0/0
>> 192.168.0.1
>>>> +check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 10.0.1.1/24
>> 11.0.1.1 lp0
>>>> +check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 10.0.0.1/24
>> 11.0.0.1
>>>> +AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
>>>> +IPv4 Routes
>>>> +Route Table rtb-1:
>>>> +              10.0.0.0/24                  11.0.0.1 dst-ip
>>>> +              10.0.1.0/24                  11.0.1.1 dst-ip lp0
>>>> +                0.0.0.0/0               192.168.0.1 dst-ip
>>>> +])
>>>> +
>>>> +check ovn-nbctl lr-route-del lr0
>>>> +AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
>>>> +])
>>>> +
>>>> +dnl Check IPv6 routes in route table
>>>> +check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 0:0:0:0:0:0:0:0/0
>> 2001:0db8:0:f101::1
>>>> +check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 2001:0db8:0::/64
>> 2001:0db8:0:f102::1 lp0
>>>> +check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 2001:0db8:1::/64
>> 2001:0db8:0:f103::1
>>>> +
>>>> +AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
>>>> +IPv6 Routes
>>>> +Route Table rtb-1:
>>>> +            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
>>>> +          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
>>>> +                     ::/0        2001:db8:0:f101::1 dst-ip
>>>> +])
>>>> +
>>>> +dnl Check IPv4 and IPv6 routes in route table
>>>> +check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 0.0.0.0/0
>> 192.168.0.1
>>>> +check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 10.0.1.1/24
>> 11.0.1.1 lp0
>>>> +check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 10.0.0.1/24
>> 11.0.0.1
>>>> +
>>>> +AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
>>>> +IPv4 Routes
>>>> +Route Table rtb-1:
>>>> +              10.0.0.0/24                  11.0.0.1 dst-ip
>>>> +              10.0.1.0/24                  11.0.1.1 dst-ip lp0
>>>> +                0.0.0.0/0               192.168.0.1 dst-ip
>>>> +
>>>> +IPv6 Routes
>>>> +Route Table rtb-1:
>>>> +            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
>>>> +          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
>>>> +                     ::/0        2001:db8:0:f101::1 dst-ip
>>>> +])
>>>> +
>>>> +# Add routes in another route table
>>>> +check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 0.0.0.0/0
>> 192.168.0.1
>>>> +check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 10.0.1.1/24
>> 11.0.1.1 lp0
>>>> +check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 10.0.0.1/24
>> 11.0.0.1
>>>> +check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 0:0:0:0:0:0:0:0/0
>> 2001:0db8:0:f101::1
>>>> +check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 2001:0db8:0::/64
>> 2001:0db8:0:f102::1 lp0
>>>> +check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 2001:0db8:1::/64
>> 2001:0db8:0:f103::1
>>>> +
>>>> +AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
>>>> +IPv4 Routes
>>>> +Route Table rtb-1:
>>>> +              10.0.0.0/24                  11.0.0.1 dst-ip
>>>> +              10.0.1.0/24                  11.0.1.1 dst-ip lp0
>>>> +                0.0.0.0/0               192.168.0.1 dst-ip
>>>> +
>>>> +Route Table rtb-2:
>>>> +              10.0.0.0/24                  11.0.0.1 dst-ip
>>>> +              10.0.1.0/24                  11.0.1.1 dst-ip lp0
>>>> +                0.0.0.0/0               192.168.0.1 dst-ip
>>>> +
>>>> +IPv6 Routes
>>>> +Route Table rtb-1:
>>>> +            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
>>>> +          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
>>>> +                     ::/0        2001:db8:0:f101::1 dst-ip
>>>> +
>>>> +Route Table rtb-2:
>>>> +            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
>>>> +          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
>>>> +                     ::/0        2001:db8:0:f101::1 dst-ip
>>>> +])
>>>> +
>>>> +# Add routes to global route table
>>>> +check ovn-nbctl lr-route-add lr0 0.0.0.0/0 192.168.0.1
>>>> +check ovn-nbctl lr-route-add lr0 10.0.1.1/24 11.0.1.1 lp0
>>>> +check ovn-nbctl lr-route-add lr0 10.0.0.1/24 11.0.0.1
>>>> +check ovn-nbctl lr-route-add lr0 0:0:0:0:0:0:0:0/0 2001:0db8:0:f101::1
>>>> +check ovn-nbctl lr-route-add lr0 2001:0db8:0::/64 2001:0db8:0:f102::1
>> lp0
>>>> +check check ovn-nbctl lr-route-add lr0 2001:0db8:1::/64
>> 2001:0db8:0:f103::1
>>>> +
>>>> +AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
>>>> +IPv4 Routes
>>>> +Route Table global:
>>>> +              10.0.0.0/24                  11.0.0.1 dst-ip
>>>> +              10.0.1.0/24                  11.0.1.1 dst-ip lp0
>>>> +                0.0.0.0/0               192.168.0.1 dst-ip
>>>> +
>>>> +Route Table rtb-1:
>>>> +              10.0.0.0/24                  11.0.0.1 dst-ip
>>>> +              10.0.1.0/24                  11.0.1.1 dst-ip lp0
>>>> +                0.0.0.0/0               192.168.0.1 dst-ip
>>>> +
>>>> +Route Table rtb-2:
>>>> +              10.0.0.0/24                  11.0.0.1 dst-ip
>>>> +              10.0.1.0/24                  11.0.1.1 dst-ip lp0
>>>> +                0.0.0.0/0               192.168.0.1 dst-ip
>>>> +
>>>> +IPv6 Routes
>>>> +Route Table global:
>>>> +            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
>>>> +          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
>>>> +                     ::/0        2001:db8:0:f101::1 dst-ip
>>>> +
>>>> +Route Table rtb-1:
>>>> +            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
>>>> +          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
>>>> +                     ::/0        2001:db8:0:f101::1 dst-ip
>>>> +
>>>> +Route Table rtb-2:
>>>> +            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
>>>> +          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
>>>> +                     ::/0        2001:db8:0:f101::1 dst-ip
>>>> +])
>>>> +
>>>> +# delete IPv4 route from rtb-1
>>>> +check ovn-nbctl --route-table=rtb-1 lr-route-del lr0 10.0.0.0/24
>>>> +AT_CHECK([ovn-nbctl --route-table=rtb-1 lr-route-list lr0], [0], [dnl
>>>> +IPv4 Routes
>>>> +Route Table rtb-1:
>>>> +              10.0.1.0/24                  11.0.1.1 dst-ip lp0
>>>> +                0.0.0.0/0               192.168.0.1 dst-ip
>>>> +
>>>> +IPv6 Routes
>>>> +Route Table rtb-1:
>>>> +            2001:db8::/64        2001:db8:0:f102::1 dst-ip lp0
>>>> +          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
>>>> +                     ::/0        2001:db8:0:f101::1 dst-ip
>>>> +])
>>>> +
>>>> +# delete IPv6 route from rtb-2
>>>> +check ovn-nbctl --route-table=rtb-2 lr-route-del lr0 2001:db8::/64
>>>> +AT_CHECK([ovn-nbctl --route-table=rtb-2 lr-route-list lr0], [0], [dnl
>>>> +IPv4 Routes
>>>> +Route Table rtb-2:
>>>> +              10.0.0.0/24                  11.0.0.1 dst-ip
>>>> +              10.0.1.0/24                  11.0.1.1 dst-ip lp0
>>>> +                0.0.0.0/0               192.168.0.1 dst-ip
>>>> +
>>>> +IPv6 Routes
>>>> +Route Table rtb-2:
>>>> +          2001:db8:1::/64        2001:db8:0:f103::1 dst-ip
>>>> +                     ::/0        2001:db8:0:f101::1 dst-ip
>>>> +])
>>>> +
>>>> +check ovn-nbctl lr-route-del lr0
>>>> +
>>>> +# ECMP route in route table
>>>> +check ovn-nbctl --route-table=rtb1 lr-route-add lr0 0.0.0.0/0
>> 192.168.0.1
>>>> +check ovn-nbctl --ecmp --route-table=rtb1 lr-route-add lr0 0.0.0.0/0
>> 192.168.0.2
>>>> +
>>>> +# Negative route table case: same prefix
>>>> +AT_CHECK([ovn-nbctl --route-table=rtb1 lr-route-add lr0 0.0.0.0/0
>> 192.168.0.1], [1], [], [dnl
>>>> +ovn-nbctl: duplicate prefix: 0.0.0.0/0 (policy: dst-ip). Use option
>> --ecmp to allow this for ECMP routing.
>>>> +])
>>>> +
>>>> +# Negative route table case: same prefix & nexthop with ecmp
>>>> +AT_CHECK([ovn-nbctl --ecmp --route-table=rtb1 lr-route-add lr0
>> 0.0.0.0/0 192.168.0.2], [1], [], [dnl
>>>> +ovn-nbctl: duplicate nexthop for the same ECMP route
>>>> +])
>>>> +
>>>> +# Add routes to global route table
>>>> +check ovn-nbctl lrp-add lr0 lrp0 00:00:00:00:00:01 1.1.1.1/24
>>>> +check ovn-nbctl lrp-set-options lrp0 route_table=rtb1
>>>> +AT_CHECK([ovn-nbctl get logical-router-port lrp0 options:route_table],
>> [0], [dnl
>>>> +rtb1
>>>> +])
>>>> +check `ovn-nbctl show lr0 | grep lrp0 -A3 | grep route_table=rtb1`
>>>> +])
>>>> 
>>>> dnl
>> ---------------------------------------------------------------------
>>>> 
>>>> diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
>>>> index 3eebb55b6..e71e65bcc 100644
>>>> --- a/tests/ovn-northd.at
>>>> +++ b/tests/ovn-northd.at
>>>> @@ -5111,7 +5111,7 @@ check ovn-nbctl --wait=sb --ecmp-symmetric-reply
>> lr-route-add lr0 1.0.0.1 192.16
>>>> 
>>>> ovn-sbctl dump-flows lr0 > lr0flows
>>>> AT_CHECK([grep -e "lr_in_ip_routing.*select" lr0flows | sed
>> 's/table=../table=??/' | sort], [0], [dnl
>>>> -  table=??(lr_in_ip_routing   ), priority=65   , match=(ip4.dst ==
>> 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1;
>> reg8[[16..31]] = select(1, 2);)
>>>> +  table=??(lr_in_ip_routing   ), priority=165  , match=(ip4.dst ==
>> 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1;
>> reg8[[16..31]] = select(1, 2);)
>>>> ])
>>>> AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed
>> 's/192\.168\.0\..0/192.168.0.??/' | sed 's/table=../table=??/' | sort],
>> [0], [dnl
>>>>   table=??(lr_in_ip_routing_ecmp), priority=100  ,
>> match=(reg8[[0..15]] == 1 && reg8[[16..31]] == 1), action=(reg0 =
>> 192.168.0.??; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport =
>> "lr0-public"; next;)
>>>> @@ -5124,7 +5124,7 @@ check ovn-nbctl --wait=sb --ecmp-symmetric-reply
>> lr-route-add lr0 1.0.0.1 192.16
>>>> 
>>>> ovn-sbctl dump-flows lr0 > lr0flows
>>>> AT_CHECK([grep -e "lr_in_ip_routing.*select" lr0flows | sed
>> 's/table=../table=??/' | sort], [0], [dnl
>>>> -  table=??(lr_in_ip_routing   ), priority=65   , match=(ip4.dst ==
>> 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1;
>> reg8[[16..31]] = select(1, 2);)
>>>> +  table=??(lr_in_ip_routing   ), priority=165  , match=(ip4.dst ==
>> 1.0.0.1/32), action=(ip.ttl--; flags.loopback = 1; reg8[[0..15]] = 1;
>> reg8[[16..31]] = select(1, 2);)
>>>> ])
>>>> AT_CHECK([grep -e "lr_in_ip_routing_ecmp" lr0flows | sed
>> 's/192\.168\.0\..0/192.168.0.??/' | sed 's/table=../table=??/' | sort],
>> [0], [dnl
>>>>   table=??(lr_in_ip_routing_ecmp), priority=100  ,
>> match=(reg8[[0..15]] == 1 && reg8[[16..31]] == 1), action=(reg0 =
>> 192.168.0.??; reg1 = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport =
>> "lr0-public"; next;)
>>>> @@ -5139,14 +5139,14 @@ check ovn-nbctl --wait=sb lr-route-add lr0
>> 1.0.0.0/24 192.168.0.10
>>>> ovn-sbctl dump-flows lr0 > lr0flows
>>>> 
>>>> AT_CHECK([grep -e "lr_in_ip_routing.*192.168.0.10" lr0flows | sed
>> 's/table=../table=??/' | sort], [0], [dnl
>>>> -  table=??(lr_in_ip_routing   ), priority=49   , match=(ip4.dst ==
>> 1.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.0.10; reg1
>> = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public";
>> flags.loopback = 1; next;)
>>>> +  table=??(lr_in_ip_routing   ), priority=149  , match=(ip4.dst ==
>> 1.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.0.10; reg1
>> = 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public";
>> flags.loopback = 1; next;)
>>>> ])
>>>> 
>>>> check ovn-nbctl --wait=sb lr-route-add lr0 2.0.0.0/24 lr0-public
>>>> 
>>>> ovn-sbctl dump-flows lr0 > lr0flows
>>>> AT_CHECK([grep -e "lr_in_ip_routing.*2.0.0.0" lr0flows | sed
>> 's/table=../table=??/' | sort], [0], [dnl
>>>> -  table=??(lr_in_ip_routing   ), priority=49   , match=(ip4.dst ==
>> 2.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg1 =
>> 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public";
>> flags.loopback = 1; next;)
>>>> +  table=??(lr_in_ip_routing   ), priority=149  , match=(ip4.dst ==
>> 2.0.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg1 =
>> 192.168.0.1; eth.src = 00:00:20:20:12:13; outport = "lr0-public";
>> flags.loopback = 1; next;)
>>>> ])
>>>> 
>>>> AT_CLEANUP
>>>> @@ -5232,3 +5232,71 @@ AT_CHECK([grep lr_in_gw_redirect lrflows | grep
>> cr-DR | sed 's/table=../table=??
>>>> 
>>>> AT_CLEANUP
>>>> ])
>>>> +
>>>> +
>>>> +OVN_FOR_EACH_NORTHD([
>>>> +AT_SETUP([route tables -- flows])
>>>> +AT_KEYWORDS([route-tables-flows])
>>>> +ovn_start
>>>> +
>>>> +check ovn-nbctl lr-add lr0
>>>> +check ovn-nbctl lrp-add lr0 lrp0 00:00:00:00:00:01 192.168.0.1/24
>>>> +check ovn-nbctl lrp-add lr0 lrp1 00:00:00:00:01:01 192.168.1.1/24
>>>> +check ovn-nbctl lrp-add lr0 lrp2 00:00:00:00:02:01 192.168.2.1/24
>>>> +check ovn-nbctl lrp-set-options lrp1 route_table=rtb-1
>>>> +check ovn-nbctl lrp-set-options lrp2 route_table=rtb-2
>>>> +
>>>> +check ovn-nbctl lr-route-add lr0 0.0.0.0/0 192.168.0.10
>>>> +check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 192.168.0.0/24
>> 192.168.1.10
>>>> +check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 0.0.0.0/0
>> 192.168.0.10
>>>> +check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 1.1.1.1/32
>> 192.168.0.20
>>>> +check ovn-nbctl --route-table=rtb-2 lr-route-add lr0 2.2.2.2/32
>> 192.168.0.30
>>>> +check ovn-nbctl --route-table=rtb-2 --ecmp lr-route-add lr0 2.2.2.2/32
>> 192.168.0.31
>>>> +check ovn-nbctl --wait=sb sync
>>>> +
>>>> +ovn-sbctl dump-flows lr0 > lr0flows
>>>> +AT_CAPTURE_FILE([lr0flows])
>>>> +
>>>> +AT_CHECK([grep -e "lr_in_ip_routing_pre.*match=(1)" lr0flows | sed
>> 's/table=../table=??/'], [0], [dnl
>>>> +  table=??(lr_in_ip_routing_pre), priority=0    , match=(1),
>> action=(next;)
>>>> +])
>>>> +
>>>> +p1_reg=$(grep -oP "lr_in_ip_routing_pre.*lrp1.*action=\(reg7 = \K."
>> lr0flows)
>>>> +p2_reg=$(grep -oP "lr_in_ip_routing_pre.*lrp2.*action=\(reg7 = \K."
>> lr0flows)
>>>> +echo $p1_reg
>>>> +echo $p2_reg
>>>> +
>>>> +# exact register values are not predictable
>>>> +if [[ $p1_reg -eq 2 ] && [ $p2_reg -eq 1 ]]; then
>>>> +  echo "swap reg values in dump"
>>>> +  sed -i -r s'/^(.*lrp2.*action=\(reg7 = )(1)(.*)/\12\3/g' lr0flows  #
>> "reg7 = 1" -> "reg7 = 2"
>>>> +  sed -i -r s'/^(.*lrp1.*action=\(reg7 = )(2)(.*)/\11\3/g' lr0flows  #
>> "reg7 = 2" -> "reg7 = 1"
>>>> +  sed -i -r s'/^(.*match=\(reg7 == )(2)( &&.*lrp1.*)/\11\3/g' lr0flows
>> # "reg7 == 2" -> "reg7 == 1"
>>>> +  sed -i -r s'/^(.*match=\(reg7 == )(1)( &&.*lrp0.*)/\12\3/g' lr0flows
>> # "reg7 == 1" -> "reg7 == 2"
>>>> +fi
>>>> +
>>>> +check test "$p1_reg" != "$p2_reg" -a $((p1_reg * p2_reg)) -eq 2
>>>> +
>>>> +AT_CHECK([grep "lr_in_ip_routing_pre" lr0flows | sed
>> 's/table=../table=??/' | sort], [0], [dnl
>>>> +  table=??(lr_in_ip_routing_pre), priority=0    , match=(1),
>> action=(next;)
>>>> +  table=??(lr_in_ip_routing_pre), priority=100  , match=(inport ==
>> "lrp1"), action=(reg7 = 1; next;)
>>>> +  table=??(lr_in_ip_routing_pre), priority=100  , match=(inport ==
>> "lrp2"), action=(reg7 = 2; next;)
>>>> +])
>>>> +
>>>> +grep -e "(lr_in_ip_routing   ).*outport" lr0flows
>>>> +
>>>> +AT_CHECK([grep -e "(lr_in_ip_routing   ).*outport" lr0flows | sed
>> 's/table=../table=??/' | sort], [0], [dnl
>>>> +  table=??(lr_in_ip_routing   ), priority=1    , match=(reg7 == 2 &&
>> ip4.dst == 0.0.0.0/0), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 =
>> 192.168.0.10; reg1 = 192.168.0.1; eth.src = 00:00:00:00:00:01; outport =
>> "lrp0"; flags.loopback = 1; next;)
>>>> +  table=??(lr_in_ip_routing   ), priority=101  , match=(ip4.dst ==
>> 0.0.0.0/0), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 192.168.0.10; reg1
>> = 192.168.0.1; eth.src = 00:00:00:00:00:01; outport = "lrp0";
>> flags.loopback = 1; next;)
>>>> +  table=??(lr_in_ip_routing   ), priority=149  , match=(ip4.dst ==
>> 192.168.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg1
>> = 192.168.0.1; eth.src = 00:00:00:00:00:01; outport = "lrp0";
>> flags.loopback = 1; next;)
>>>> +  table=??(lr_in_ip_routing   ), priority=149  , match=(ip4.dst ==
>> 192.168.1.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg1
>> = 192.168.1.1; eth.src = 00:00:00:00:01:01; outport = "lrp1";
>> flags.loopback = 1; next;)
>>>> +  table=??(lr_in_ip_routing   ), priority=149  , match=(ip4.dst ==
>> 192.168.2.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = ip4.dst; reg1
>> = 192.168.2.1; eth.src = 00:00:00:00:02:01; outport = "lrp2";
>> flags.loopback = 1; next;)
>>>> +  table=??(lr_in_ip_routing   ), priority=229  , match=(inport ==
>> "lrp0" && ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0;
>> xxreg0 = ip6.dst; xxreg1 = fe80::200:ff:fe00:1; eth.src =
>> 00:00:00:00:00:01; outport = "lrp0"; flags.loopback = 1; next;)
>>>> +  table=??(lr_in_ip_routing   ), priority=229  , match=(inport ==
>> "lrp1" && ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0;
>> xxreg0 = ip6.dst; xxreg1 = fe80::200:ff:fe00:101; eth.src =
>> 00:00:00:00:01:01; outport = "lrp1"; flags.loopback = 1; next;)
>>>> +  table=??(lr_in_ip_routing   ), priority=229  , match=(inport ==
>> "lrp2" && ip6.dst == fe80::/64), action=(ip.ttl--; reg8[[0..15]] = 0;
>> xxreg0 = ip6.dst; xxreg1 = fe80::200:ff:fe00:201; eth.src =
>> 00:00:00:00:02:01; outport = "lrp2"; flags.loopback = 1; next;)
>>>> +  table=??(lr_in_ip_routing   ), priority=49   , match=(reg7 == 1 &&
>> ip4.dst == 192.168.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 =
>> 192.168.1.10; reg1 = 192.168.1.1; eth.src = 00:00:00:00:01:01; outport =
>> "lrp1"; flags.loopback = 1; next;)
>>>> +  table=??(lr_in_ip_routing   ), priority=65   , match=(reg7 == 2 &&
>> ip4.dst == 1.1.1.1/32), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 =
>> 192.168.0.20; reg1 = 192.168.0.1; eth.src = 00:00:00:00:00:01; outport =
>> "lrp0"; flags.loopback = 1; next;)
>>>> +])
>>>> +
>>>> +AT_CLEANUP
>>>> +])
>>>> diff --git a/tests/ovn.at b/tests/ovn.at
>>>> index 49ece8735..60783a14b 100644
>>>> --- a/tests/ovn.at
>>>> +++ b/tests/ovn.at
>>>> @@ -18145,7 +18145,7 @@ eth_dst=00000000ff01
>>>> ip_src=$(ip_to_hex 10 0 0 10)
>>>> ip_dst=$(ip_to_hex 172 168 0 101)
>>>> send_icmp_packet 1 1 $eth_src $eth_dst $ip_src $ip_dst c4c9
>> 0000000000000000000000
>>>> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | awk '/table=25,
>> n_packets=1, n_bytes=45/{print $7" "$8}'],[0],[dnl
>>>> +AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | awk '/table=26,
>> n_packets=1, n_bytes=45/{print $7" "$8}'],[0],[dnl
>>>> priority=80,ip,reg15=0x3,metadata=0x3,nw_src=10.0.0.10 actions=drop
>>>> ])
>>>> 
>>>> @@ -22577,6 +22577,433 @@ OVN_CLEANUP([hv1])
>>>> AT_CLEANUP
>>>> ])
>>>> 
>>>> +
>>>> +OVN_FOR_EACH_NORTHD([
>>>> +AT_SETUP([route tables -- global routes])
>>>> +ovn_start
>>>> +
>>>> +# Logical network:
>>>> +# ls1 (192.168.1.0/24) - lrp-lr1-ls1 - lr1 - lrp-lr1-ls2 - ls2 (
>> 192.168.2.0/24)
>>>> +#
>>>> +# ls1 has lsp11 (192.168.1.11) and ls2 has lsp21 (192.168.2.21) and
>> lsp22
>>>> +# (192.168.2.22)
>>>> +#
>>>> +# lrp-lr1-ls1 set options:route_table=rtb-1
>>>> +#
>>>> +# Static routes on lr1:
>>>> +# 0.0.0.0/0 nexthop 192.168.2.21
>>>> +# 1.1.1.1/32 nexthop 192.168.2.22 route_table=rtb-1
>>>> +#
>>>> +# Test 1:
>>>> +# lsp11 send packet to 2.2.2.2
>>>> +#
>>>> +# Expected result:
>>>> +# lsp21 should receive traffic, lsp22 should not receive any traffic
>>>> +#
>>>> +# Test 2:
>>>> +# lsp11 send packet to 1.1.1.1
>>>> +#
>>>> +# Expected result:
>>>> +# lsp21 should receive traffic, lsp22 should not receive any traffic
>>>> +
>>>> +ovn-nbctl lr-add lr1
>>>> +
>>>> +for i in 1 2; do
>>>> +    ovn-nbctl ls-add ls${i}
>>>> +    ovn-nbctl lrp-add lr1 lrp-lr1-ls${i} 00:00:00:01:0${i}:01
>> 192.168.${i}.1/24
>>>> +    ovn-nbctl lsp-add ls${i} lsp-ls${i}-lr1 -- lsp-set-type
>> lsp-ls${i}-lr1 router \
>>>> +        -- lsp-set-options lsp-ls${i}-lr1 router-port=lrp-lr1-ls${i} \
>>>> +        -- lsp-set-addresses lsp-ls${i}-lr1 router
>>>> +done
>>>> +
>>>> +# install static routes
>>>> +ovn-nbctl lr-route-add lr1 0.0.0.0/0 192.168.2.21
>>>> +ovn-nbctl --route-table=rtb-1 lr-route-add lr1 1.1.1.1/32 192.168.2.22
>>>> +
>>>> +# set lrp-lr1-ls1 route table
>>>> +ovn-nbctl lrp-set-options lrp-lr1-ls1 route_table=rtb-1
>>>> +
>>>> +# Create logical ports
>>>> +ovn-nbctl lsp-add ls1 lsp11 -- \
>>>> +    lsp-set-addresses lsp11 "f0:00:00:00:01:11 192.168.1.11"
>>>> +ovn-nbctl lsp-add ls2 lsp21 -- \
>>>> +    lsp-set-addresses lsp21 "f0:00:00:00:02:21 192.168.2.21"
>>>> +ovn-nbctl lsp-add ls2 lsp22 -- \
>>>> +    lsp-set-addresses lsp22 "f0:00:00:00:02:22 192.168.2.22"
>>>> +
>>>> +net_add n1
>>>> +sim_add hv1
>>>> +as hv1
>>>> +ovs-vsctl add-br br-phys
>>>> +ovn_attach n1 br-phys 192.168.0.1
>>>> +ovs-vsctl -- add-port br-int hv1-vif1 -- \
>>>> +    set interface hv1-vif1 external-ids:iface-id=lsp11 \
>>>> +    options:tx_pcap=hv1/vif1-tx.pcap \
>>>> +    options:rxq_pcap=hv1/vif1-rx.pcap \
>>>> +    ofport-request=1
>>>> +
>>>> +ovs-vsctl -- add-port br-int hv1-vif2 -- \
>>>> +    set interface hv1-vif2 external-ids:iface-id=lsp21 \
>>>> +    options:tx_pcap=hv1/vif2-tx.pcap \
>>>> +    options:rxq_pcap=hv1/vif2-rx.pcap \
>>>> +    ofport-request=2
>>>> +
>>>> +ovs-vsctl -- add-port br-int hv1-vif3 -- \
>>>> +    set interface hv1-vif3 external-ids:iface-id=lsp22 \
>>>> +    options:tx_pcap=hv1/vif3-tx.pcap \
>>>> +    options:rxq_pcap=hv1/vif3-rx.pcap \
>>>> +    ofport-request=3
>>>> +
>>>> +# wait for earlier changes to take effect
>>>> +check ovn-nbctl --wait=hv sync
>>>> +wait_for_ports_up
>>>> +
>>>> +for i in 1 2; do
>>>> +    packet="inport==\"lsp11\" && eth.src==f0:00:00:00:01:11 &&
>> eth.dst==00:00:00:01:01:01 &&
>>>> +            ip4 && ip.ttl==64 && ip4.src==192.168.1.11 &&
>> ip4.dst==$i.$i.$i.$i && icmp"
>>>> +    AT_CHECK([as hv1 ovs-appctl -t ovn-controller inject-pkt
>> "$packet"])
>>>> +
>>>> +    # Assume all packets go to lsp21.
>>>> +    exp_packet="eth.src==00:00:00:01:02:01 &&
>> eth.dst==f0:00:00:00:02:21 &&
>>>> +            ip4 && ip.ttl==63 && ip4.src==192.168.1.11 &&
>> ip4.dst==$i.$i.$i.$i && icmp"
>>>> +    echo $exp_packet | ovstest test-ovn expr-to-packets >>
>> expected_lsp21
>>>> +done
>>>> +> expected_lsp22
>>>> +
>>>> +# lsp21 should recieve 2 packets and lsp22 should recieve no packets
>>>> +OVS_WAIT_UNTIL([
>>>> +    rcv_n1=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in"
>> hv1/vif2-tx.pcap > lsp21.packets && cat lsp21.packets | wc -l`
>>>> +    rcv_n2=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in"
>> hv1/vif3-tx.pcap > lsp22.packets && cat lsp22.packets | wc -l`
>>>> +    echo $rcv_n1 $rcv_n2
>>>> +    test $rcv_n1 -eq 2 -a $rcv_n2 -eq 0])
>>>> +
>>>> +for i in 1 2; do
>>>> +    sort expected_lsp2$i > expout
>>>> +    AT_CHECK([cat lsp2${i}.packets | sort], [0], [expout])
>>>> +done
>>>> +
>>>> +OVN_CLEANUP([hv1])
>>>> +AT_CLEANUP
>>>> +])
>>>> +
>>>> +
>>>> +OVN_FOR_EACH_NORTHD([
>>>> +AT_SETUP([route tables -- directly connected routes])
>>>> +ovn_start
>>>> +
>>>> +# Logical network:
>>>> +# ls1 (192.168.1.0/24) - lrp-lr1-ls1 - lr1 - lrp-lr1-ls2 - ls2 (
>> 192.168.2.0/24)
>>>> +#
>>>> +# ls1 has lsp11 (192.168.1.11) and ls2 has lsp21 (192.168.2.21)
>>>> +#
>>>> +# lrp-lr1-ls1 set options:route_table=rtb-1
>>>> +#
>>>> +# Static routes on lr1:
>>>> +# 192.168.2.0/25 nexthop 192.168.1.11 route_table=rtb-1
>>>> +#
>>>> +# Test 1:
>>>> +# lsp11 send packet to 192.168.2.21
>>>> +#
>>>> +# Expected result:
>>>> +# lsp21 should receive traffic, lsp11 should not receive any traffic
>>>> +
>>>> +ovn-nbctl lr-add lr1
>>>> +
>>>> +for i in 1 2; do
>>>> +    ovn-nbctl ls-add ls${i}
>>>> +    ovn-nbctl lrp-add lr1 lrp-lr1-ls${i} 00:00:00:01:0${i}:01
>> 192.168.${i}.1/24
>>>> +    ovn-nbctl lsp-add ls${i} lsp-ls${i}-lr1 -- lsp-set-type
>> lsp-ls${i}-lr1 router \
>>>> +        -- lsp-set-options lsp-ls${i}-lr1 router-port=lrp-lr1-ls${i} \
>>>> +        -- lsp-set-addresses lsp-ls${i}-lr1 router
>>>> +done
>>>> +
>>>> +# install static route, which overrides directly-connected routes
>>>> +ovn-nbctl --route-table=rtb-1 lr-route-add lr1 192.168.2.0/25
>> 192.168.1.11
>>>> +
>>>> +# set lrp-lr1-ls1 route table
>>>> +ovn-nbctl lrp-set-options lrp-lr1-ls1 route_table=rtb-1
>>>> +
>>>> +# Create logical ports
>>>> +ovn-nbctl lsp-add ls1 lsp11 -- \
>>>> +    lsp-set-addresses lsp11 "f0:00:00:00:01:11 192.168.1.11"
>>>> +ovn-nbctl lsp-add ls2 lsp21 -- \
>>>> +    lsp-set-addresses lsp21 "f0:00:00:00:02:21 192.168.2.21"
>>>> +
>>>> +net_add n1
>>>> +sim_add hv1
>>>> +as hv1
>>>> +ovs-vsctl add-br br-phys
>>>> +ovn_attach n1 br-phys 192.168.0.1
>>>> +ovs-vsctl -- add-port br-int hv1-vif1 -- \
>>>> +    set interface hv1-vif1 external-ids:iface-id=lsp11 \
>>>> +    options:tx_pcap=hv1/vif1-tx.pcap \
>>>> +    options:rxq_pcap=hv1/vif1-rx.pcap \
>>>> +    ofport-request=1
>>>> +
>>>> +ovs-vsctl -- add-port br-int hv1-vif2 -- \
>>>> +    set interface hv1-vif2 external-ids:iface-id=lsp21 \
>>>> +    options:tx_pcap=hv1/vif2-tx.pcap \
>>>> +    options:rxq_pcap=hv1/vif2-rx.pcap \
>>>> +    ofport-request=2
>>>> +
>>>> +# wait for earlier changes to take effect
>>>> +check ovn-nbctl --wait=hv sync
>>>> +wait_for_ports_up
>>>> +
>>>> +packet="inport==\"lsp11\" && eth.src==f0:00:00:00:01:11 &&
>> eth.dst==00:00:00:01:01:01 &&
>>>> +        ip4 && ip.ttl==64 && ip4.src==192.168.1.11 &&
>> ip4.dst==192.168.2.21 && icmp"
>>>> +AT_CHECK([as hv1 ovs-appctl -t ovn-controller inject-pkt "$packet"])
>>>> +
>>>> +# Assume all packets go to lsp21.
>>>> +exp_packet="eth.src==00:00:00:01:02:01 && eth.dst==f0:00:00:00:02:21 &&
>>>> +        ip4 && ip.ttl==63 && ip4.src==192.168.1.11 &&
>> ip4.dst==192.168.2.21 && icmp"
>>>> +echo $exp_packet | ovstest test-ovn expr-to-packets >> expected_lsp21
>>>> +> expected_lsp11
>>>> +
>>>> +# lsp21 should recieve 1 icmp packet and lsp11 should recieve no
>> packets
>>>> +OVS_WAIT_UNTIL([
>>>> +    rcv_n11=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in"
>> hv1/vif1-tx.pcap > lsp11.packets && cat lsp11.packets | wc -l`
>>>> +    rcv_n21=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in"
>> hv1/vif2-tx.pcap > lsp21.packets && cat lsp21.packets | wc -l`
>>>> +    echo $rcv_n11 $rcv_n21
>>>> +    test $rcv_n11 -eq 0 -a $rcv_n21 -eq 1])
>>>> +
>>>> +for i in 11 21; do
>>>> +    sort expected_lsp$i > expout
>>>> +    AT_CHECK([cat lsp${i}.packets | sort], [0], [expout])
>>>> +done
>>>> +
>>>> +OVN_CLEANUP([hv1])
>>>> +AT_CLEANUP
>>>> +])
>>>> +
>>>> +
>>>> +OVN_FOR_EACH_NORTHD([
>>>> +AT_SETUP([route tables -- overlapping subnets])
>>>> +ovn_start
>>>> +
>>>> +# Logical network:
>>>> +#
>>>> +# ls1 (192.168.1.0/24) - lrp-lr1-ls1 -\   /- lrp-lr1-ls2 - ls2 (
>> 192.168.2.0/24)
>>>> +#                                      lr1
>>>> +# ls3 (192.168.3.0/24) - lrp-lr1-ls3 -/   \- lrp-lr1-ls4 - ls4 (
>> 192.168.4.0/24)
>>>> +#
>>>> +# ls1 has lsp11 (192.168.1.11)
>>>> +# ls2 has lsp21 (192.168.2.21)
>>>> +# ls3 has lsp31 (192.168.3.31)
>>>> +# ls4 has lsp41 (192.168.4.41)
>>>> +#
>>>> +# lrp-lr1-ls1 set options:route_table=rtb-1
>>>> +# lrp-lr1-ls2 set options:route_table=rtb-2
>>>> +#
>>>> +# Static routes on lr1:
>>>> +# 10.0.0.0/24 nexthop 192.168.3.31 route_table=rtb-1
>>>> +# 10.0.0.0/24 nexthop 192.168.4.41 route_table=rtb-2
>>>> +#
>>>> +# Test 1:
>>>> +# lsp11 send packet to 10.0.0.1
>>>> +#
>>>> +# Expected result:
>>>> +# lsp31 should receive traffic, lsp41 should not receive any traffic
>>>> +#
>>>> +# Test 2:
>>>> +# lsp21 send packet to 10.0.0.1
>>>> +#
>>>> +# Expected result:
>>>> +# lsp41 should receive traffic, lsp31 should not receive any traffic
>>>> +
>>>> +ovn-nbctl lr-add lr1
>>>> +
>>>> +# Create logical topology
>>>> +for i in $(seq 1 4); do
>>>> +    ovn-nbctl ls-add ls${i}
>>>> +    ovn-nbctl lrp-add lr1 lrp-lr1-ls${i} 00:00:00:01:0${i}:01
>> 192.168.${i}.1/24
>>>> +    ovn-nbctl lsp-add ls${i} lsp-ls${i}-lr1 -- lsp-set-type
>> lsp-ls${i}-lr1 router \
>>>> +        -- lsp-set-options lsp-ls${i}-lr1 router-port=lrp-lr1-ls${i} \
>>>> +        -- lsp-set-addresses lsp-ls${i}-lr1 router
>>>> +    ovn-nbctl lsp-add ls$i lsp${i}1 -- \
>>>> +        lsp-set-addresses lsp${i}1 "f0:00:00:00:0${i}:1${i}
>> 192.168.${i}.${i}1"
>>>> +done
>>>> +
>>>> +# install static routes
>>>> +ovn-nbctl --route-table=rtb-1 lr-route-add lr1 10.0.0.0/24 192.168.3.31
>>>> +ovn-nbctl --route-table=rtb-2 lr-route-add lr1 10.0.0.0/24 192.168.4.41
>>>> +
>>>> +# set lrp-lr1-ls{1,2} route tables
>>>> +ovn-nbctl lrp-set-options lrp-lr1-ls1 route_table=rtb-1
>>>> +ovn-nbctl lrp-set-options lrp-lr1-ls2 route_table=rtb-2
>>>> +
>>>> +net_add n1
>>>> +sim_add hv1
>>>> +as hv1
>>>> +ovs-vsctl add-br br-phys
>>>> +ovn_attach n1 br-phys 192.168.0.1
>>>> +
>>>> +for i in $(seq 1 4); do
>>>> +    ovs-vsctl -- add-port br-int hv1-vif${i} -- \
>>>> +        set interface hv1-vif${i} external-ids:iface-id=lsp${i}1 \
>>>> +        options:tx_pcap=hv1/vif${i}-tx.pcap \
>>>> +        options:rxq_pcap=hv1/vif${i}-rx.pcap \
>>>> +        ofport-request=${i}
>>>> +done
>>>> +
>>>> +# wait for earlier changes to take effect
>>>> +check ovn-nbctl --wait=hv sync
>>>> +wait_for_ports_up
>>>> +
>>>> +# lsp31 should recieve packet coming from lsp11
>>>> +# lsp41 should recieve packet coming from lsp21
>>>> +for i in $(seq 1 2); do
>>>> +    di=$(( i + 2))  # dst index
>>>> +    ri=$(( 5 - i))  # reverse index
>>>> +    packet="inport==\"lsp${i}1\" && eth.src==f0:00:00:00:0${i}:1${i} &&
>>>> +            eth.dst==00:00:00:01:0${i}:01 && ip4 && ip.ttl==64 &&
>>>> +            ip4.src==192.168.${i}.${i}1 && ip4.dst==10.0.0.1 && icmp"
>>>> +    AT_CHECK([as hv1 ovs-appctl -t ovn-controller inject-pkt
>> "$packet"])
>>>> +
>>>> +    # Assume all packets go to lsp${di}1.
>>>> +    exp_packet="eth.src==00:00:00:01:0${di}:01 &&
>> eth.dst==f0:00:00:00:0${di}:1${di} &&
>>>> +            ip4 && ip.ttl==63 && ip4.src==192.168.${i}.${i}1 &&
>> ip4.dst==10.0.0.1 && icmp"
>>>> +    echo $exp_packet | ovstest test-ovn expr-to-packets >>
>> expected_lsp${di}1
>>>> +    > expected_lsp${ri}1
>>>> +
>>>> +    OVS_WAIT_UNTIL([
>>>> +        rcv_n1=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in"
>> hv1/vif${di}-tx.pcap > lsp${di}1.packets && cat lsp${di}1.packets | wc -l`
>>>> +        rcv_n2=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in"
>> hv1/vif${ri}-tx.pcap > lsp${ri}1.packets && cat lsp${ri}1.packets | wc -l`
>>>> +        echo $rcv_n1 $rcv_n2
>>>> +        test $rcv_n1 -eq 1 -a $rcv_n2 -eq 0])
>>>> +
>>>> +    for j in "${di}1" "${ri}1"; do
>>>> +        sort expected_lsp${j} > expout
>>>> +        AT_CHECK([cat lsp${j}.packets | sort], [0], [expout])
>>>> +    done
>>>> +
>>>> +    # cleanup tx pcap files
>>>> +    for j in "${di}1" "${ri}1"; do
>>>> +        ovs-vsctl -- remove interface hv1-vif${di} options tx_pcap
>>>> +        > hv1/vif${di}-tx.pcap
>>>> +        ovs-vsctl -- set interface hv1-vif${di}
>> external-ids:iface-id=lsp${di}1 \
>>>> +            options:tx_pcap=hv1/vif${di}-tx.pcap
>>>> +    done
>>>> +done
>>>> +
>>>> +OVN_CLEANUP([hv1])
>>>> +AT_CLEANUP
>>>> +])
>>>> +
>>>> +
>>>> +OVN_FOR_EACH_NORTHD([
>>>> +AT_SETUP([route tables IPv6 -- overlapping subnets])
>>>> +ovn_start
>>>> +
>>>> +# Logical network:
>>>> +#
>>>> +# ls1 (2001:db8:1::/64) - lrp-lr1-ls1 -\   /- lrp-lr1-ls2 - ls2
>> (2001:db8:2::/64)
>>>> +#                                       lr1
>>>> +# ls3 (2001:db8:3::/64) - lrp-lr1-ls3 -/   \- lrp-lr1-ls4 - ls4
>> (2001:db8:4::/64)
>>>> +#
>>>> +# ls1 has lsp11 (2001:db8:1::11)
>>>> +# ls2 has lsp21 (2001:db8:2::21)
>>>> +# ls3 has lsp31 (2001:db8:3::31)
>>>> +# ls4 has lsp41 (2001:db8:4::41)
>>>> +#
>>>> +# lrp-lr1-ls1 set options:route_table=rtb-1
>>>> +# lrp-lr1-ls2 set options:route_table=rtb-2
>>>> +#
>>>> +# Static routes on lr1:
>>>> +# 2001:db8:2000::/64 nexthop 2001:db8:3::31 route_table=rtb-1
>>>> +# 2001:db8:2000::/64 nexthop 2001:db8:3::41 route_table=rtb-2
>>>> +#
>>>> +# Test 1:
>>>> +# lsp11 send packet to 2001:db8:2000::1
>>>> +#
>>>> +# Expected result:
>>>> +# lsp31 should receive traffic, lsp41 should not receive any traffic
>>>> +#
>>>> +# Test 2:
>>>> +# lsp21 send packet to 2001:db8:2000::1
>>>> +#
>>>> +# Expected result:
>>>> +# lsp41 should receive traffic, lsp31 should not receive any traffic
>>>> +
>>>> +ovn-nbctl lr-add lr1
>>>> +
>>>> +# Create logical topology
>>>> +for i in $(seq 1 4); do
>>>> +    ovn-nbctl ls-add ls${i}
>>>> +    ovn-nbctl lrp-add lr1 lrp-lr1-ls${i} 00:00:00:01:0${i}:01
>> 2001:db8:${i}::1/64
>>>> +    ovn-nbctl lsp-add ls${i} lsp-ls${i}-lr1 -- lsp-set-type
>> lsp-ls${i}-lr1 router \
>>>> +        -- lsp-set-options lsp-ls${i}-lr1 router-port=lrp-lr1-ls${i} \
>>>> +        -- lsp-set-addresses lsp-ls${i}-lr1 router
>>>> +    ovn-nbctl lsp-add ls$i lsp${i}1 -- \
>>>> +        lsp-set-addresses lsp${i}1 "f0:00:00:00:0${i}:1${i}
>> 2001:db8:${i}::${i}1"
>>>> +done
>>>> +
>>>> +# install static routes
>>>> +ovn-nbctl --route-table=rtb-1 lr-route-add lr1 2001:db8:2000::/64
>> 2001:db8:3::31
>>>> +ovn-nbctl --route-table=rtb-2 lr-route-add lr1 2001:db8:2000::/64
>> 2001:db8:4::41
>>>> +
>>>> +# set lrp-lr1-ls{1,2} route tables
>>>> +ovn-nbctl lrp-set-options lrp-lr1-ls1 route_table=rtb-1
>>>> +ovn-nbctl lrp-set-options lrp-lr1-ls2 route_table=rtb-2
>>>> +
>>>> +net_add n1
>>>> +sim_add hv1
>>>> +as hv1
>>>> +ovs-vsctl add-br br-phys
>>>> +ovn_attach n1 br-phys 192.168.0.1
>>>> +
>>>> +for i in $(seq 1 4); do
>>>> +    ovs-vsctl -- add-port br-int hv1-vif${i} -- \
>>>> +        set interface hv1-vif${i} external-ids:iface-id=lsp${i}1 \
>>>> +        options:tx_pcap=hv1/vif${i}-tx.pcap \
>>>> +        options:rxq_pcap=hv1/vif${i}-rx.pcap \
>>>> +        ofport-request=${i}
>>>> +done
>>>> +
>>>> +# wait for earlier changes to take effect
>>>> +AT_CHECK([ovn-nbctl --timeout=3 --wait=hv sync], [0], [ignore])
>>>> +
>>>> +# lsp31 should recieve packet coming from lsp11
>>>> +# lsp41 should recieve packet coming from lsp21
>>>> +for i in $(seq 1 2); do
>>>> +    di=$(( i + 2))  # dst index
>>>> +    ri=$(( 5 - i))  # reverse index
>>>> +    packet="inport==\"lsp${i}1\" && eth.src==f0:00:00:00:0${i}:1${i} &&
>>>> +            eth.dst==00:00:00:01:0${i}:01 && ip6 && ip.ttl==64 &&
>>>> +            ip6.src==2001:db8:${i}::${i}1 && ip6.dst==2001:db8:2000::1
>> && icmp6"
>>>> +    AT_CHECK([as hv1 ovs-appctl -t ovn-controller inject-pkt
>> "$packet"])
>>>> +
>>>> +    # Assume all packets go to lsp${di}1.
>>>> +    exp_packet="eth.src==00:00:00:01:0${di}:01 &&
>> eth.dst==f0:00:00:00:0${di}:1${di} && ip6 &&
>>>> +                ip.ttl==63 && ip6.src==2001:db8:${i}::${i}1 &&
>> ip6.dst==2001:db8:2000::1 && icmp6"
>>>> +    echo $exp_packet | ovstest test-ovn expr-to-packets >>
>> expected_lsp${di}1
>>>> +    > expected_lsp${ri}1
>>>> +
>>>> +    OVS_WAIT_UNTIL([
>>>> +        rcv_n1=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in"
>> hv1/vif${di}-tx.pcap > lsp${di}1.packets && cat lsp${di}1.packets | wc -l`
>>>> +        rcv_n2=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in"
>> hv1/vif${ri}-tx.pcap > lsp${ri}1.packets && cat lsp${ri}1.packets | wc -l`
>>>> +        echo $rcv_n1 $rcv_n2
>>>> +        test $rcv_n1 -eq 1 -a $rcv_n2 -eq 0])
>>>> +
>>>> +    for j in "${di}1" "${ri}1"; do
>>>> +        sort expected_lsp${j} > expout
>>>> +        AT_CHECK([cat lsp${j}.packets | sort], [0], [expout])
>>>> +    done
>>>> +
>>>> +    # cleanup tx pcap files
>>>> +    for j in "${di}1" "${ri}1"; do
>>>> +        ovs-vsctl -- remove interface hv1-vif${di} options tx_pcap
>>>> +        > hv1/vif${di}-tx.pcap
>>>> +        ovs-vsctl -- set interface hv1-vif${di}
>> external-ids:iface-id=lsp${di}1 \
>>>> +            options:tx_pcap=hv1/vif${di}-tx.pcap
>>>> +    done
>>>> +done
>>>> +
>>>> +OVN_CLEANUP([hv1])
>>>> +AT_CLEANUP
>>>> +])
>>>> +
>>>> +
>>>> OVN_FOR_EACH_NORTHD([
>>>> AT_SETUP([forwarding group: 3 HVs, 1 LR, 2 LS])
>>>> AT_KEYWORDS([forwarding-group])
>>>> @@ -23332,7 +23759,7 @@ ovn-sbctl dump-flows > sbflows
>>>> AT_CAPTURE_FILE([sbflows])
>>>> AT_CAPTURE_FILE([offlows])
>>>> OVS_WAIT_UNTIL([
>>>> -    as hv1 ovs-ofctl dump-flows br-int table=20 > offlows
>>>> +    as hv1 ovs-ofctl dump-flows br-int table=21 > offlows
>>>>     test $(grep -c "load:0x64->NXM_NX_PKT_MARK" offlows) = 1 && \
>>>>     test $(grep -c "load:0x3->NXM_NX_PKT_MARK" offlows) = 1 && \
>>>>     test $(grep -c "load:0x4->NXM_NX_PKT_MARK" offlows) = 1 && \
>>>> @@ -23425,12 +23852,12 @@ send_ipv4_pkt hv1 hv1-vif1 505400000003
>> 00000000ff01 \
>>>>     $(ip_to_hex 10 0 0 3) $(ip_to_hex 172 168 0 120)
>>>> 
>>>> OVS_WAIT_UNTIL([
>>>> -    test 1 -eq $(as hv1 ovs-ofctl dump-flows br-int table=20 | \
>>>> +    test 1 -eq $(as hv1 ovs-ofctl dump-flows br-int table=21 | \
>>>>     grep "load:0x2->NXM_NX_PKT_MARK" -c)
>>>> ])
>>>> 
>>>> AT_CHECK([
>>>> -    test 0 -eq $(as hv1 ovs-ofctl dump-flows br-int table=20 | \
>>>> +    test 0 -eq $(as hv1 ovs-ofctl dump-flows br-int table=21 | \
>>>>     grep "load:0x64->NXM_NX_PKT_MARK" -c)
>>>> ])
>>>> 
>>>> @@ -24133,7 +24560,7 @@ AT_CHECK([
>>>>         grep "priority=100" | \
>>>>         grep -c
>> "ct(commit,zone=NXM_NX_REG11\\[[0..15\\]],.*exec(move:NXM_OF_ETH_SRC\\[[\\]]->NXM_NX_CT_LABEL\\[[32..79\\]],load:0x[[0-9]]->NXM_NX_CT_LABEL\\[[80..95\\]]))"
>>>> 
>>>> -        grep table=22 hv${hv}flows | \
>>>> +        grep table=23 hv${hv}flows | \
>>>>         grep "priority=200" | \
>>>>         grep -c
>> "actions=move:NXM_NX_CT_LABEL\\[[32..79\\]]->NXM_OF_ETH_DST\\[[\\]]"
>>>>     done; :], [0], [dnl
>>>> @@ -24258,7 +24685,7 @@ AT_CHECK([
>>>>         grep "priority=100" | \
>>>>         grep -c
>> "ct(commit,zone=NXM_NX_REG11\\[[0..15\\]],.*exec(move:NXM_OF_ETH_SRC\\[[\\]]->NXM_NX_CT_LABEL\\[[32..79\\]],load:0x[[0-9]]->NXM_NX_CT_LABEL\\[[80..95\\]]))"
>>>> 
>>>> -        grep table=22 hv${hv}flows | \
>>>> +        grep table=23 hv${hv}flows | \
>>>>         grep "priority=200" | \
>>>>         grep -c
>> "actions=move:NXM_NX_CT_LABEL\\[[32..79\\]]->NXM_OF_ETH_DST\\[[\\]]"
>>>>     done; :], [0], [dnl
>>>> @@ -24880,7 +25307,7 @@ AT_CHECK([as hv1 ovs-ofctl dump-flows br-int |
>> grep "actions=controller" | grep
>>>> ])
>>>> 
>>>> # The packet should've been dropped in the lr_in_arp_resolve stage.
>>>> -AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep -E "table=22,
>> n_packets=1,.* priority=1,ip,metadata=0x${sw_key},nw_dst=10.0.1.1
>> actions=drop" -c], [0], [dnl
>>>> +AT_CHECK([as hv1 ovs-ofctl dump-flows br-int | grep -E "table=23,
>> n_packets=1,.* priority=1,ip,metadata=0x${sw_key},nw_dst=10.0.1.1
>> actions=drop" -c], [0], [dnl
>>>> 1
>>>> ])
>>>> 
>>>> diff --git a/utilities/ovn-nbctl.c b/utilities/ovn-nbctl.c
>>>> index e34bb65f7..0ff10618b 100644
>>>> --- a/utilities/ovn-nbctl.c
>>>> +++ b/utilities/ovn-nbctl.c
>>>> @@ -329,6 +329,8 @@ Logical router port commands:\n\
>>>>                             add logical port PORT on ROUTER\n\
>>>>   lrp-set-gateway-chassis PORT CHASSIS [PRIORITY]\n\
>>>>                             set gateway chassis for port PORT\n\
>>>> +  lrp-set-options PORT KEY=VALUE [KEY=VALUE]...\n\
>>>> +                            set router port options\n\
>>>>   lrp-del-gateway-chassis PORT CHASSIS\n\
>>>>                             delete gateway chassis from port PORT\n\
>>>>   lrp-get-gateway-chassis PORT\n\
>>>> @@ -351,11 +353,17 @@ Logical router port commands:\n\
>>>>                             ('overlay' or 'bridged')\n\
>>>> \n\
>>>> Route commands:\n\
>>>> -  [--policy=POLICY] [--ecmp] [--ecmp-symmetric-reply] lr-route-add
>> ROUTER \n\
>>>> -                            PREFIX NEXTHOP [PORT]\n\
>>>> +  [--policy=POLICY]\n\
>>>> +  [--ecmp]\n\
>>>> +  [--ecmp-symmetric-reply]\n\
>>>> +  [--route-table=ROUTE_TABLE]\n\
>>>> +  lr-route-add ROUTER PREFIX NEXTHOP [PORT]\n\
>>>>                             add a route to ROUTER\n\
>>>> -  [--policy=POLICY] lr-route-del ROUTER [PREFIX [NEXTHOP [PORT]]]\n\
>>>> +  [--policy=POLICY]\n\
>>>> +  [--route-table=ROUTE_TABLE]\n\
>>>> +  lr-route-del ROUTER [PREFIX [NEXTHOP [PORT]]]\n\
>>>>                             remove routes from ROUTER\n\
>>>> +  [--route-table=ROUTE_TABLE]\n\
>>>>   lr-route-list ROUTER      print routes for ROUTER\n\
>>>> \n\
>>>> Policy commands:\n\
>>>> @@ -743,6 +751,11 @@ print_lr(const struct nbrec_logical_router *lr,
>> struct ds *s)
>>>>             ds_put_cstr(s, "]\n");
>>>>         }
>>>> 
>>>> +        const char *route_table = smap_get(&lrp->options,
>> "route_table");
>>>> +        if (route_table) {
>>>> +            ds_put_format(s, "        route-table: %s\n", route_table);
>>>> +        }
>>>> +
>>>>         if (lrp->n_gateway_chassis) {
>>>>             const struct nbrec_gateway_chassis **gcs;
>>>> 
>>>> @@ -862,6 +875,7 @@ nbctl_pre_show(struct ctl_context *ctx)
>>>>     ovsdb_idl_add_column(ctx->idl,
>> &nbrec_logical_router_port_col_name);
>>>>     ovsdb_idl_add_column(ctx->idl, &nbrec_logical_router_port_col_mac);
>>>>     ovsdb_idl_add_column(ctx->idl,
>> &nbrec_logical_router_port_col_networks);
>>>> +    ovsdb_idl_add_column(ctx->idl,
>> &nbrec_logical_router_port_col_options);
>>>>     ovsdb_idl_add_column(ctx->idl,
>> &nbrec_logical_router_port_col_gateway_chassis);
>>>> 
>>>>     ovsdb_idl_add_column(ctx->idl,
>> &nbrec_gateway_chassis_col_chassis_name);
>>>> @@ -4000,11 +4014,19 @@ nbctl_lr_policy_list(struct ctl_context *ctx)
>>>> 
>>>> static struct nbrec_logical_router_static_route *
>>>> nbctl_lr_get_route(const struct nbrec_logical_router *lr, char *prefix,
>>>> -                   char *next_hop, bool is_src_route, bool ecmp)
>>>> +                   char *next_hop, bool is_src_route, bool ecmp,
>>>> +                   char *route_table)
>>>> {
>>>>     for (int i = 0; i < lr->n_static_routes; i++) {
>>>>         struct nbrec_logical_router_static_route *route =
>> lr->static_routes[i];
>>>> 
>>>> +        /* Strict compare for route_table.
>>>> +         * If route_table was not specified,
>>>> +         * lookup for routes with empty route_table value. */
>>>> +        if (strcmp(route->route_table, route_table ? route_table :
>> "")) {
>>>> +            continue;
>>>> +        }
>>>> +
>>>>         /* Compare route policy. */
>>>>         char *nb_policy = route->policy;
>>>>         bool nb_is_src_route = false;
>>>> @@ -4060,6 +4082,8 @@ nbctl_pre_lr_route_add(struct ctl_context *ctx)
>>>>                          &nbrec_logical_router_static_route_col_bfd);
>>>>     ovsdb_idl_add_column(ctx->idl,
>>>> 
>> &nbrec_logical_router_static_route_col_options);
>>>> +    ovsdb_idl_add_column(ctx->idl,
>>>> +
>> &nbrec_logical_router_static_route_col_route_table);
>>>> }
>>>> 
>>>> static char * OVS_WARN_UNUSED_RESULT
>>>> @@ -4090,6 +4114,7 @@ nbctl_lr_route_add(struct ctl_context *ctx)
>>>>         }
>>>>     }
>>>> 
>>>> +    char *route_table = shash_find_data(&ctx->options,
>> "--route-table");
>>>>     bool v6_prefix = false;
>>>>     prefix = normalize_ipv4_prefix_str(ctx->argv[2]);
>>>>     if (!prefix) {
>>>> @@ -4166,7 +4191,8 @@ nbctl_lr_route_add(struct ctl_context *ctx)
>>>>     bool ecmp = shash_find(&ctx->options, "--ecmp") != NULL ||
>>>>                 ecmp_symmetric_reply;
>>>>     struct nbrec_logical_router_static_route *route =
>>>> -        nbctl_lr_get_route(lr, prefix, next_hop, is_src_route, ecmp);
>>>> +        nbctl_lr_get_route(lr, prefix, next_hop, is_src_route, ecmp,
>>>> +                           route_table);
>>>> 
>>>>     /* Validations for nexthop = "discard" */
>>>>     if (is_discard_route) {
>>>> @@ -4230,7 +4256,8 @@ nbctl_lr_route_add(struct ctl_context *ctx)
>>>>     }
>>>> 
>>>>     struct nbrec_logical_router_static_route *discard_route =
>>>> -        nbctl_lr_get_route(lr, prefix, "discard", is_src_route, true);
>>>> +        nbctl_lr_get_route(lr, prefix, "discard", is_src_route, true,
>>>> +                           route_table);
>>>>     if (discard_route) {
>>>>         ctl_error(ctx, "discard nexthop for the same ECMP route
>> exists.");
>>>>         goto cleanup;
>>>> @@ -4246,6 +4273,9 @@ nbctl_lr_route_add(struct ctl_context *ctx)
>>>>     if (policy) {
>>>>         nbrec_logical_router_static_route_set_policy(route, policy);
>>>>     }
>>>> +    if (route_table) {
>>>> +        nbrec_logical_router_static_route_set_route_table(route,
>> route_table);
>>>> +    }
>>>> 
>>>>     if (ecmp_symmetric_reply) {
>>>>         const struct smap options = SMAP_CONST1(&options,
>>>> @@ -4289,6 +4319,8 @@ nbctl_pre_lr_route_del(struct ctl_context *ctx)
>>>> 
>> &nbrec_logical_router_static_route_col_nexthop);
>>>>     ovsdb_idl_add_column(ctx->idl,
>>>> 
>> &nbrec_logical_router_static_route_col_output_port);
>>>> +    ovsdb_idl_add_column(ctx->idl,
>>>> +
>> &nbrec_logical_router_static_route_col_route_table);
>>>> 
>>>> }
>>>> 
>>>> @@ -4302,6 +4334,7 @@ nbctl_lr_route_del(struct ctl_context *ctx)
>>>>         return;
>>>>     }
>>>> 
>>>> +    const char *route_table = shash_find_data(&ctx->options,
>> "--route-table");
>>>>     const char *policy = shash_find_data(&ctx->options, "--policy");
>>>>     bool is_src_route = false;
>>>>     if (policy) {
>>>> @@ -4392,6 +4425,14 @@ nbctl_lr_route_del(struct ctl_context *ctx)
>>>>             }
>>>>         }
>>>> 
>>>> +        /* Strict compare for route_table.
>>>> +         * If route_table was not specified,
>>>> +         * lookup for routes with empty route_table value. */
>>>> +        if (strcmp(lr->static_routes[i]->route_table,
>>>> +                   route_table ? route_table : "")) {
>>>> +            continue;
>>>> +        }
>>>> +
>>>>         /* Compare output_port, if specified. */
>>>>         if (output_port) {
>>>>             char *rt_output_port = lr->static_routes[i]->output_port;
>>>> @@ -5115,6 +5156,41 @@ nbctl_pre_lrp_del_gateway_chassis(struct
>> ctl_context *ctx)
>>>>     ovsdb_idl_add_column(ctx->idl,
>> &nbrec_gateway_chassis_col_chassis_name);
>>>> }
>>>> 
>>>> +static void
>>>> +nbctl_pre_lrp_options(struct ctl_context *ctx)
>>>> +{
>>>> +    ovsdb_idl_add_column(ctx->idl,
>> &nbrec_logical_router_port_col_name);
>>>> +    ovsdb_idl_add_column(ctx->idl,
>> &nbrec_logical_router_port_col_options);
>>>> +}
>>>> +
>>>> +static void
>>>> +nbctl_lrp_set_options(struct ctl_context *ctx)
>>>> +{
>>>> +    const char *id = ctx->argv[1];
>>>> +    const struct nbrec_logical_router_port *lrp = NULL;
>>>> +    size_t i;
>>>> +    struct smap options = SMAP_INITIALIZER(&options);
>>>> +
>>>> +    char *error = lrp_by_name_or_uuid(ctx, id, true, &lrp);
>>>> +    if (error) {
>>>> +        ctx->error = error;
>>>> +        return;
>>>> +    }
>>>> +    for (i = 2; i < ctx->argc; i++) {
>>>> +        char *key, *value;
>>>> +        value = xstrdup(ctx->argv[i]);
>>>> +        key = strsep(&value, "=");
>>>> +        if (value) {
>>>> +            smap_add(&options, key, value);
>>>> +        }
>>>> +        free(key);
>>>> +    }
>>>> +
>>>> +    nbrec_logical_router_port_set_options(lrp, &options);
>>>> +
>>>> +    smap_destroy(&options);
>>>> +}
>>>> +
>>>> /* Removes logical router port 'lrp->gateway_chassis[idx]'. */
>>>> static void
>>>> remove_gc(const struct nbrec_logical_router_port *lrp, size_t idx)
>>>> @@ -5891,6 +5967,7 @@ route_cmp_details(const struct
>> nbrec_logical_router_static_route *r1,
>>>>     }
>>>>     return r1->output_port ? 1 : -1;
>>>> }
>>>> +
>>>> struct ipv4_route {
>>>>     int priority;
>>>>     ovs_be32 addr;
>>>> @@ -5900,6 +5977,11 @@ struct ipv4_route {
>>>> static int
>>>> __ipv4_route_cmp(const struct ipv4_route *r1, const struct ipv4_route
>> *r2)
>>>> {
>>>> +    int rtb_cmp = strcmp(r1->route->route_table,
>>>> +                         r2->route->route_table);
>>>> +    if (rtb_cmp) {
>>>> +        return rtb_cmp;
>>>> +    }
>>>>     if (r1->priority != r2->priority) {
>>>>         return r1->priority > r2->priority ? -1 : 1;
>>>>     }
>>>> @@ -5931,6 +6013,11 @@ struct ipv6_route {
>>>> static int
>>>> __ipv6_route_cmp(const struct ipv6_route *r1, const struct ipv6_route
>> *r2)
>>>> {
>>>> +    int rtb_cmp = strcmp(r1->route->route_table,
>>>> +                         r2->route->route_table);
>>>> +    if (rtb_cmp) {
>>>> +        return rtb_cmp;
>>>> +    }
>>>>     if (r1->priority != r2->priority) {
>>>>         return r1->priority > r2->priority ? -1 : 1;
>>>>     }
>>>> @@ -6018,6 +6105,8 @@ nbctl_pre_lr_route_list(struct ctl_context *ctx)
>>>> 
>> &nbrec_logical_router_static_route_col_options);
>>>>     ovsdb_idl_add_column(ctx->idl,
>>>>                          &nbrec_logical_router_static_route_col_bfd);
>>>> +    ovsdb_idl_add_column(ctx->idl,
>>>> +
>> &nbrec_logical_router_static_route_col_route_table);
>>>> }
>>>> 
>>>> static void
>>>> @@ -6035,12 +6124,17 @@ nbctl_lr_route_list(struct ctl_context *ctx)
>>>>         return;
>>>>     }
>>>> 
>>>> +    char *route_table = shash_find_data(&ctx->options,
>> "--route-table");
>>>> +
>>>>     ipv4_routes = xmalloc(sizeof *ipv4_routes * lr->n_static_routes);
>>>>     ipv6_routes = xmalloc(sizeof *ipv6_routes * lr->n_static_routes);
>>>> 
>>>>     for (int i = 0; i < lr->n_static_routes; i++) {
>>>>         const struct nbrec_logical_router_static_route *route
>>>>             = lr->static_routes[i];
>>>> +        if (route_table && strcmp(route->route_table, route_table)) {
>>>> +            continue;
>>>> +        }
>>>>         unsigned int plen;
>>>>         ovs_be32 ipv4;
>>>>         const char *policy = route->policy ? route->policy : "dst-ip";
>>>> @@ -6081,6 +6175,7 @@ nbctl_lr_route_list(struct ctl_context *ctx)
>>>>     if (n_ipv4_routes) {
>>>>         ds_put_cstr(&ctx->output, "IPv4 Routes\n");
>>>>     }
>>>> +    const struct nbrec_logical_router_static_route *route;
>>>>     for (int i = 0; i < n_ipv4_routes; i++) {
>>>>         bool ecmp = false;
>>>>         if (i < n_ipv4_routes - 1 &&
>>>> @@ -6091,6 +6186,15 @@ nbctl_lr_route_list(struct ctl_context *ctx)
>>>>                                      &ipv4_routes[i - 1])) {
>>>>             ecmp = true;
>>>>         }
>>>> +
>>>> +        route = ipv4_routes[i].route;
>>>> +        if (!i || (i > 0 && strcmp(route->route_table,
>>>> +                                   ipv4_routes[i -
>> 1].route->route_table))) {
>>>> +            ds_put_format(&ctx->output, "%sRoute Table %s:\n", i ?
>> "\n" : "",
>>>> +                          strlen(route->route_table) ?
>> route->route_table
>>>> +                                                     : "global");
>>>> +        }
>>>> +
>>>>         print_route(ipv4_routes[i].route, &ctx->output, ecmp);
>>>>     }
>>>> 
>>>> @@ -6108,6 +6212,15 @@ nbctl_lr_route_list(struct ctl_context *ctx)
>>>>                                      &ipv6_routes[i - 1])) {
>>>>             ecmp = true;
>>>>         }
>>>> +
>>>> +        route = ipv6_routes[i].route;
>>>> +        if (!i || (i > 0 && strcmp(route->route_table,
>>>> +                                   ipv6_routes[i -
>> 1].route->route_table))) {
>>>> +            ds_put_format(&ctx->output, "%sRoute Table %s:\n", i ?
>> "\n" : "",
>>>> +                          strlen(route->route_table) ?
>> route->route_table
>>>> +                                                     : "global");
>>>> +        }
>>>> +
>>>>         print_route(ipv6_routes[i].route, &ctx->output, ecmp);
>>>>     }
>>>> 
>>>> @@ -6926,6 +7039,8 @@ static const struct ctl_command_syntax
>> nbctl_commands[] = {
>>>>       "PORT CHASSIS [PRIORITY]",
>>>>       nbctl_pre_lrp_set_gateway_chassis, nbctl_lrp_set_gateway_chassis,
>>>>       NULL, "--may-exist", RW },
>>>> +    { "lrp-set-options", 1, INT_MAX, "PORT KEY=VALUE [KEY=VALUE]...",
>>>> +      nbctl_pre_lrp_options, nbctl_lrp_set_options, NULL, "", RW },
>>>>     { "lrp-del-gateway-chassis", 2, 2, "PORT CHASSIS",
>>>>       nbctl_pre_lrp_del_gateway_chassis, nbctl_lrp_del_gateway_chassis,
>>>>       NULL, "", RW },
>>>> @@ -6949,12 +7064,13 @@ static const struct ctl_command_syntax
>> nbctl_commands[] = {
>>>>     /* logical router route commands. */
>>>>     { "lr-route-add", 3, 4, "ROUTER PREFIX NEXTHOP [PORT]",
>>>>       nbctl_pre_lr_route_add, nbctl_lr_route_add, NULL,
>>>> -      "--may-exist,--ecmp,--ecmp-symmetric-reply,--policy=,--bfd?", RW
>> },
>>>> +
>> "--may-exist,--ecmp,--ecmp-symmetric-reply,--policy=,--route-table=,--bfd?",
>>>> +      RW },
>>>>     { "lr-route-del", 1, 4, "ROUTER [PREFIX [NEXTHOP [PORT]]]",
>>>>       nbctl_pre_lr_route_del, nbctl_lr_route_del, NULL,
>>>> -      "--if-exists,--policy=", RW },
>>>> +      "--if-exists,--policy=,--route-table=", RW },
>>>>     { "lr-route-list", 1, 1, "ROUTER", nbctl_pre_lr_route_list,
>>>> -      nbctl_lr_route_list, NULL, "", RO },
>>>> +      nbctl_lr_route_list, NULL, "--route-table=", RO },
>>>> 
>>>>     /* Policy commands */
>>>>     { "lr-policy-add", 4, INT_MAX,
>>>> --
>>>> 2.30.0
>>>> 
>>>> _______________________________________________
>>>> dev mailing list
>>>> dev at openvswitch.org
>>>> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>>> 
>>> 
> _______________________________________________
> dev mailing list
> dev at openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev



More information about the dev mailing list