[ovs-dev] [ovn] Need advice for multiple routing tables support in LR

Tue Aug 10 11:59:47 UTC 2021

Hi Numan,

thanks for the answer.

You said that it’s possible to use reg2-reg7 for ipv4.
I’ve checked usage of xxreg0 and xxreg1 to find out where they’re used in router pipeline.

In ovn-northd.c it’s written that those ipv6-related registers are used in >= IP_INPUT. Am I right, its about lrouter stage "lr_in_ip_input"?
If yes, I’ve analyzed ovn-northd code and it looks like first time when xxreg0 is used to write is lr_in_defrag. And xxreg1 is used in write mode first time in lr_in_ip_routing.

Can you please cofirm I am right, that I can use xxreg1’s subregisters (for instance, reg7) in one stage prior to lr_in_ip_routing?
If yes, I think we should fix router’s registers usage documentation:
xxreg0: ">= IP_INPUT" -> ">= DEFRAG"
xxreg1: ">= IP_INPUT" -> ">= IP_ROUTING"

Right?

Regards,
Vladislav Odintsov

> On 9 Aug 2021, at 00:09, Numan Siddique <numans at ovn.org> wrote:
> 
> On Thu, Aug 5, 2021 at 1:29 PM Vladislav Odintsov <odivlad at gmail.com <mailto:odivlad at gmail.com>> wrote:
>> 
>> Hi,
>> 
>> I’m trying to implement multiple routing tables support for Logical Routers and met some difficulties, need help/advice.
>> 
>> How I see it can be used by administrators:
>> 
>> 1. In LRP’s options field is added key route_table with the routing table’s name - any string.
>> 2. When adding new logical router static route record, supply optional parameter - route_table.
>> 3. If route_table values in LRP and LR route match, packets coming from this particular LRP are looked up against routes with mentioned route_table value.
>> 4. Routes to directly-connected networks of LRs have higher priority that routes from route_tables.
>> 
>> So, this functionality can help achieve different routes loockup for different LR’s subnets.
>> 
>> Planned changes:
>> - add route_table field to Logical_Router_Static_Route table’s schema
>> - add new optional field to Logical_Router_Static_Routes table, which when absent indicates that route is global for LR (current behaviour), and when there is a string with route_table name
>> 
>> Regards,
>> Vladislav Odintsov
>> - add new stage in logical router ingress pipeline right before lr_in_ip_routing - lr_in_ip_routing_pre. In this stage packets are checked against inport and route_table’s id (computed somehow, uniq per route_table of LR) writes to some of OVS registers and then passes to next table - lr_in_ip_routing. Currently in my dev env it looks like this:
>> 
>>  table=9 (lr_in_ip_routing_pre), priority=100  , match=(inport == "subnet-8E3CDCC0"), action=(reg7 = 110; next;)
>>  table=9 (lr_in_ip_routing_pre), priority=100  , match=(inport == "subnet-E99CDA60"), action=(reg7 = 100; next;)
>>  table=9 (lr_in_ip_routing_pre), priority=0    , match=(1), action=(next;)
>> 
>> - routes, which have route_table set, are added to lflow with match on previously set value of computed uniq route_table id, stored in OVS register. Like this:
>> 
>>  table=10(lr_in_ip_routing   ), priority=1    , match=(reg7 == 100 && ip4.dst == 0.0.0.0/0), action=(ip.ttl--; reg8[0..15] = 0; reg0 = 169.254.0.1; reg1 = 169.254.0.2; eth.src = 06:00:b1:5f:12:77; outport = "vpc-B15F1277-up"; flags.loopback = 1; next;)
>>  table=10(lr_in_ip_routing   ), priority=1    , match=(reg7 == 110 && ip4.dst == 0.0.0.0/0), action=(ip.ttl--; reg8[0..15] = 0; reg0 = 169.254.0.1; reg1 = 169.254.0.2; eth.src = 06:00:b1:5f:12:77; outport = "vpc-B15F1277-up"; flags.loopback = 1; next;)
>> 
>> 
>> My question is about OVS register choice.
>> 
> 
>> I see in ovn-northd.c described OVS fields and (please correct me, if I’m wrong) it looks like there are no registers available to store route_table id? It looks for me that all registers are busy. What should I do in this particular situation? If I set reg10, for instance, ovn-controller can’t process such match and action, thus it’s an unknown symbol. If I increase the MFF_N_LOG_REGS from 10 to 12, ovn connectivity tests fail and even arp/ip connectivity drops. (see commit: https://github.com/ovn-org/ovn/commit/18e71b5240009c519dc44eb06d101b3edd9dc103 <https://github.com/ovn-org/ovn/commit/18e71b5240009c519dc44eb06d101b3edd9dc103> <https://github.com/ovn-org/ovn/commit/18e71b5240009c519dc44eb06d101b3edd9dc103 <https://github.com/ovn-org/ovn/commit/18e71b5240009c519dc44eb06d101b3edd9dc103>>)
>> 
>> I’m new to OVN development, please help :)
>> Thanks in advance!
> 
> From your email,  I'm not clear about the new feature you're planning
> to add.  But since your question is about registers,  I guess I can
> answer your question.
> 
> Registers R0 - R9 are used by ovn-northd  and regsiters R10 - 15 are
> used by ovn-controller -
> https://github.com/ovn-org/ovn/blob/master/include/ovn/logical-fields.h#L34 <https://github.com/ovn-org/ovn/blob/master/include/ovn/logical-fields.h#L34>
> The reason why the tests failed when your increared the MFF_N_LOG_REGS
> to 12 is because R10 is used by ovn-controller.
> 
> One possible solution I see is to use any register from R2 to R7 to
> store route table id.  But this would restrict the router table ids
> only for IPv4 traffic.
> 
> Looks to me we have run out of OVS registers in the router pipeline.
> 
> Thanks
> Numan
> 
>> 
>> 
>> Regards,
>> Vladislav Odintsov
>> 
>> _______________________________________________
>> dev mailing list
>> dev at openvswitch.org <mailto:dev at openvswitch.org>
>> https://mail.openvswitch.org/mailman/listinfo/ovs-dev <https://mail.openvswitch.org/mailman/listinfo/ovs-dev>