[ovs-dev] Inter-op openvswitch and openfcoe

[Anoob Soman]

On 27/07/15 22:53, Jesse Gross wrote:
> On Mon, Jul 27, 2015 at 9:59 AM, Anoob Soman <anoob.soman at citrix.com> wrote:
>> Hi All,
>>
>> I am aware there had been some discussion on running lldpad and open-fcoe
>> utils (like fcoemon) on top of ovs, but couldn’t figure out a way to
>> successfully run lldpad and fcoemon on top of openvswitch.
>>
>> I did some experimentation with openvswitch and open-fcoe and now I am
>> trying to get opinion from you guys about what would be the best approach to
>> make fcoeuilts run on top of openvswitch. I used openvswitch v2.3.2 for my
>> testing.
>> My experimentation was carried out using XenServer and in my experimentation
>> I could see FIP and LLDP don’t reach lldpad and fcoemon daemons.
>>
>> Bit of background what these daemon are. You can skip this paragraph, if you
>> know about these daemons.
>> FCoE’s lossless ethernet can be achieved using dcb (dcbx protocol). dcb, in
>> open-fcoe, is enabled using lldpad (LLDP agent daemon) which handles dbcx
>> options and much more. dcb needs to be enabled and available on a interface
>> for fcoemon to start fcoe operations.
>>
>> I have figured out why lldpad and fcoemon don’t receive ETH_P_LLDP and
>> ETH_P_FIP packets.
>> lldpad and fcoemon open ETH_P_LLDP and ETH_P_FIP raw sockets respectively.
>> Since the processing of raw socket packets (ptype_base) happens after
>> openvswitch (rx_handler), when running in XenServer any raw socket listening
>> on ethX interface, will not receive any packets, as openvswitch always
>> return RX_HANDLER_CONSUMED. rx_handlers are only set for ethX and xenbrX
>> (bridge interface) doesn’t have any rx_handler set. Ideally, if raw socket
>> is listening on xenbrX interface it should receive the packets. But in case
>> of lldpad it is not possible to make this daemon receive packet on xenbrX
>> interface.
>>
>> lldpad daemon opens PF_LLDP raw socket and it listens on all ports. If you
>> consider XenServer, then it listens on xenbrX (bridge interface) and ethX
>> interface. llpdad also queries for the dcb state, from the netdevice, using
>> dcbnl_ops. As of now xenbrX (bridge interface) doesn’t have any dcbnl_ops,
>> which mean lldpad will only be able to configure, listen, send, receive lldp
>> pdu on ethX interfaces. Without dcb support on xenbrX interface, fcoemon
>> will not start fcoe handling on xenbrX interface. This rules out having
>> these daemons run on top of xenbrX (bridge interface).
>>
>> So I am looking for an interface, similar to linux bridge, where setting an
>> action of drop doesn’t drop the packet instead it returns RX_HANDLER_PASS
>> and ptype_base handler (including packet_handler) can pickup the packet.
>>
>> I did a proof of concept, by having a hack, in netdev_frame_hook()
>> datapath/vport-netdev.c
>> if (skb->protocol == cpu_to_be16(ETH_P_FIP) ||  skb->protocol ==
>> cpu_to_be16(0x88CC))
>> return RX_HANDLER_PASS;
>> and this makes fcoemon and lldpad daemon work as expected.
>>
>> If returning RX_HANDLER_PASS is the right approach to go forward, how do we
>> design this into ovs. Can we have an action called “ignore" which will
>> ignore certain flows from being processed by openvswitch. Is there any other
>> approach that I can take.
> I think there's roughly three possible approaches:
>   #1: Replicate support for these protocols in OVS, which would be able
> to install the appropriate flows directly in the datapath to get these
> packets itself.
This is certainly possible and cleaner as well. But I think its a huge 
amount to work to replicate the support in ovs.
>   #2: Add some mechanism where these daemons can talk to OVS and have
> them cooperate to get the appropriate packets, while still keeping the
> logic in the original daemon.
Does it mean that daemon would be like openflow controllers and can use 
packet_[in/out] to receive/send daemon specific traffic ?
>   #3: The method that you are describing, where OVS has a mechanism to
> allow the existing daemons to work unmodified.
>
> There is already a little LLDP code in OVS to support certain
> functionality (auto-attach), so I guess that means that we have
> roughly started down path #1. However, in that case, I think that was
> new functionality that did not also exist in lldpad.
I haven't really thought about how approach #3 would interact with 
existing lldp code. Let me think about this at bit more.
> I think the main issue with #3 is finding a way to make things work
> together cleanly. If OVS is managing the switch configuration, how
> does it interact with these external daemons? What happens if they are
> used together with the OVS LLDP code? Can this be handled
> automatically without making the user understand the details of the
> kernel implementation?
Thanks,
Anoob.