[ovs-dev] [PATCH 2/2] netdev-dpdk: add support for vhost IOMMU feature

Kavanagh, Mark B mark.b.kavanagh at intel.com
Mon Nov 27 14:24:17 UTC 2017


>From: Jan Scheurich [mailto:jan.scheurich at ericsson.com]
>Sent: Monday, November 27, 2017 2:15 PM
>To: Kavanagh, Mark B <mark.b.kavanagh at intel.com>; Mooney, Sean K
><sean.k.mooney at intel.com>; Kevin Traynor <ktraynor at redhat.com>;
>dev at openvswitch.org
>Cc: maxime.coquelin at redhat.com; Flavio Leitner <fbl at redhat.com>; Franck Baudin
><fbaudin at redhat.com>; Ilya Maximets <i.maximets at samsung.com>; Stokes, Ian
><ian.stokes at intel.com>; Loftus, Ciara <ciara.loftus at intel.com>; Darrell Ball
><dball at vmware.com>; Aaron Conole <aconole at redhat.com>
>Subject: RE: [ovs-dev] [PATCH 2/2] netdev-dpdk: add support for vhost IOMMU
>feature
>
>> >> >> Hi Mark, All,
>> >> >>
>> >> >> I'm thinking about this and whether the current approach provides
>> >> >> more than what is actually needed by users at the cost of making the
>> >> >> user interface more complex.
>> >[Mooney, Sean K] I am personally split on this. To enable iommu support in
>> >openstack with the above interface we would have to do two things. 1 extend
>> >the image metatdata
>> >or flavor extra specs to allow requesting a vIOMMU. Second we would have to
>> >modify os-vif to produce
>> >the add-port command above. Using this interfaces gives us a nice parity in
>> >our api
>> >as we only enable iommu support in ovs if we enable for qemu. E.g. if the
>> >falvor/image does not request
>> >a virtualized iommu we wont declare its support in the options.
>> >> >>
>> >> >> As an alternative, how about having a global other_config (to be set
>> >> >> like vhost-socket-dir can be) for this instead of having to set it
>> >> >> for each individual interface. It would mean that it would only have
>> >> >> to be set once, instead of having this (ugly?!) option every time a
>> >> >> vhost port is added, so it's a less intrusive change and I can't
>> >> >> really think that a user would require to do this per vhostclient
>> >[Mooney, Sean K]  well one taught that instantly comes to mind is if I set
>> >The global other_config option what happens if I do not request a iommu in
>> >qemu
>> >Or I have an old qemu.  If it would have any negative effect on network
>> >connectivity
>> >Or prevent the vm from functioning, that would require the nova scheduler
>to
>> >be
>> >Aware of which node had this option set and take that into account when
>> >placing vms.
>> >I assume if it had no negative effects  then we would not need a option,
>> >global or per
>> >Interface.
>> >> >> interface??? It's pain to have to add this at all for a bug in QEMU
>> >> >> and I'm sure in 1/2/3 years time someone will say that users could
>> >> >> still be using QEMU < 2.9.1 and we can't remove it, so it would be
>> >> >> nice to keep it as discreet as possible as we're going to be stuck
>> >> with it for a while.
>> >> >>
>> >> >> I assume that a user would only use one version of QEMU at a time
>> >> and
>> >> >> would either want or not want this feature. In the worst case, if
>> >> >> that proved completely wrong in the future, then a per interface
>> >> >> override could easily be added. Once there's a way to maintain
>> >> >> backwards compatibility (which there would be) I'd rather err on the
>> >> >> side of introducing just enough enough functionality over increasing
>> >> >> complexity for the user.
>> >> >>
>> >> >> What do you think?
>> >[Mooney, Sean K] I'm not sure that a single qemu version is a valid
>assumption
>> >to make.
>> >At least in an OpenStack context where rolling upgrades are a thing. But
>you
>> >are right
>> >That it will be less common however if it was no existent we would not have
>> >the issue with
>> >Live migration between nodes with different feature sets that is also
>trying
>> >to be addressed this
>> >Cycle. If we add a global config option for iommu support that is yet
>another
>> >thing that needs
>> >To be accounted for during live migration.
>> >> >>
>>
>> Hi Kevin, Jan,
>>
>> Any thoughts on Sean's concerns?
>>
>> I'll hold off on implementation until we have consensus.
>>
>> Thanks,
>> Mark
>
>Here are my 2cts:
>
>As I see it, vIOMMU for vhostuser is a pure infrastructure feature negotiated
>between guest driver,  Qemu and OVS. If both Qemu and OVS support it and the
>driver requests it, it should be used, otherwise not.
>
>My understanding is that the vhostuser library in DPDK 17.11 supports vIOMMU,
>such that OVS could always signal support for this feature. The only reason
>not to do so is to work around the problem that Qemu claims vIOMMU support in
>certain releases but the vhostuser backend implementation is broken (prior to
>2.9.1). For these cases it should be sufficient to globally disable vIOMMU
>support in OVS.
>
>I don't see the need for one OVS instance to interwork with multiple different
>Qemu versions on the same host. And even if that were the case in an upgrade
>scenario, one could keep vIOMMU disabled until the old Qemu is removed.
>
>The specific enabling of vIOMMU per tenant VM port is done by supplying an
>iommu device to the VM in the Libvirt XML and enabling iommu for the device in
>the driver element of the interface section. These are the places that
>Nova/OpenStack would use to control the vIOMMU per VM based on image
>properties or flavor extra specs. I do not see any value to additionally
>configure the iommu support per vhostuser port through OS-VIF.
>
>Conclusion: A global other_config parameter to enable/disable vhostuser IOMMU
>is sufficient. By default this could be OFF for now and changed to ON when
>broken Qemu versions are largely gone.
>
>BR, Jan

Thanks Jan, Kevin for your respective inputs.

I've spoken to Sean about this offline, and we came to the same conclusion regarding the global config option.

I'll proceed with same, as discussed.

Thanks again,
Mark


More information about the dev mailing list