[ovs-discuss] Phy-VM over OVS-DPDK no connectivity

Flavio Leitner fbl at sysclose.org
Wed Mar 13 19:54:29 UTC 2019


On Tue, Mar 05, 2019 at 03:17:04PM +0530, ppnaik wrote:
> Hi All,
> 
> Our setup is as follows:
> 
> We have two servers which are connected peer to peer over 40G interfaces.
> 
> On one server we have setup OVS and added the physical 40G interface as a
> DPDK interface to the ovs bridge.
> 
> We created another dpdkvhostuser interface for the VM. We added this
> interface to the VM (by editing the XML). We are able to see this interface
> inside the VM and have configure IP to the interface.
> 
> We want to communicate between the other server and VM inside this server
> through the OVS interface created for the VM.
> 
> The steps we followed (on the server with OVS) are:
> 
> modprobe uio
> 
> cd /usr/src/dpdk-18.11/x86_64-native-linuxapp-gcc/kmod/
> 
> insmod igb_uio.ko

VFIO is safer than UIO.

Please check out our guides at:
https://github.com/openvswitch/ovs/blob/master/Documentation/intro/install/dpdk.rst
https://github.com/openvswitch/ovs/blob/master/Documentation/howto/dpdk.rst

You might want to try PHY-to-PHY first to make sure your basic
plumbing is okay and then add the VM.
fbl



> cd /usr/src/dpdk-18.11/usertools/
> ./dpdk-devbind.py --bind=igb_uio 0000:81:00.1
> 
>  export PATH=$PATH:/usr/local/share/openvswitch/scripts
>  export DB_SOCK=/usr/local/var/run/openvswitch/db.sock
> 
>  ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock
> --remote=db:Open_vSwitch,Open_vSwitch,manager_options
> --private-key=db:Open_vSwitch,SSL,private_key
> --certificate=db:Open_vSwitch,SSL,certificate
> --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --pidfile --detach
> --log-file
> 
>  ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true
>  ovs-ctl --no-ovsdb-server --db-sock="$DB_SOCK" start
> 
> ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev
> ovs-vsctl add-port br0 dpdk-p0 -- set Interface dpdk-p0 type=dpdk
> options:dpdk-devargs=0000:81:00.1
> ovs-vsctl add-port br0 dpdkvhostuser0     -- set Interface dpdkvhostuser0
> type=dpdkvhostuser ofport_request=3
> 
> ovs-ofctl add-flow br0 in_port=1,action=output:3
> ovs-ofctl add-flow br0 in_port=3,action=output:1
> 
> echo 'vm.nr_hugepages=2048' > /etc/sysctl.d/hugepages.conf
> grep HugePages_ /proc/meminfo
> 
> edit VM XML to add this interface:
> 
> first line:
> <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
> 
> add before </domain> tag:
> 
> <qemu:commandline>
>     <qemu:arg value='-chardev'/>
>     <qemu:arg
> value='socket,id=char1,path=/usr/local/var/run/openvswitch/dpdkvhostuser0'/>
>     <qemu:arg value='-netdev'/>
>     <qemu:arg
> value='vhost-user,id=mynet1,chardev=char1,vhostforce=on,queues=1'/>
>     <qemu:arg value='-device'/>
>     <qemu:arg
> value='virtio-net-pci,mac=00:00:00:00:00:02,netdev=mynet1,mq=on,vectors=4'/>
>     <qemu:arg value='-m'/>
>     <qemu:arg value='4096'/>
>     <qemu:arg value='-object'/>
>     <qemu:arg value='memory-backend-file,id=mem1,size=4096M,mem-path=/dev/hugepages,share=on'/>
>     <qemu:arg value='-mem-prealloc'/>
>     <qemu:arg value='-numa'/>
>     <qemu:arg value='node,memdev=mem1'/>
>   </qemu:commandline>
> 
> Please help us resolve this issue. I assumed ping would work between the
> other server and the VM. But it is not working in our case. Also, let us
> know if we are missing some setup step or if there is some misconfiguration.
> If ping would not work can you let us know a way to verify the connectivity?
> 
> Thanks,
> Priyanka
> 
> _______________________________________________
> discuss mailing list
> discuss at openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss



More information about the discuss mailing list