[ovs-discuss] Phy-VM over OVS-DPDK no connectivity
Flavio Leitner
fbl at sysclose.org
Wed Mar 13 19:54:29 UTC 2019
On Tue, Mar 05, 2019 at 03:17:04PM +0530, ppnaik wrote:
> Hi All,
>
> Our setup is as follows:
>
> We have two servers which are connected peer to peer over 40G interfaces.
>
> On one server we have setup OVS and added the physical 40G interface as a
> DPDK interface to the ovs bridge.
>
> We created another dpdkvhostuser interface for the VM. We added this
> interface to the VM (by editing the XML). We are able to see this interface
> inside the VM and have configure IP to the interface.
>
> We want to communicate between the other server and VM inside this server
> through the OVS interface created for the VM.
>
> The steps we followed (on the server with OVS) are:
>
> modprobe uio
>
> cd /usr/src/dpdk-18.11/x86_64-native-linuxapp-gcc/kmod/
>
> insmod igb_uio.ko
VFIO is safer than UIO.
Please check out our guides at:
https://github.com/openvswitch/ovs/blob/master/Documentation/intro/install/dpdk.rst
https://github.com/openvswitch/ovs/blob/master/Documentation/howto/dpdk.rst
You might want to try PHY-to-PHY first to make sure your basic
plumbing is okay and then add the VM.
fbl
> cd /usr/src/dpdk-18.11/usertools/
> ./dpdk-devbind.py --bind=igb_uio 0000:81:00.1
>
> export PATH=$PATH:/usr/local/share/openvswitch/scripts
> export DB_SOCK=/usr/local/var/run/openvswitch/db.sock
>
> ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock
> --remote=db:Open_vSwitch,Open_vSwitch,manager_options
> --private-key=db:Open_vSwitch,SSL,private_key
> --certificate=db:Open_vSwitch,SSL,certificate
> --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --pidfile --detach
> --log-file
>
> ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true
> ovs-ctl --no-ovsdb-server --db-sock="$DB_SOCK" start
>
> ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev
> ovs-vsctl add-port br0 dpdk-p0 -- set Interface dpdk-p0 type=dpdk
> options:dpdk-devargs=0000:81:00.1
> ovs-vsctl add-port br0 dpdkvhostuser0 -- set Interface dpdkvhostuser0
> type=dpdkvhostuser ofport_request=3
>
> ovs-ofctl add-flow br0 in_port=1,action=output:3
> ovs-ofctl add-flow br0 in_port=3,action=output:1
>
> echo 'vm.nr_hugepages=2048' > /etc/sysctl.d/hugepages.conf
> grep HugePages_ /proc/meminfo
>
> edit VM XML to add this interface:
>
> first line:
> <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
>
> add before </domain> tag:
>
> <qemu:commandline>
> <qemu:arg value='-chardev'/>
> <qemu:arg
> value='socket,id=char1,path=/usr/local/var/run/openvswitch/dpdkvhostuser0'/>
> <qemu:arg value='-netdev'/>
> <qemu:arg
> value='vhost-user,id=mynet1,chardev=char1,vhostforce=on,queues=1'/>
> <qemu:arg value='-device'/>
> <qemu:arg
> value='virtio-net-pci,mac=00:00:00:00:00:02,netdev=mynet1,mq=on,vectors=4'/>
> <qemu:arg value='-m'/>
> <qemu:arg value='4096'/>
> <qemu:arg value='-object'/>
> <qemu:arg value='memory-backend-file,id=mem1,size=4096M,mem-path=/dev/hugepages,share=on'/>
> <qemu:arg value='-mem-prealloc'/>
> <qemu:arg value='-numa'/>
> <qemu:arg value='node,memdev=mem1'/>
> </qemu:commandline>
>
> Please help us resolve this issue. I assumed ping would work between the
> other server and the VM. But it is not working in our case. Also, let us
> know if we are missing some setup step or if there is some misconfiguration.
> If ping would not work can you let us know a way to verify the connectivity?
>
> Thanks,
> Priyanka
>
> _______________________________________________
> discuss mailing list
> discuss at openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
More information about the discuss
mailing list