[ovs-dev] FW: performance issue with ovs + dpdk2.0 with vhost
Rao, Ravi
Ravi.Rao at us.fujitsu.com
Tue May 26 18:30:52 UTC 2015
Hi,
If I add flows so that the packet gets looped at the Host itself it works fine. Issue is mainly at the OVS port that is suppose to xmit packets to the guest VM.
Regards,
Ravi
-----Original Message-----
From: Flavio Leitner [mailto:fbl at sysclose.org]
Sent: Tuesday, May 26, 2015 12:34 PM
To: Rao, Ravi
Cc: dev at openvswitch.org
Subject: Re: [ovs-dev] FW: performance issue with ovs + dpdk2.0 with vhost
On Sun, May 24, 2015 at 08:55:59AM -0500, Ravi Rao wrote:
> Hi
> Below are the exact sequence of steps that I followed..
>
> This is what I am trying to Do.
> Below is the setup..
>
> | +----------------------+ |
> | guest | |
> | | |
> | | | guest
> | eth0 L3fwd eth1 | |
> | | | | |
> +---+--------------+---+ __|
> ^ :
> | |
> : v __
> +-----------------+--------------+-----------------+ |
> | | ovs-br0 | |
> | +--------------+ | |
> | ^ : | |
> | +----------+ +---------+ | | host
> | : v | |
> | +--------------+ +--------------+ | |
> | | dpdk0 | ovs-dpdk | dpdk1 | | |
> +---+--------------+------------+--------------+---+ __|
> ^ :
> | |
> : v
> +--------------------------------------------------+
> | |
> | traffic generator |
> | |
> +--------------------------------------------------+|
>
>
> Step1: Use the latest ovs and dpdk2.0 to get the ovs running with 2
> dpdk interfaces that are bound to 2 10GB physical interfaces
> #** Inser the required Modules
> cd /root/dpdk-2.0.0
> modprobe uio
> modprobe cuse
> rmmod igb_uio
> rmmod rte_kni
> insmod x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
>
> #**** Assign the dpdk capable interfaces to igb_uio driver
> tools/dpdk_nic_bind.py --status tools/dpdk_nic_bind.py -b igb_uio
> 0000:02:00.0 tools/dpdk_nic_bind.py -b igb_uio 0000:02:00.1
> tools/dpdk_nic_bind.py --status
>
> #--- Setup the openVswitch
> cd /root/ovs
> pkill -9 ovs
> mkdir -p /usr/local/etc/openvswitch
> mkdir -p /usr/local/var/run/openvswitch rm -rf
> /usr/local/etc/openvswitch/conf.db
> ovsdb/ovsdb-tool create /usr/local/etc/openvswitch/conf.db
> vswitchd/vswitch.ovsschema
>
> #Start ovsdb-server
> ovsdb/ovsdb-server
> --remote=punix:/usr/local/var/run/openvswitch/db.sock
> --remote=db:Open_vSwitch,Open_vSwitch,manager_options --pidfile
> --detach utilities/ovs-vsctl --no-wait init
>
> #Start vswitchd:
> export DB_SOCK=/usr/local/var/run/openvswitch/db.sock
> rm /dev/vhost-net
> vswitchd/ovs-vswitchd --dpdk -c 0x3 -n 4 --socket-mem 1024,0 --
> unix:$DB_SOCK --pidfile --detach
>
> #Add bridge & ports
> utilities/ovs-vsctl add-br ovs-br0 -- set bridge ovs-br0
> datapath_type=netdev utilities/ovs-vsctl add-port ovs-br0 dpdk0 -- set
> Interface dpdk0 type=dpdk utilities/ovs-vsctl add-port ovs-br0 dpdk1
> -- set Interface dpdk1 type=dpdk
>
> Step2: Create the dpdkvhost interfaces and bring up the guestVM using
> QEMU export DPDK_DIR=/root/dpdk-2.0.0 insmod
> $DPDK_DIR/lib/librte_vhost/eventfd_link/eventfd_link.ko
> cd /root/ovs
> utilities/ovs-vsctl add-port ovs-br0 dpdkvhost0 -- set Interface
> dpdkvhost0 type=dpdkvhost utilities/ovs-vsctl add-port ovs-br0
> dpdkvhost1 -- set Interface dpdkvhost1 type=dpdkvhost
>
> #**** Start the guest ubuntu VM1 from a terminal that is logged in as
> root
> qemu-system-x86_64 --enable-kvm -k fr -m 1G \
> -cpu host -smp cores=2,threads=1,sockets=1 \
> -serial telnet::4444,server,nowait -monitor
> telnet::5555,server,nowait \
> -hda /root/VMs/images/ubuntu-14.04-template.qcow2 \
> -object
> memory-backend-file,id=mem,size=1G,mem-path=/mnt/huge_1GB,share=on \
> -numa node,memdev=mem \
> -netdev
> type=tap,id=dpdkvhost0,script=no,downscript=no,ifname=dpdkvhost0,vhost=on \
> -device
> virtio-net-pci,netdev=dpdkvhost0,mac=52:54:00:12:34:56,csum=off,gso=of
> f,guest_tso4=off,guest_tso6=off,guest_ecn=off
> \
> -netdev
> type=tap,id=dpdkvhost1,script=no,downscript=no,ifname=dpdkvhost1,vhost=on \
> -device
> virtio-net-pci,netdev=dpdkvhost1,mac=52:54:00:12:34:57,csum=off,gso=of
> f,guest_tso4=off,guest_tso6=off,guest_ecn=off
> \
> -device ne2k_pci,mac=DE:AD:DE:01:02:03,netdev=user.0 -netdev
> user,id=user.0,hostfwd=tcp::2222-:22 &
>
> # **** Add flows between ports
> utilities/ovs-ofctl del-flows ovs-br0
> utilities/ovs-ofctl add-flow ovs-br0 in_port=1,action=output:3
> utilities/ovs-ofctl add-flow ovs-br0 in_port=2,action=output:4
> utilities/ovs-ofctl add-flow ovs-br0 in_port=3,action=output:1
> utilities/ovs-ofctl add-flow ovs-br0 in_port=4,action=output:2
>
> Once I complete the above settings. I log into the VM and enable ipv4
> forwarding so that it could do l3 fwd between eth0 and eth1
>
> The issue I am seeing is when I start pumping packets on IXIA port
> connected to physical port dpdk0 I see lots of tx_errors on dpdk0.
> I can only pass about 1000 pps without getting any errors. Is there
> anything I am doing wrong or missing in the above setup.
I looked at your setup and I am not finding anything wrong.
Maybe there is an error inside of guest that packets are being flooded? What happens if you change the flows in such way that it loops the packet back to IXIA instead of guest? Does it work?
fbl
>
> *Qemu vesion is 2.2.1**
> *
> Thanks & Regards,
> Ravi..
>
> On 05/22/15 19:38, Flavio Leitner wrote:
> >On Fri, May 15, 2015 at 02:07:07PM +0000, Rao, Ravi wrote:
> >>Hi All,
> >> I am trying to get a Guest VM connected to the dpdkvhost
> >> interface on a host which has the ovs running from the latest ovs
> >> git and dpdk2.0. Looks like I am missing something as 95% of
> >> traffic is not getting to the VM. Can one of you please let me
> >> know which mailing list I should be posting the details for
> >> getting a resolution. Is it this dpdk list OR would it be the
> >> openvswitch list?
> >This is the right place, but it would be great if you could tell us
> >the configuration, how you're testing and the qemu version too.
> >
> >fbl
> >
>
> _______________________________________________
> dev mailing list
> dev at openvswitch.org
> http://openvswitch.org/mailman/listinfo/dev
More information about the dev
mailing list