[ovs-discuss] It doesn't work when Packet-Out to a Group and in_port=Controller, where the type of group is "select".
Han Zhou
hzhou at ovn.org
Fri May 22 06:19:57 UTC 2020
Thanks for the test scenarios. It seems to be a problem related to dp_hash
when in_port=CONTROLLER. I guess it would work if you specify
selection_method=hash in the group. Could you try that? By default it is
using dp_hash, which requires recirculation as the trace output shows. It
is possible that the dp_hash didn't get any value (0) from kernel datapath
so that it would keep recirculating, something similar as [0]. However, it
needs to be verified by ovs-dpctl dump-flows (instead of ovs-ofctl
dump-flows) and dmesg | tail. I didn't find it included in your logs. This
command needs to be executed immediately after the packet-out before the
cache is cleared.
Although I suspect it could be the recirculation problem, I don't think it
is the exact same root cause of [0] because in your case the packet doesn't
need to be slowpathed, so I guess the patch would not help for your case,
and I can't explain yet why in_port=LOCAL works. I will stop speculating
before getting more information.
[0] -
https://patchwork.ozlabs.org/project/openvswitch/patch/1589527067-91901-1-git-send-email-hzhou@ovn.org/
On Thu, May 21, 2020 at 9:58 PM 陳奕任 <yiren at cs.nctu.edu.tw> wrote:
> Hello,
>
> Thanks for your quick response. The following contains 4 scenarios (0 ~
> 3):
>
> scenario 0: packet-out to "select" group and set in_port=CONTROLLER
> scenario 1: packet-out to "fast-failover" group and set
> in_port=CONTROLLER
> scenario 2: packet-out to "SELECT" group and set in_port=LOCAL
> scenario 3: packet-out to "TABLE" port and set in_port=CONTROLLER
>
> In scenario 1 and 2, my composed packets are sent, but they cannot be sent
> in scenario 0 and 3.
>
> ====
> root at host1# ovs-ofctl -O OpenFlow15 show adapter
> OFPT_FEATURES_REPLY (OF1.5) (xid=0x2): dpid:0000bad0babe0051
> n_tables:254, n_buffers:0
> capabilities: FLOW_STATS TABLE_STATS PORT_STATS GROUP_STATS QUEUE_STATS
> BUNDLES
> OFPST_PORT_DESC reply (OF1.5) (xid=0x3):
> 1(eth2): addr:06:ef:e9:1a:0c:96
> config: 0
> state: LIVE
> current: 10GB-FD COPPER
> speed: 10000 Mbps now, 0 Mbps max
> LOCAL(adapter): addr:a2:2b:ed:d2:1b:4b
> config: 0
> state: LIVE
> speed: 0 Mbps now, 0 Mbps max
> OFPT_GET_CONFIG_REPLY (OF1.5) (xid=0x9): frags=normal miss_send_len=0
>
>
> *====== scenario 0: packet-out to "select" group and set
> in_port=CONTROLLER ====== *
> ======== part of the code (OpenFlow 1.5)========
> buffer_id=ofproto.OFP_NO_BUFFER
> in_port = ofproto.OFPP_CONTROLLER
> actions = [ parser.OFPActionGroup(1) ]
> msg = parser.OFPPacketOut(datapath=datapath, buffer_id=buffer_id,
> match=parser.OFPMatch(in_port=in_port), actions=actions, data=packet.data)
> datapath.send_msg(msg)
>
> ======== after RyuApp runs, before packet-out ========
> root at host1# ovs-ofctl -O OpenFlow15 dump-flows adapter
> cookie=0x0, duration=11.384s, table=0, n_packets=0, n_bytes=0,
> idle_age=11, priority=8192,tcp,in_port=CONTROLLER,tp_src=8080
> actions=group:1
> cookie=0x0, duration=11.384s, table=0, n_packets=0, n_bytes=0,
> idle_age=11, priority=4096,tcp,in_port=eth2,tp_dst=8080
> actions=CONTROLLER:65509
>
> root at host1# ovs-ofctl -O OpenFlow15 dump-groups adapter
> OFPST_GROUP_DESC reply (OF1.5) (xid=0x2):
> group_id=1,type=select,bucket=bucket_id:0,actions=output:eth2
>
> root at host1# ovs-ofctl -O OpenFlow15 dump-group-stats adapter
> OFPST_GROUP reply (OF1.5) (xid=0x6):
>
> group_id=1,duration=94.104s,ref_count=1,packet_count=0,byte_count=0,bucket0:packet_count=0,byte_count=0
>
> root at host1# ovs-appctl ofproto/trace-packet-out adapter
> 'tcp,in_port=CONTROLLER,tp_src=8080' 'group:1'
> Flow:
> tcp,in_port=CONTROLLER,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,nw_src=0.0.0.0,nw_dst=0.0.0.0,nw_tos=0,nw_ecn=0,nw_ttl=0,tp_src=8080,tp_dst=0,tcp_flags=0
>
> bridge("adapter")
> -----------------
> group:1
> -> no live bucket
>
> Final flow: unchanged
> Megaflow: recirc_id=0,eth,ip,in_port=CONTROLLER,nw_frag=no
> Datapath actions: hash(l4(0)),recirc(0x1)
>
> ======== after packet-out (telnet to port 8080 from other host) ========
> root at host1# ovs-ofctl -O OpenFlow15 dump-flows adapter
> cookie=0x0, duration=148.805s, table=0, n_packets=0, n_bytes=0,
> idle_age=148, priority=8192,tcp,in_port=CONTROLLER,tp_src=8080
> actions=group:1
> cookie=0x0, duration=148.806s, table=0, n_packets=4, n_bytes=296,
> idle_age=6, priority=4096,tcp,in_port=eth2,tp_dst=8080
> actions=CONTROLLER:65509
>
> root at host1# ovs-ofctl -O OpenFlow15 dump-group-stats adapter
> OFPST_GROUP reply (OF1.5) (xid=0x6):
>
> group_id=1,duration=208.300s,ref_count=1,packet_count=0,byte_count=0,bucket0:packet_count=0,byte_count=0
>
> root at host1# ovs-appctl ofproto/trace-packet-out adapter
> 'tcp,in_port=CONTROLLER,tp_src=8080' 'group:1'
> Flow:
> tcp,in_port=CONTROLLER,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,nw_src=0.0.0.0,nw_dst=0.0.0.0,nw_tos=0,nw_ecn=0,nw_ttl=0,tp_src=8080,tp_dst=0,tcp_flags=0
>
> bridge("adapter")
> -----------------
> group:1
> -> no live bucket
>
> Final flow: unchanged
> Megaflow: recirc_id=0,eth,ip,in_port=CONTROLLER,nw_frag=no
> Datapath actions: hash(l4(0)),recirc(0x18)
>
>
>
> *====== scenario 1: packet-out to "fast-failover" group and set
> in_port=CONTROLLER ====== *
> ======== part of the code (OpenFlow 1.5)========
> buffer_id=ofproto.OFP_NO_BUFFER
> in_port = ofproto.OFPP_CONTROLLER
> actions = [ parser.OFPActionGroup(1) ]
> msg = parser.OFPPacketOut(datapath=datapath, buffer_id=buffer_id,
> match=parser.OFPMatch(in_port=in_port), actions=actions, data=packet.data)
> datapath.send_msg(msg)
>
> ======== after RyuApp runs, before packet-out ========
> root at host1# ovs-ofctl -O OpenFlow15 dump-flows adapter
> cookie=0x0, duration=20.749s, table=0, n_packets=0, n_bytes=0,
> idle_age=20, priority=8192,tcp,in_port=CONTROLLER,tp_src=8080
> actions=group:1
> cookie=0x0, duration=20.748s, table=0, n_packets=0, n_bytes=0,
> idle_age=20, priority=4096,tcp,in_port=eth2,tp_dst=8080
> actions=CONTROLLER:65509
>
> root at host1# ovs-ofctl -O OpenFlow15 dump-groups adapter
> OFPST_GROUP_DESC reply (OF1.5) (xid=0x2):
> group_id=1,type=ff,bucket=bucket_id:0,watch_port:eth2,actions=output:eth2
>
> root at host1# ovs-ofctl -O OpenFlow15 dump-group-stats adapter
> OFPST_GROUP reply (OF1.5) (xid=0x6):
>
> group_id=1,duration=173.081s,ref_count=1,packet_count=0,byte_count=0,bucket0:packet_count=0,byte_count=0
>
> root at host1# ovs-appctl ofproto/trace-packet-out adapter
> 'tcp,in_port=CONTROLLER,tp_src=8080' 'group:1'
> Flow:
> tcp,in_port=CONTROLLER,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,nw_src=0.0.0.0,nw_dst=0.0.0.0,nw_tos=0,nw_ecn=0,nw_ttl=0,tp_src=8080,tp_dst=0,tcp_flags=0
>
> bridge("adapter")
> -----------------
> group:1
> -> using bucket 0
> bucket 0
> output:1
>
> Final flow: unchanged
> Megaflow: recirc_id=0,eth,ip,in_port=CONTROLLER,nw_frag=no
> Datapath actions: 2
>
> ======== after packet-out (telnet to port 8080 from other host) ========
> root at host1# ovs-ofctl -O OpenFlow15 dump-flows adapter
> cookie=0x0, duration=333.717s, table=0, n_packets=0, n_bytes=0,
> idle_age=333, priority=8192,tcp,in_port=CONTROLLER,tp_src=8080
> actions=group:1
> cookie=0x0, duration=333.716s, table=0, n_packets=5, n_bytes=346,
> idle_age=32, priority=4096,tcp,in_port=eth2,tp_dst=8080
> actions=CONTROLLER:65509
>
> root at host1# ovs-ofctl -O OpenFlow15 dump-group-stats adapter
> OFPST_GROUP reply (OF1.5) (xid=0x6):
>
> group_id=1,duration=365.835s,ref_count=1,packet_count=3,byte_count=194,bucket0:packet_count=3,byte_count=194
>
> root at host1# ovs-appctl ofproto/trace-packet-out adapter
> 'tcp,in_port=CONTROLLER,tp_src=8080' 'group:1'
> Flow:
> tcp,in_port=CONTROLLER,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,nw_src=0.0.0.0,nw_dst=0.0.0.0,nw_tos=0,nw_ecn=0,nw_ttl=0,tp_src=8080,tp_dst=0,tcp_flags=0
>
> bridge("adapter")
> -----------------
> group:1
> -> using bucket 0
> bucket 0
> output:1
>
> Final flow: unchanged
> Megaflow: recirc_id=0,eth,ip,in_port=CONTROLLER,nw_frag=no
> Datapath actions: 2
>
>
>
> *====== scenario 2: packet-out to "SELECT" group and set in_port=LOCAL
> ====== *
> ======== part of the code (OpenFlow 1.5)========
> buffer_id=ofproto.OFP_NO_BUFFER
> in_port = ofproto.OFPP_LOCAL
> actions = [ parser.OFPActionGroup(1) ]
> msg = parser.OFPPacketOut(datapath=datapath, buffer_id=buffer_id,
> match=parser.OFPMatch(in_port=in_port), actions=actions, data=packet.data)
> datapath.send_msg(msg)
>
> ======== after RyuApp runs, before packet-out ========
> root at host1# ovs-ofctl -O OpenFlow15 dump-flows adapter
> cookie=0x0, duration=65.134s, table=0, n_packets=0, n_bytes=0,
> idle_age=65, priority=8192,tcp,in_port=LOCAL,tp_src=8080 actions=group:1
> cookie=0x0, duration=65.134s, table=0, n_packets=0, n_bytes=0,
> idle_age=65, priority=4096,tcp,in_port=eth2,tp_dst=8080
> actions=CONTROLLER:65509
>
> root at host1# ovs-ofctl -O OpenFlow15 dump-groups adapter
> OFPST_GROUP_DESC reply (OF1.5) (xid=0x2):
> group_id=1,type=select,bucket=bucket_id:0,actions=output:eth2
>
> root at host1# ovs-ofctl -O OpenFlow15 dump-group-stats adapter
> OFPST_GROUP reply (OF1.5) (xid=0x6):
>
> group_id=1,duration=172.172s,ref_count=1,packet_count=0,byte_count=0,bucket0:packet_count=0,byte_count=0
>
> root at host1# ovs-appctl ofproto/trace-packet-out adapter
> 'tcp,in_port=LOCAL,tp_src=8080' 'group:1'
> Flow:
> tcp,in_port=LOCAL,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,nw_src=0.0.0.0,nw_dst=0.0.0.0,nw_tos=0,nw_ecn=0,nw_ttl=0,tp_src=8080,tp_dst=0,tcp_flags=0
>
> bridge("adapter")
> -----------------
> group:1
> -> no live bucket
>
> Final flow: unchanged
> Megaflow: recirc_id=0,eth,ip,in_port=LOCAL,nw_frag=no
> Datapath actions: hash(l4(0)),recirc(0x1a)
>
> ======== after packet-out (telnet to port 8080 from other host) ========
> root at host1# ovs-ofctl -O OpenFlow15 dump-flows adapter
> cookie=0x0, duration=296.942s, table=0, n_packets=0, n_bytes=0,
> idle_age=296, priority=8192,tcp,in_port=LOCAL,tp_src=8080 actions=group:1
> cookie=0x0, duration=296.942s, table=0, n_packets=5, n_bytes=346,
> idle_age=17, priority=4096,tcp,in_port=eth2,tp_dst=8080
> actions=CONTROLLER:65509
>
> root at host1# ovs-ofctl -O OpenFlow15 dump-group-stats adapter
> OFPST_GROUP reply (OF1.5) (xid=0x6):
>
> group_id=1,duration=334.464s,ref_count=1,packet_count=3,byte_count=194,bucket0:packet_count=3,byte_count=194
>
> root at host1# ovs-ofctl -O OpenFlow15 dump-group-stats adapter
> OFPST_GROUP reply (OF1.5) (xid=0x6):
>
> group_id=1,duration=334.464s,ref_count=1,packet_count=3,byte_count=194,bucket0:packet_count=3,byte_count=194
>
> root at host1# ovs-appctl ofproto/trace-packet-out adapter
> 'tcp,in_port=LOCAL,tp_src=8080' 'group:1'
> Flow:
> tcp,in_port=LOCAL,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,nw_src=0.0.0.0,nw_dst=0.0.0.0,nw_tos=0,nw_ecn=0,nw_ttl=0,tp_src=8080,tp_dst=0,tcp_flags=0
>
> bridge("adapter")
> -----------------
> group:1
> -> no live bucket
>
> Final flow: unchanged
> Megaflow: recirc_id=0,eth,ip,in_port=LOCAL,nw_frag=no
> Datapath actions: hash(l4(0)),recirc(0x1f)
>
>
>
> *====== scenario 3: packet-out to "TABLE" port and set in_port=CONTROLLER
> ====== *
> ======== part of the code (OpenFlow 1.5)========
> buffer_id=ofproto.OFP_NO_BUFFER
> in_port = ofproto.OFPP_CONTROLLER
> actions = [ parser.OFPActionOutput(ofproto.OFPP_TABLE) ]
> msg = parser.OFPPacketOut(datapath=datapath, buffer_id=buffer_id,
> match=parser.OFPMatch(in_port=in_port), actions=actions, data=packet.data)
> datapath.send_msg(msg)
>
> ======== after RyuApp runs, before packet-out ========
> root at host1# ovs-ofctl -O OpenFlow15 dump-flows adapter
> cookie=0x0, duration=21.992s, table=0, n_packets=0, n_bytes=0,
> idle_age=21, priority=8192,tcp,in_port=CONTROLLER,tp_src=8080
> actions=group:1
> cookie=0x0, duration=21.992s, table=0, n_packets=0, n_bytes=0,
> idle_age=21, priority=4096,tcp,in_port=eth2,tp_dst=8080
> actions=CONTROLLER:65509
>
> root at host1# ovs-ofctl -O OpenFlow15 dump-groups adapter
> OFPST_GROUP_DESC reply (OF1.5) (xid=0x2):
> group_id=1,type=select,bucket=bucket_id:0,actions=output:eth2
>
> root at host1# ovs-ofctl -O OpenFlow15 dump-group-stats adapter
> OFPST_GROUP reply (OF1.5) (xid=0x6):
>
> group_id=1,duration=93.852s,ref_count=1,packet_count=0,byte_count=0,bucket0:packet_count=0,byte_count=0
>
> root at host1# ovs-appctl ofproto/trace-packet-out adapter
> 'tcp,in_port=CONTROLLER,tp_src=8080' 'group:1'
> Flow:
> tcp,in_port=CONTROLLER,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,nw_src=0.0.0.0,nw_dst=0.0.0.0,nw_tos=0,nw_ecn=0,nw_ttl=0,tp_src=8080,tp_dst=0,tcp_flags=0
>
> bridge("adapter")
> -----------------
> group:1
> -> no live bucket
>
> Final flow: unchanged
> Megaflow: recirc_id=0,eth,ip,in_port=CONTROLLER,nw_frag=no
> Datapath actions: hash(l4(0)),recirc(0x21)
>
> ======== after packet-out (telnet to port 8080 from other host) ========
> root at host1# ovs-ofctl -O OpenFlow15 dump-flows adapter
> cookie=0x0, duration=242.223s, table=0, n_packets=5, n_bytes=370,
> idle_age=2, priority=8192,tcp,in_port=CONTROLLER,tp_src=8080 actions=group:1
> cookie=0x0, duration=242.223s, table=0, n_packets=5, n_bytes=370,
> idle_age=2, priority=4096,tcp,in_port=eth2,tp_dst=8080
> actions=CONTROLLER:65509
>
> root at host1# ovs-ofctl -O OpenFlow15 dump-groups adapter
> OFPST_GROUP_DESC reply (OF1.5) (xid=0x2):
> group_id=1,type=select,bucket=bucket_id:0,actions=output:eth2
>
> root at host1# ovs-ofctl -O OpenFlow15 dump-group-stats adapter
> OFPST_GROUP reply (OF1.5) (xid=0x6):
>
> group_id=1,duration=279.572s,ref_count=1,packet_count=0,byte_count=0,bucket0:packet_count=0,byte_count=0
>
> root at host1# ovs-appctl ofproto/trace-packet-out adapter
> 'tcp,in_port=CONTROLLER,tp_src=8080' 'group:1'
> Flow:
> tcp,in_port=CONTROLLER,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,nw_src=0.0.0.0,nw_dst=0.0.0.0,nw_tos=0,nw_ecn=0,nw_ttl=0,tp_src=8080,tp_dst=0,tcp_flags=0
>
> bridge("adapter")
> -----------------
> group:1
> -> no live bucket
>
> Final flow: unchanged
> Megaflow: recirc_id=0,eth,ip,in_port=CONTROLLER,nw_frag=no
> Datapath actions: hash(l4(0)),recirc(0x2c)
>
>
>
>
>
> Han Zhou <hzhou at ovn.org> 於 2020年5月22日 週五 上午8:56寫道：
>
>>
>>
>>
>> On Thu, May 21, 2020 at 12:20 PM Ben Pfaff <blp at ovn.org> wrote:
>> >
>> > On Thu, May 21, 2020 at 10:50:21PM +0800, 陳奕任 wrote:
>> > > Hello,
>> > >
>> > >
>> > > ==== What you did that make the problem appear. ====
>> > > I use Ryu as the OpenFlow controller.
>> > > I try to use Packet-Out with Actions=ActionGroup(1) and
>> in_port=Controller,
>> > > where the type of group 1 is "select".
>> > > There is only one bucket and the bucket has only one ActionOutput to a
>> > > physical port, because I am doing a small test for a project.
>> > >
>> > > I use OpenFlow13 firstly, and then I try to use OpenFlow15. both of
>> them
>> > > have the same problem.
>> > >
>> > > ==== What you expected to happen. ====
>> > > My composed packets should be sent
>> > >
>> > > ==== What actually happened. ====
>> > > no packet is sent.
>> > >
>> > > ==== other trials (other information that I think might be relevant.)
>> ====
>> > > (1) If I change the group type to "FastFailover", it works.
>> > > (2) if I change the in_port to "Local", it works.
>> > > (3) if I change the Action=ActionOutput(Table), and add a flow entry
>> > > which match is "in_port=controller" and "actions=group(1)", it doesn't
>> > > work. I use " ovs-ofctl dump-group-stats" to check packet_count and
>> > > byte_count, but they are not increased.
>> > > However, the packet_count and byte_count are increased on the flow
>> entry
>> > > when I use ovs-ofctl dump-flows to check them.
>> >
>> > That's quite odd. Have you tried ofproto/trace-packet-out to see what's
>> > going on? See ovs-vswitchd(8) for documentation.
>>
>> In addition, could you attach the output of "ovs-dpctl dump-flows" and
>> "ovs-ofctl dump-groups" when it happens?
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20200521/142cd276/attachment-0001.html>
More information about the discuss
mailing list