[ovs-dev] Issue when using ovn with Openstack
Chen Li
lichen.hangzhou at gmail.com
Wed Jul 20 03:44:27 UTC 2016
Hi list,
I have an all-in-one devstack environment with ovn enabled.
I create a neutron network.
Create a port A from the network with secgroup A
Create a vm from the network with secgroup B.
Secgroup B has both ICMP and tcp 22 enabled.
Then I try to ping the VM from the dhcp namespace, since the Secgroup B has
enabled ICMP, I suppose this should work. But, unfortunately, this do not
work. And, the ssh failed too.
Anyone can help me to solve this issue ?
I did some basic checks and looks like flows are missing in table 52.
Here are flows in table 52:
sudo ovs-ofctl dump-flows br-int |grep table=52
cookie=0x0, duration=7766.195s, table=52, n_packets=0, n_bytes=0,
idle_age=7766, priority=65535,icmp6,metadata=0x4,icmp_type=135,icmp_code=0
actions=resubmit(,53)
cookie=0x0, duration=7766.195s, table=52, n_packets=0, n_bytes=0,
idle_age=7766, priority=65535,icmp6,metadata=0x4,icmp_type=136,icmp_code=0
actions=resubmit(,53)
cookie=0x0, duration=7766.195s, table=52, n_packets=4, n_bytes=1474,
idle_age=7744, priority=2002,udp,reg15=0x2,metadata=0x4,nw_src=
192.168.0.0/24,tp_src=67,tp_dst=68
actions=load:0x1->NXM_NX_REG0[1],resubmit(,53)
cookie=0x0, duration=7557.209s, table=52, n_packets=2, n_bytes=759,
idle_age=7548, priority=2002,udp,reg15=0x3,metadata=0x4,nw_src=
192.168.0.0/24,tp_src=67,tp_dst=68
actions=load:0x1->NXM_NX_REG0[1],resubmit(,53)
cookie=0x0, duration=7766.195s, table=52, n_packets=0, n_bytes=0,
idle_age=7766, priority=2001,ipv6,reg15=0x2,metadata=0x4 actions=drop
cookie=0x0, duration=7766.195s, table=52, n_packets=2, n_bytes=676,
idle_age=7548, priority=2001,ip,reg15=0x2,metadata=0x4 actions=drop
cookie=0x0, duration=7557.209s, table=52, n_packets=0, n_bytes=0,
idle_age=7557, priority=2001,ipv6,reg15=0x3,metadata=0x4 actions=drop
cookie=0x0, duration=7557.209s, table=52, n_packets=3979, n_bytes=389774,
idle_age=413, priority=2001,ip,reg15=0x3,metadata=0x4 actions=drop
cookie=0x0, duration=7766.195s, table=52, n_packets=0, n_bytes=0,
idle_age=7766, priority=1,ipv6,metadata=0x4
actions=load:0x1->NXM_NX_REG0[1],resubmit(,53)
cookie=0x0, duration=7766.195s, table=52, n_packets=8, n_bytes=2733,
idle_age=7548, priority=1,ip,metadata=0x4
actions=load:0x1->NXM_NX_REG0[1],resubmit(,53)
cookie=0x0, duration=7926.354s, table=52, n_packets=0, n_bytes=0,
idle_age=7926, priority=0,metadata=0x1 actions=resubmit(,53)
cookie=0x0, duration=7790.771s, table=52, n_packets=129, n_bytes=5418,
idle_age=408, priority=0,metadata=0x4 actions=resubmit(,53)
Here are steps how I find flows are missing in table52:
ovs-dpctl show
port 0: ovs-system (internal)
port 1: br-int (internal)
port 2: tap446ef382-f0 (internal)
port 3: tapc7c9f581-2d (internal) => the dhcp port for the testing
network
port 4: o-hm0 (internal) => the port create from the
testing network with security group A
port 5: tap275a5a25-79 => the port for the vm in the
testing network with security group B
sudo ip netns exec qdhcp-e8586b01-6441-4c3d-a90d-91bb0a54ec80 arp -n
Address HWtype HWaddress Flags Mask
Iface
192.168.0.6 ether fa:16:3e:40:85:41 C
tapc7c9f581-2d
192.168.0.12 ether fa:16:3e:5c:fe:86 C
tapc7c9f581-2d
sudo ip netns exec qdhcp-e8586b01-6441-4c3d-a90d-91bb0a54ec80 ping
192.168.0.12 => This is the IP for the VM.
PING 192.168.0.12 (192.168.0.12) 56(84) bytes of data.
ovs-dpctl dump-flows
recirc_id(0),in_port(3),eth(src=fa:16:3e:b6:f6:25,dst=fa:16:3e:5c:fe:86),eth_type(0x0806),arp(sip=192.168.0.2,tip=192.168.0.12,op=1/0xff,sha=fa:16:3e:b6:f6:25,tha=00:00:00:00:00:00),
packets:0, bytes:0, used:never,
actions:userspace(pid=4294958325,slow_path(action))
recirc_id(0),in_port(3),eth(src=00:00:00:00:00:00/01:00:00:00:00:00,dst=fa:16:3e:5c:fe:86),eth_type(0x0800),ipv4(src=
192.168.0.0/255.255.255.0,proto=1,frag=no), packets:14, bytes:1372,
used:0.974s, actions:drop
sudo ovs-appctl ofproto/trace
"recirc_id(0),in_port(3),eth(src=00:00:00:00:00:00/01:00:00:00:00:00,dst=fa:16:3e:5c:fe:86),eth_type(0x0800),ipv4(src=
192.168.0.0/255.255.255.0,proto=1,frag=no)"
=> This produce a long output, and here are the end of the output:
OpenFlow actions=resubmit(,52)
Resubmitted flow: unchanged
Resubmitted regs: reg0=0x1 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0
reg6=0x0 reg7=0x0 reg8=0x0 reg9=0x0 reg10=0x0 reg11=0x0 reg12=0x0
reg13=0x0 reg14=0x1 reg15=0x3
Resubmitted odp: drop
Resubmitted megaflow:
recirc_id=0,icmp,reg0=0,reg1=0,reg2=0,reg3=0,reg4=0,reg5=0,reg6=0,reg7=0,reg8=0,reg9=0,reg14=0,reg15=0,metadata=0,in_port=4,vlan_tci=0x0000/0x1000
,dl_src=00:00:00:00:00:00/01:00:00:00:00:00,dl_dst=fa:16:3e:5c:fe:86,nw_src=
192.168.0.0/24,nw_frag=no
Rule: table=52 cookie=0 priority=2001,ip,reg15=0x3,metadata=0x4
OpenFlow actions=drop
Final flow:
icmp,reg0=0x1,reg14=0x1,reg15=0x3,metadata=0x4,in_port=4,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=fa:16:3e:5c:fe:86,nw_src=192.168.0.0,nw_d
st=0.0.0.0,nw_tos=0,nw_ecn=0,nw_ttl=0,icmp_type=0,icmp_code=0
Megaflow:
recirc_id=0,icmp,in_port=4,vlan_tci=0x0000/0x1000,dl_src=00:00:00:00:00:00/01:00:00:00:00:00,dl_dst=fa:16:3e:5c:fe:86,nw_src=
192.168.0.0/24,nw_fra
g=no
Datapath actions: drop
Here are some output from OVN commands:
sudo ovn-nbctl show
switch 3ce05ec4-f591-4ca7-ba54-dc4fab2ffd1b
(neutron-e8586b01-6441-4c3d-a90d-91bb0a54ec80)
port 2c713237-ffc7-4ff1-9e4c-95c1337545e6
addresses: ["fa:16:3e:40:85:41 192.168.0.6"]
port c7c9f581-2db9-4b06-86c6-bde2d1aa8ffb
addresses: ["fa:16:3e:b6:f6:25 192.168.0.2"]
port 275a5a25-794f-47b9-9b04-8a8da053c143
addresses: ["fa:16:3e:5c:fe:86 192.168.0.12"]
ovn-nbctl acl-list 3ce05ec4-f591-4ca7-ba54-dc4fab2ffd1b
from-lport 1002 (inport == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip4)
allow-related
from-lport 1002 (inport == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip4
&& (ip4.dst == 255.255.255.255 || ip4.dst == 192.168.0.0/24) && udp &&
udp.src == 68 && udp.dst == 67) allow
from-lport 1002 (inport == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip6)
allow-related
from-lport 1002 (inport == "2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip4)
allow-related
from-lport 1002 (inport == "2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip4
&& (ip4.dst == 255.255.255.255 || ip4.dst == 192.168.0.0/24) && udp &&
udp.src == 68 && udp.dst == 67) allow
from-lport 1002 (inport == "2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip6)
allow-related
from-lport 1001 (inport == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip)
drop
from-lport 1001 (inport == "2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip)
drop
to-lport 1002 (outport == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip4
&& icmp4) allow-related
to-lport 1002 (outport == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip4
&& ip4.src == 192.168.0.0/24 && udp && udp.src == 67 && udp.dst == 68) allow
to-lport 1002 (outport == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip4
&& tcp && tcp.dst == 22) allow-related
to-lport 1002 (outport == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip4
&& tcp && tcp.dst == 9443) allow-related
to-lport 1002 (outport == "2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip4
&& ip4.src == 192.168.0.0/24 && udp && udp.src == 67 && udp.dst == 68) allow
to-lport 1002 (outport == "2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip4
&& udp && udp.dst == 5555) allow-related
to-lport 1001 (outport == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip)
drop
to-lport 1001 (outport == "2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip)
drop
ovn-sbctl lflow-list | grep ls_out_acl
table=4 (ls_out_acl ), priority=0 , match=(1), action=(next;)
table=4 (ls_out_acl ), priority=0 , match=(1), action=(next;)
table=4 (ls_out_acl ), priority=65535, match=(!ct.est && ct.rel
&& !ct.new && !ct.inv), action=(next;)
table=4 (ls_out_acl ), priority=65535, match=(ct.est && !ct.rel
&& !ct.new && !ct.inv), action=(next;)
table=4 (ls_out_acl ), priority=65535, match=(ct.inv),
action=(drop;)
table=4 (ls_out_acl ), priority=65535, match=(nd), action=(next;)
table=4 (ls_out_acl ), priority=2002 , match=(ct.new && (outport
== "275a5a25-794f-47b9-9b04-8a8da053c143" && ip4 && icmp4)),
action=(reg0[1] = 1; next;)
table=4 (ls_out_acl ), priority=2002 , match=(ct.new && (outport
== "275a5a25-794f-47b9-9b04-8a8da053c143" && ip4 && tcp && tcp.dst == 22)),
action=(reg0[1] = 1; next;)
table=4 (ls_out_acl ), priority=2002 , match=(ct.new && (outport
== "275a5a25-794f-47b9-9b04-8a8da053c143" && ip4 && tcp && tcp.dst ==
9443)), action=(reg0[1] = 1; next;)
table=4 (ls_out_acl ), priority=2002 , match=(ct.new && (outport
== "2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip4 && udp && udp.dst ==
5555)), action=(reg0[1] = 1; next;)
table=4 (ls_out_acl ), priority=2002 , match=(outport ==
"275a5a25-794f-47b9-9b04-8a8da053c143" && ip4 && ip4.src == 192.168.0.0/24
&& udp && udp.src == 67 && udp.dst == 68), action=(reg0[1] = 1; next;)
table=4 (ls_out_acl ), priority=2002 , match=(outport ==
"2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip4 && ip4.src == 192.168.0.0/24
&& udp && udp.src == 67 && udp.dst == 68), action=(reg0[1] = 1; next;)
table=4 (ls_out_acl ), priority=2001 , match=(outport ==
"275a5a25-794f-47b9-9b04-8a8da053c143" && ip), action=(drop;)
table=4 (ls_out_acl ), priority=2001 , match=(outport ==
"2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip), action=(drop;)
table=4 (ls_out_acl ), priority=1 , match=(ip),
action=(reg0[1] = 1; next;)
table=4 (ls_out_acl ), priority=0 , match=(1), action=(next;)
The last commit in my ovs code:
commit 7efb1e09bb06270248d29c787978593b57101d4f
Author: Pravin B Shelar <pshelar at ovn.org>
Date: Sun Jul 17 19:24:07 2016 -0700
datapath: Add support for kernel 4.5
Signed-off-by: Pravin B Shelar <pshelar at ovn.org>
Acked-by: Jesse Gross <jesse at kernel.org>
Here are some detailed information from openstack:
1. The vm:
nova show test01
+--------------------------------------+----------------------------------------------------------------+
| Property |
Value |
+--------------------------------------+----------------------------------------------------------------+
| OS-DCF:diskConfig |
MANUAL |
| OS-EXT-AZ:availability_zone |
nova |
| OS-EXT-SRV-ATTR:host | LB-dev-chenli |
| OS-EXT-SRV-ATTR:hostname |
test01 |
| OS-EXT-SRV-ATTR:hypervisor_hostname | LB-dev-chenli |
| OS-EXT-SRV-ATTR:instance_name |
instance-00000001 |
| OS-EXT-SRV-ATTR:kernel_id |
261ca209-430e-4b8f-ac39-0e397df30a46 |
| OS-EXT-SRV-ATTR:launch_index |
0 |
| OS-EXT-SRV-ATTR:ramdisk_id |
04b6a65d-3cff-4eaf-b30b-582caa2379d7 |
| OS-EXT-SRV-ATTR:reservation_id |
r-wehkr5gi |
| OS-EXT-SRV-ATTR:root_device_name |
/dev/vda |
| OS-EXT-SRV-ATTR:user_data |
- |
| OS-EXT-STS:power_state |
1 |
| OS-EXT-STS:task_state |
- |
| OS-EXT-STS:vm_state |
active |
| OS-SRV-USG:launched_at |
2016-07-20T01:18:48.000000 |
| OS-SRV-USG:terminated_at |
- |
| accessIPv4
| |
| accessIPv6
| |
| config_drive |
True |
| created |
2016-07-20T01:18:42Z |
| description |
- |
| flavor | m1.tiny
(1) |
| hostId |
36ef28d2b661e38d2d07645d814903a15d62da769828b57029306ec0 |
| host_status |
UP |
| id |
27264d62-6a7c-4fe9-be81-c06fca56ec00 |
| image | cirros-0.3.4-x86_64-uec
(aa86e8b5-0699-46a0-a624-7af794b21404) |
| key_name |
- |
| lb-mgmt-net network |
192.168.0.12 |
| locked |
False |
| metadata |
{} |
| name |
test01 |
| os-extended-volumes:volumes_attached |
[] |
| progress |
0 |
| security_groups |
lb-mgmt-sec-grp |
| status |
ACTIVE |
| tags |
[] |
| tenant_id |
73aebe8aa8ab41f58d5e375a03e279bf |
| updated |
2016-07-20T01:18:48Z |
| user_id |
53f8c8e491e94d2fa9210f3a8e6a85e4 |
+--------------------------------------+----------------------------------------------------------------+
2. the security group:
neutron security-group-show lb-mgmt-sec-grp
+----------------------+--------------------------------------------------------------------+
| Field |
Value |
+----------------------+--------------------------------------------------------------------+
| description
| |
| id |
fbc0c663-f32d-4ddf-9bef-eabac9dfd8ee |
| name |
lb-mgmt-sec-grp |
| security_group_rules |
{ |
| | "remote_group_id":
null, |
| | "direction":
"ingress", |
| | "protocol":
"icmp", |
| | "description":
"", |
| | "ethertype":
"IPv4", |
| | "remote_ip_prefix":
null, |
| | "port_range_max":
null, |
| | "security_group_id":
"fbc0c663-f32d-4ddf-9bef-eabac9dfd8ee", |
| | "port_range_min":
null, |
| | "tenant_id":
"73aebe8aa8ab41f58d5e375a03e279bf", |
| | "id":
"140677a5-5308-48b2-a5a2-bb5e17994ed5" |
| |
} |
| |
{ |
| | "remote_group_id":
null, |
| | "direction":
"ingress", |
| | "protocol":
"tcp", |
| | "description":
"", |
| | "ethertype":
"IPv4", |
| | "remote_ip_prefix":
null, |
| | "port_range_max":
22, |
| | "security_group_id":
"fbc0c663-f32d-4ddf-9bef-eabac9dfd8ee", |
| | "port_range_min":
22, |
| | "tenant_id":
"73aebe8aa8ab41f58d5e375a03e279bf", |
| | "id":
"39fccc0c-f832-497a-b03d-fa0e40e3f407" |
| |
} |
| |
{ |
| | "remote_group_id":
null, |
| | "direction":
"egress", |
| | "protocol":
null, |
| | "description":
"", |
| | "ethertype":
"IPv6", |
| | "remote_ip_prefix":
null, |
| | "port_range_max":
null, |
| | "security_group_id":
"fbc0c663-f32d-4ddf-9bef-eabac9dfd8ee", |
| | "port_range_min":
null, |
| | "tenant_id":
"73aebe8aa8ab41f58d5e375a03e279bf", |
| | "id":
"54d134c0-f4bc-4f3d-bf49-0e1d0ac9ef1c" |
| |
} |
| |
{ |
| | "remote_group_id":
null, |
| | "direction":
"ingress", |
| | "protocol":
"tcp", |
| | "description":
"", |
| | "ethertype":
"IPv4", |
| | "remote_ip_prefix":
null, |
| | "port_range_max":
9443, |
| | "security_group_id":
"fbc0c663-f32d-4ddf-9bef-eabac9dfd8ee", |
| | "port_range_min":
9443, |
| | "tenant_id":
"73aebe8aa8ab41f58d5e375a03e279bf", |
| | "id":
"b3e00b04-d398-450b-b1cf-b92fd3dc37a1" |
| |
} |
| |
{ |
| | "remote_group_id":
null, |
| | "direction":
"egress", |
| | "protocol":
null, |
| | "description":
"", |
| | "ethertype":
"IPv4", |
| | "remote_ip_prefix":
null, |
| | "port_range_max":
null, |
| | "security_group_id":
"fbc0c663-f32d-4ddf-9bef-eabac9dfd8ee", |
| | "port_range_min":
null, |
| | "tenant_id":
"73aebe8aa8ab41f58d5e375a03e279bf", |
| | "id":
"c528b1cf-b065-4498-986c-13adac4c2a0a" |
| |
} |
| tenant_id |
73aebe8aa8ab41f58d5e375a03e279bf |
+----------------------+--------------------------------------------------------------------+
More information about the dev
mailing list