[ovs-dev] [PATCH RFC] ovn: distributed logical port for VM metadata access
Ramu Ramamurthy
ramu.ramamurthy at gmail.com
Mon Apr 11 19:35:58 UTC 2016
A new logical-port-type called "distributed" is introduced with this change.
A distributed logical port is not bound to any chassis. An instance of
the distributed port is created as an ovs internal port at all chasses
where the datapath has at least 1 vif. VIFs on a chassis can communicate
with the locally created distributed port on the same chassis. A distributed
port cannot be reached remotely via tunnels or via localnet from other
logical ports on the logical switch. Unicast traffic originated from
external localnet VMs cannot reach the distributed port.
The distributed logical port allows the implementation of services
in a logical switch that are available at the same IP,mac address on each
chassis to serve local VMs on that logical switch bound to that chassis.
The motivating usecase is the Openstack dhcp-based metadata
access which is described in detail below.
Please suggest any feedback on a) the usecase, b) the approach suggested
here to implement it, c) any alternatives. If it makes sense
I can pursue this work further.
Usecase - DHCP-based metadata access
VMs (using cloudinit) use metadata access at the url http://169.254.169.254.
(https://cloudinit.readthedocs.org/en/latest/topics/datasources.html#ec2)
In Openstack DHCP-based metadata access, the DHCP server advertizes a static route
to the link-local address 169.254.169.254 pointing to the DHCP port's IP address
using DHCP option 249. A neutron-metadata-proxy server listens on the DHCP port
which is also aliased with the link local address 169.254.169.254, and
serves metadata requests from VMs. The metadata-proxy server extracts
the VMs IP address from the http request, and sets the "X-Forwarded-For" and
"X-Neutron-Network-Id" http headers, and forwards the request to the
neutron-metadata-agent which in turn forwards the request to nova.
With proposed native-dhcp implementation in OVN, there is no need to run the
neutron-DHCP-agent in Openstack. Since DHCP-based metadata was implemented by
the dhcp-agent, A new approach is needed to implement the metadata-service.
The following is a proposed approach.
1) The DHCP logical port's type is configured to be "distributed" in the
NorthBound DB. Options on the
logical port are configured to indicate the neutron network id.
2) OVN controller creates internal OVS port on each chassis where VIFs on that
datapath are present. External IDs on that interface denote the neutron-network id,
and dhcp-port ip address, and the link-local ip address to which a static route
was advertized.
3) An ovn-metadata entity monitors changes to the OVSDB interface table, and
upon seeing a new interface does the following using information from the
external-ids on that interface: a) creates namespace and sets the
internal port in the namespace, b) spawns the neutron-metadata-proxy server
listening on that port
Example
1) Create a distributed port (This would be done by neutron-ovn-plugin, upon creation of the DHCP port)
ovn-nbctl lport-add 369611f4-f6a5-4a97-a9a2-75e241dc0803 vif-1
ovn-nbctl lport-set-addresses vif-1 "fe:16:3e:ab:cd:01 10.0.0.100"
ovn-nbctl lport-set-port-security vif-1 "fe:16:3e:ab:cd:01"
ovn-nbctl lport-set-type vif-1 distributed
ovn-nbctl lport-set-options vif-1 name="vif1" network="c827c93e-76f3-4fc7-8e98-1df0b97cf927" mac="fe:16:3e:ab:cd:01" ipv4="10.0.0.100/24" application="neutron-metadata"
2) Boot a VM, after VM boots, set the static route inside the VM (this would be done
by native-DHCP server automatically by injecting a static route in the DHCP offer)
sudo ip route add 169.254.169.254 via 10.0.0.100
3) On the hypervisor (This would be done by a new neutron-ovn-metadata entity running on
each hypervisor upon monitoring ovs interface table and detecting a new vif1 interface
with options indicating that the application is metadata)
sudo ip netns add vif-1
sudo ip link set vif-1 netns vif-1
sudo ip netns exec vif-1 ip link set dev vif1 address fe:16:3e:ab:cd:01
sudo ip netns exec vif-1 ip addr add 10.0.0.100/24 dev vif1
sudo ip netns exec vif-1 ip addr add 169.254.169.254 dev vif1
sudo ip netns exec vif-1 ip link set dev vif1 up
sudo ip netns exec vif-1 /usr/bin/python /usr/bin/neutron-ns-metadata-proxy --pid_file=/opt/stack/data/neutron/external/pids/vif-1.pid --metadata_proxy_socket=/opt/stack/data/neutron/metadata_proxy --network_id=c827c93e-76f3-4fc7-8e98-1df0b97cf927 --state_path=/opt/stack/data/neutron --metadata_port=80 --metadata_proxy_user=1001 --metadata_proxy_group=1001 --debug --verbose
Signed-off-by: Ramu Ramamurthy <ramu.ramamurthy at us.ibm.com>
---
ovn/controller/binding.c | 73 +++++++++++++++++++++++++++++++++++++++++++++--
ovn/controller/physical.c | 10 +++++--
2 files changed, 79 insertions(+), 4 deletions(-)
diff --git a/ovn/controller/binding.c b/ovn/controller/binding.c
index d3ca9c9..abf9556 100644
--- a/ovn/controller/binding.c
+++ b/ovn/controller/binding.c
@@ -50,7 +50,55 @@ binding_register_ovs_idl(struct ovsdb_idl *ovs_idl)
}
static void
-get_local_iface_ids(const struct ovsrec_bridge *br_int, struct shash *lports)
+create_distributed_port(struct controller_ctx *ctx,
+ const struct ovsrec_bridge *br_int,
+ const struct sbrec_port_binding *binding_rec,
+ struct hmap *local_datapaths)
+{
+ /* if local datapath doesnt exist return */
+ struct local_datapath *ld;
+ ld = CONTAINER_OF(hmap_first_with_hash(local_datapaths,
+ binding_rec->datapath->tunnel_key),
+ struct local_datapath, hmap_node);
+ const char *name = smap_get(&binding_rec->options,
+ "name");
+
+ if (!ld || !ctx->ovs_idl_txn || !name) {
+ return;
+ }
+ /* create the internal port */
+ struct ovsrec_interface *iface;
+ iface = ovsrec_interface_insert(ctx->ovs_idl_txn);
+ ovsrec_interface_set_name(iface, name);
+ ovsrec_interface_set_type(iface, "internal");
+
+ struct smap ids;
+ smap_clone(&ids, &binding_rec->options);
+ ovsrec_interface_set_external_ids(iface, &ids);
+ smap_destroy(&ids);
+
+ struct ovsrec_port *port;
+ port = ovsrec_port_insert(ctx->ovs_idl_txn);
+ ovsrec_port_set_name(port, name);
+ ovsrec_port_set_interfaces(port, &iface, 1);
+ const struct smap port_ids = SMAP_CONST1(&port_ids,
+ "ovn-distributed-port",
+ binding_rec->logical_port);
+ ovsrec_port_set_external_ids(port, &port_ids);
+
+ struct ovsrec_port **ports;
+ ports = xmalloc(sizeof *ports * (br_int->n_ports + 1));
+ memcpy(ports, br_int->ports, sizeof *ports * br_int->n_ports);
+ ports[br_int->n_ports] = port;
+ ovsrec_bridge_verify_ports(br_int);
+ ovsrec_bridge_set_ports(br_int, ports, br_int->n_ports + 1);
+
+ free(ports);
+}
+
+static void
+get_local_iface_ids(const struct ovsrec_bridge *br_int, struct shash *lports,
+ struct shash *distributed_ports)
{
int i;
@@ -63,10 +111,18 @@ get_local_iface_ids(const struct ovsrec_bridge *br_int, struct shash *lports)
continue;
}
+ const char *distributed_port = smap_get(&port_rec->external_ids,
+ "ovn-distributed-port");
for (j = 0; j < port_rec->n_interfaces; j++) {
const struct ovsrec_interface *iface_rec;
iface_rec = port_rec->interfaces[j];
+
+ if (distributed_port) {
+ shash_add(distributed_ports, distributed_port, iface_rec);
+ break;
+ }
+
iface_id = smap_get(&iface_rec->external_ids, "iface-id");
if (!iface_id) {
continue;
@@ -160,8 +216,9 @@ binding_run(struct controller_ctx *ctx, const struct ovsrec_bridge *br_int,
}
struct shash lports = SHASH_INITIALIZER(&lports);
+ struct shash distributed_ports = SHASH_INITIALIZER(&distributed_ports);
if (br_int) {
- get_local_iface_ids(br_int, &lports);
+ get_local_iface_ids(br_int, &lports, &distributed_ports);
} else {
/* We have no integration bridge, therefore no local logical ports.
* We'll remove our chassis from all port binding records below. */
@@ -218,6 +275,14 @@ binding_run(struct controller_ctx *ctx, const struct ovsrec_bridge *br_int,
* a conntrack zone ID for each one, as we'll be creating
* a patch port for each one. */
sset_add(&all_lports, binding_rec->logical_port);
+ } else if (!binding_rec->chassis
+ && !strcmp(binding_rec->type, "distributed")) {
+ sset_add(&all_lports, binding_rec->logical_port);
+ /* create the distributed port if it doesnt exist */
+ if (!shash_find_and_delete(&distributed_ports,
+ binding_rec->logical_port)) {
+ create_distributed_port(ctx, br_int, binding_rec, local_datapaths);
+ }
}
}
@@ -227,7 +292,11 @@ binding_run(struct controller_ctx *ctx, const struct ovsrec_bridge *br_int,
update_ct_zones(&all_lports, ct_zones, ct_zone_bitmap);
+ /* FIXME cleanup removed distributed ports here
+ */
+
shash_destroy(&lports);
+ shash_destroy(&distributed_ports);
sset_destroy(&all_lports);
}
diff --git a/ovn/controller/physical.c b/ovn/controller/physical.c
index 795db31..b929e8d 100644
--- a/ovn/controller/physical.c
+++ b/ovn/controller/physical.c
@@ -169,6 +169,8 @@ physical_run(struct controller_ctx *ctx, enum mf_field_id mff_ovn_geneve,
"ovn-localnet-port");
const char *logpatch = smap_get(&port_rec->external_ids,
"ovn-logical-patch-port");
+ const char *distributed = smap_get(&port_rec->external_ids,
+ "ovn-distributed-port");
for (int j = 0; j < port_rec->n_interfaces; j++) {
const struct ovsrec_interface *iface_rec = port_rec->interfaces[j];
@@ -185,7 +187,11 @@ physical_run(struct controller_ctx *ctx, enum mf_field_id mff_ovn_geneve,
/* Record as patch to local net, logical patch port, chassis, or
* local logical port. */
bool is_patch = !strcmp(iface_rec->type, "patch");
- if (is_patch && localnet) {
+ if (distributed) {
+ simap_put(&localvif_to_ofport, distributed, ofport);
+ break;
+ }
+ else if (is_patch && localnet) {
/* localnet patch ports can be handled just like VIFs. */
simap_put(&localvif_to_ofport, localnet, ofport);
break;
@@ -308,7 +314,7 @@ physical_run(struct controller_ctx *ctx, enum mf_field_id mff_ovn_geneve,
const struct sbrec_port_binding *localnet_port =
get_localnet_port(local_datapaths,
binding->datapath->tunnel_key);
- if (!ofport) {
+ if (!ofport && strcmp(binding->type, "distributed")) {
/* It is remote port, may be reached by tunnel or localnet port */
is_remote = true;
if (!binding->chassis) {
--
2.3.9
More information about the dev
mailing list