[ovs-dev] [PATCH 5/5] ofproto: Report controller rate limiting statistics in database.
Gurucharan Shetty
shettyg at nicira.com
Thu Jul 17 17:08:16 UTC 2014
On Wed, Jul 16, 2014 at 3:05 PM, Ben Pfaff <blp at nicira.com> wrote:
> Signed-off-by: Ben Pfaff <blp at nicira.com>
I did not test it, but looks correct.
Acked-by: Gurucharan Shetty <gshetty at nicira.com>
> ---
> NEWS | 1 +
> ofproto/connmgr.c | 26 +++++++++++-
> ofproto/pinsched.c | 17 +++++---
> ofproto/pinsched.h | 11 ++++-
> vswitchd/vswitch.xml | 112 +++++++++++++++++++++++++++++++++++++++-----------
> 5 files changed, 135 insertions(+), 32 deletions(-)
>
> diff --git a/NEWS b/NEWS
> index 2504f5c..d7c0d0a 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -7,6 +7,7 @@ Post-v2.3.0
> possible to generate or match options. This is planned for a future
> release. The protocol is documented at
> http://tools.ietf.org/html/draft-gross-geneve-00
> + - The OVS database now reports controller rate limiting statistics.
>
>
> v2.3.0 - xx xxx xxxx
> diff --git a/ofproto/connmgr.c b/ofproto/connmgr.c
> index c7deea3..80b0f6d 100644
> --- a/ofproto/connmgr.c
> +++ b/ofproto/connmgr.c
> @@ -414,7 +414,10 @@ connmgr_get_memory_usage(const struct connmgr *mgr, struct simap *usage)
>
> packets += rconn_count_txqlen(ofconn->rconn);
> for (i = 0; i < N_SCHEDULERS; i++) {
> - packets += pinsched_count_txqlen(ofconn->schedulers[i]);
> + struct pinsched_stats stats;
> +
> + pinsched_get_stats(ofconn->schedulers[i], &stats);
> + packets += stats.n_queued;;
> }
> packets += pktbuf_count_packets(ofconn->pktbuf);
> }
> @@ -471,6 +474,7 @@ connmgr_get_controller_info(struct connmgr *mgr, struct shash *info)
> time_t last_connection = rconn_get_last_connection(rconn);
> time_t last_disconnect = rconn_get_last_disconnect(rconn);
> int last_error = rconn_get_last_error(rconn);
> + int i;
>
> shash_add(info, target, cinfo);
>
> @@ -494,6 +498,26 @@ connmgr_get_controller_info(struct connmgr *mgr, struct shash *info)
> smap_add_format(&cinfo->pairs, "sec_since_disconnect",
> "%ld", (long int) (now - last_disconnect));
> }
> +
> + for (i = 0; i < N_SCHEDULERS; i++) {
> + if (ofconn->schedulers[i]) {
> + const char *name = i ? "miss" : "action";
> + struct pinsched_stats stats;
> +
> + pinsched_get_stats(ofconn->schedulers[i], &stats);
> + smap_add_nocopy(&cinfo->pairs,
> + xasprintf("packet-in-%s-backlog", name),
> + xasprintf("%u", stats.n_queued));
> + smap_add_nocopy(&cinfo->pairs,
> + xasprintf("packet-in-%s-bypassed", name),
> + xasprintf("%llu", stats.n_normal));
> + smap_add_nocopy(&cinfo->pairs,
> + xasprintf("packet-in-%s-queued", name),
> + xasprintf("%llu", stats.n_limited));
> + smap_add_nocopy(&cinfo->pairs,
> + xasprintf("packet-in-%s-dropped", name),
> + xasprintf("%llu", stats.n_queue_dropped));
> + }
> }
> }
> }
> diff --git a/ofproto/pinsched.c b/ofproto/pinsched.c
> index 6b66774..e1eeeb3 100644
> --- a/ofproto/pinsched.c
> +++ b/ofproto/pinsched.c
> @@ -295,10 +295,17 @@ pinsched_set_limits(struct pinsched *ps, int rate_limit, int burst_limit)
> }
> }
>
> -/* Returns the number of packets scheduled to be sent eventually by 'ps'.
> - * Returns 0 if 'ps' is null. */
> -unsigned int
> -pinsched_count_txqlen(const struct pinsched *ps)
> +/* Retrieves statistics for 'ps'. The statistics will be all zero if 'ps' is
> + * null. */
> +void
> +pinsched_get_stats(const struct pinsched *ps, struct pinsched_stats *stats)
> {
> - return ps ? ps->n_queued : 0;
> + if (ps) {
> + stats->n_queued = ps->n_queued;
> + stats->n_normal = ps->n_normal;
> + stats->n_limited = ps->n_limited;
> + stats->n_queue_dropped = ps->n_queue_dropped;
> + } else {
> + memset(stats, 0, sizeof *stats);
> + }
> }
> diff --git a/ofproto/pinsched.h b/ofproto/pinsched.h
> index 8cce1f2..8bbdf96 100644
> --- a/ofproto/pinsched.h
> +++ b/ofproto/pinsched.h
> @@ -1,5 +1,5 @@
> /*
> - * Copyright (c) 2008, 2009, 2010, 2011, 2012, 2013 Nicira, Inc.
> + * Copyright (c) 2008, 2009, 2010, 2011, 2012, 2013, 2014 Nicira, Inc.
> *
> * Licensed under the Apache License, Version 2.0 (the "License");
> * you may not use this file except in compliance with the License.
> @@ -33,6 +33,13 @@ void pinsched_send(struct pinsched *, ofp_port_t port_no, struct ofpbuf *,
> void pinsched_run(struct pinsched *, struct list *txq);
> void pinsched_wait(struct pinsched *);
>
> -unsigned int pinsched_count_txqlen(const struct pinsched *);
> +struct pinsched_stats {
> + unsigned int n_queued; /* # currently queued to send. */
> + unsigned long long n_normal; /* # txed w/o rate limit queuing. */
> + unsigned long long n_limited; /* # queued for rate limiting. */
> + unsigned long long n_queue_dropped; /* # dropped due to queue overflow. */
> +};
> +
> +void pinsched_get_stats(const struct pinsched *, struct pinsched_stats *);
>
> #endif /* pinsched.h */
> diff --git a/vswitchd/vswitch.xml b/vswitchd/vswitch.xml
> index ee2892b..d47fc1a 100644
> --- a/vswitchd/vswitch.xml
> +++ b/vswitchd/vswitch.xml
> @@ -3223,7 +3223,7 @@
> </column>
> </group>
>
> - <group title="Asynchronous Message Configuration">
> + <group title="Asynchronous Messages">
> <p>
> OpenFlow switches send certain messages to controllers spontanenously,
> that is, not in response to any request from the controller. These
> @@ -3243,38 +3243,102 @@
> on any messages that it does want to receive, if any.
> </column>
>
> - <column name="controller_rate_limit">
> + <group title="Controller Rate Limiting">
> <p>
> - The maximum rate at which the switch will forward packets to the
> - OpenFlow controller, in packets per second. This feature prevents a
> - single bridge from overwhelming the controller. If not specified,
> - the default is implementation-specific.
> + A switch can forward packets to a controller over the OpenFlow
> + protocol. Forwarding packets this way at too high a rate can
> + overwhelm a controller, frustrate use of the OpenFlow connection for
> + other purposes, increase the latency of flow setup, and use an
> + unreasonable amount of bandwidth. Therefore, Open vSwitch supports
> + limiting the rate of packet forwarding to a controller.
> </p>
>
> <p>
> - In addition, when a high rate triggers rate-limiting, Open vSwitch
> - queues controller packets for each port and transmits them to the
> - controller at the configured rate. The <ref
> - column="controller_burst_limit"/> value limits the number of queued
> - packets. Ports on a bridge share the packet queue fairly.
> + There are two main reasons in OpenFlow for a packet to be sent to a
> + controller: either the packet ``misses'' in the flow table, that is,
> + there is no matching flow, or a flow table action says to send the
> + packet to the controller. Open vSwitch limits the rate of each kind
> + of packet separately at the configured rate. Therefore, the actual
> + rate that packets are sent to the controller can be up to twice the
> + configured rate, when packets are sent for both reasons.
> </p>
>
> <p>
> - Open vSwitch maintains two such packet rate-limiters per bridge: one
> - for packets sent up to the controller because they do not correspond
> - to any flow, and the other for packets sent up to the controller by
> - request through flow actions. When both rate-limiters are filled with
> - packets, the actual rate that packets are sent to the controller is
> - up to twice the specified rate.
> + This feature is specific to forwarding packets over an OpenFlow
> + connection. It is not general-purpose QoS. See the <ref
> + table="QoS"/> table for quality of service configuration, and <ref
> + column="ingress_policing_rate" table="Interface"/> in the <ref
> + table="Interface"/> table for ingress policing configuration.
> </p>
> - </column>
>
> - <column name="controller_burst_limit">
> - In conjunction with <ref column="controller_rate_limit"/>,
> - the maximum number of unused packet credits that the bridge will
> - allow to accumulate, in packets. If not specified, the default
> - is implementation-specific.
> - </column>
> + <column name="controller_rate_limit">
> + <p>
> + The maximum rate at which the switch will forward packets to the
> + OpenFlow controller, in packets per second. If no value is
> + specified, rate limiting is disabled.
> + </p>
> + </column>
> +
> + <column name="controller_burst_limit">
> + <p>
> + When a high rate triggers rate-limiting, Open vSwitch queues
> + packets to the controller for each port and transmits them to the
> + controller at the configured rate. This value limits the number of
> + queued packets. Ports on a bridge share the packet queue fairly.
> + </p>
> +
> + <p>
> + This value has no effect unless <ref
> + column="controller_rate_limit"/> is configured. The current
> + default when this value is not specified is one-quarter of <ref
> + column="controller_rate_limit"/>, meaning that queuing can delay
> + forwarding a packet to the controller by up to 250 ms.
> + </p>
> + </column>
> +
> + <group title="Controller Rate Limiting Statistics">
> + <p>
> + These values report the effects of rate limiting. Their values are
> + relative to establishment of the most recent OpenFlow connection,
> + or since rate limiting was enabled, whichever happened more
> + recently. Each consists of two values, one with <code>TYPE</code>
> + replaced by <code>miss</code> for rate limiting flow table misses,
> + and the other with <code>TYPE</code> replaced by
> + <code>action</code> for rate limiting packets sent by OpenFlow
> + actions.
> + </p>
> +
> + <p>
> + These statistics are reported only when controller rate limiting is
> + enabled.
> + </p>
> +
> + <column name="status" key="packet-in-TYPE-bypassed"
> + type='{"type": "integer", "minInteger": 0}'>
> + Number of packets sent directly to the controller, without queuing,
> + because the rate did not exceed the configured maximum.
> + </column>
> +
> + <column name="status" key="packet-in-TYPE-queued"
> + type='{"type": "integer", "minInteger": 0}'>
> + Number of packets added to the queue to send later.
> + </column>
> +
> + <column name="status" key="packet-in-TYPE-dropped"
> + type='{"type": "integer", "minInteger": 0}'>
> + Number of packets added to the queue that were later dropped due to
> + overflow. This value is less than or equal to <ref column="status"
> + key="packet-in-TYPE-queued"/>.
> + </column>
> +
> + <column name="status" key="packet-in-TYPE-backlog"
> + type='{"type": "integer", "minInteger": 0}'>
> + Number of packets currently queued. The other statistics increase
> + monotonically, but this one fluctuates between 0 and the <ref
> + column="controller_burst_limit"/> as conditions change.
> + </column>
> + </group>
> + </group>
> </group>
>
> <group title="Additional In-Band Configuration">
> --
> 1.7.10.4
>
> _______________________________________________
> dev mailing list
> dev at openvswitch.org
> http://openvswitch.org/mailman/listinfo/dev
More information about the dev
mailing list