[ovs-dev] [PATCH 2/2] daemon: Throttle max respawning rate.

Ben Pfaff blp at nicira.com
Wed May 12 17:25:54 UTC 2010


On Tue, May 11, 2010 at 05:26:46PM -0700, Jesse Gross wrote:
> On Tue, May 11, 2010 at 11:15 AM, Ben Pfaff <blp at nicira.com> wrote:
> >
> > +                /* Throttle restarts to no more than once every 10
> > seconds. */
> > +                if (time(NULL) < last_restart + 10) {
> > +                    VLOG_WARN("%s, waiting until 10 seconds since last "
> > +                              "restart", status_msg);
> > +                    while (time(NULL) < last_restart + 10) {
> > +                        sleep(1);
> > +                    }
> 
> 
> Why do we sleep in a loop here rather than computing the amount of time
> once?

Mostly laziness.  I figured we needed a loop in case sleep() wakes up
with EINTR.  But sleeping for the remaining time is probably a better
approach.

Here's a revised patch.  Do you like it better?

--8<--------------------------cut here-------------------------->8--

>From a3aa59700fc0f43dc4dd0d3acfd45392b0d8463d Mon Sep 17 00:00:00 2001
From: Ben Pfaff <blp at nicira.com>
Date: Wed, 12 May 2010 10:02:23 -0700
Subject: [PATCH] daemon: Throttle max respawning rate.

If a monitored daemon dies quickly at startup, the system can waste a lot
of CPU time continually restarting it.  This commit prevents a given
daemon from restarting more than once every 10 seconds.
---
 lib/daemon.c |   18 +++++++++++++++++-
 1 files changed, 17 insertions(+), 1 deletions(-)

diff --git a/lib/daemon.c b/lib/daemon.c
index 292be14..78f88a0 100644
--- a/lib/daemon.c
+++ b/lib/daemon.c
@@ -323,14 +323,15 @@ should_restart(int status)
 static void
 monitor_daemon(pid_t daemon_pid)
 {
-    /* XXX Should limit the rate at which we restart the daemon. */
     /* XXX Should log daemon's stderr output at startup time. */
     const char *saved_program_name;
+    time_t last_restart;
     char *status_msg;
 
     saved_program_name = program_name;
     program_name = xasprintf("monitor(%s)", program_name);
     status_msg = xstrdup("healthy");
+    last_restart = TIME_MIN;
     for (;;) {
         int retval;
         int status;
@@ -365,6 +366,21 @@ monitor_daemon(pid_t daemon_pid)
                     }
                 }
 
+                /* Throttle restarts to no more than once every 10 seconds. */
+                if (time(NULL) < last_restart + 10) {
+                    VLOG_WARN("%s, waiting until 10 seconds since last "
+                              "restart", status_msg);
+                    for (;;) {
+                        time_t now = time(NULL);
+                        time_t wakeup = last_restart + 10;
+                        if (now >= wakeup) {
+                            break;
+                        }
+                        sleep(wakeup - now);
+                    }
+                }
+                last_restart = time(NULL);
+
                 VLOG_ERR("%s, restarting", status_msg);
                 daemon_pid = fork_and_wait_for_startup(&daemonize_fd);
                 if (!daemon_pid) {
-- 
1.7.1





More information about the dev mailing list