[ovs-dev] [ovs-discuss][OVN][SSL] replacing ssl key and certs file at runtime has no effect on ovn-controller connection

Girish Moodalbail gmoodalbail at gmail.com
Tue Dec 29 02:52:33 UTC 2020


It looks to me that the function stream_ssl_set_key_and_cert() in
lib/stream-ssl.c is incorrect.

void
stream_ssl_set_key_and_cert(const char *private_key_file,
                            const char *certificate_file)
{
    if (update_ssl_config(&private_key, private_key_file)
        && update_ssl_config(&certificate, certificate_file)) {
        stream_ssl_set_certificate_file__(certificate_file);
        stream_ssl_set_private_key_file__(private_key_file);
    }
}

1. Say, the private key and the corresponding certificate file was replaced
on the file system at T0 and T2 respectively.
2. At T1, the ovn-controller code calls update_ssl_config(private_key) and
update_ssl_config(certificate_file)
2a: The first call to update_ssl_config(private_key) returns true and the
file `mtime` is updated. The second call to
update_ssl_config(certificate_file) returns False
3. At T3, the ovn-controller code calls to update_ssl_config(private_key)
will return False, and the modified `certifcate file` will never be picked?

Because of 1 - 3 above, the new files will never be picked by the
ovn-controller. What we have found is that if I delete both the files and
then copy over the private key and certificate files, then it works. This
may be because of how we handle the ENOENT case in update_ssl_config()

Regards,
~Girish

On Fri, Dec 11, 2020 at 9:54 AM Girish Moodalbail <gmoodalbail at gmail.com>
wrote:

> In ovn-kubernetes K8s CNI project we use SSL connections between
> ovn-controller and OVN SB DB. Our goal is to rotate the
> privateKey/signedCert used by ovn-controller very often. When the rotation
> occurs, we want ovn-controller to redo the TLS handshake without dropping
> the TCP connection or without requiring ovn-controller restart.
>
> In ovn-controller code, I see that in the main loop we call
> update_ssl_config(), which through a series of functions checks if SSL
> files are modified, and if so, calls into OpenSSL library updating the SSL
> context. At this point, the expectation is for ovn-controller to restart
> the TLS handshake so that we are using new SSL keys/certs. However, we
> don't see this happening.
>
> I ran tcpdump on the ovn-controller side to check for TLS handshake
> packets, but I didn't see any TLS related packets. With `stream_ssl` module
> logging set to debug, I don't see any SSL control plane messages in
> ovn-controller.log.
>
> I also created certs with expiry time of 10mins using OVS-PKI and
> restarted ovn-controller. My expectation was that after 10mins the SSL
> connection should error out with certificates already expired. I don't see
> that is happening as well. When I run the `ovn-sbctl` command using the
> expired certs it obviously fails.
>
> Looks to me that once the JSON-RPC session is created between
> ovn-controller and  OVN SB DB process, then nothing seems to re-trigger the
> TLS handshake.
>
> Are we missing something?
>
> Regards,
> ~Girish
>


More information about the dev mailing list