[ovs-dev] [PATCH v6] ovsdb-tool: Convert clustered db to standalone db.
Han Zhou
zhouhan at gmail.com
Thu Aug 29 20:59:42 UTC 2019
On Thu, Aug 29, 2019 at 1:14 PM <amginwal at gmail.com> wrote:
>
> From: Aliasgar Ginwala <aginwala at ebay.com>
>
> Add support in ovsdb-tool for migrating clustered dbs to standalone dbs.
> E.g. usage to migrate nb/sb db to standalone db from raft:
> ovsdb-tool cluster-to-standalone ovnnb_db.db ovnnb_db_cluster.db
>
> Signed-off-by: Aliasgar Ginwala <aginwala at ebay.com>
> ---
> Documentation/ref/ovsdb.7.rst | 3 +
> NEWS | 3 +
> ovsdb/ovsdb-tool.1.in | 8 +++
> ovsdb/ovsdb-tool.c | 101 +++++++++++++++++++++++++++++++++-
> tests/ovsdb-tool.at | 46 ++++++++++++++++
> 5 files changed, 160 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/ref/ovsdb.7.rst b/Documentation/ref/ovsdb.7.rst
> index cd1c63d64..b12d8066c 100644
> --- a/Documentation/ref/ovsdb.7.rst
> +++ b/Documentation/ref/ovsdb.7.rst
> @@ -514,6 +514,9 @@ standalone database from the contents of a running
clustered database.
> When the cluster is down and cannot be revived, ``ovsdb-client backup``
will
> not work.
>
> +Use ``ovsdb-tool cluster-to-standalone`` to convert clustered database to
> +standalone database when the cluster is down and cannot be revived.
> +
> Upgrading or Downgrading a Database
> -----------------------------------
>
> diff --git a/NEWS b/NEWS
> index c5caa13d6..a02f9f1a6 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -49,6 +49,9 @@ v2.12.0 - xx xxx xxxx
> quickly after a brief disconnection, saving bandwidth and CPU
time.
> See section 4.1.15 of ovsdb-server(7) for details of related OVSDB
> protocol extension.
> + * Support to convert from cluster database to standalone database
is now
> + available when clustered is down and cannot be revived using
ovsdb-tool
> + . Check "Database Migration Commands" in ovsdb-tool man section.
> - OVN:
> * IPAM/MACAM:
> - select IPAM mac_prefix in a random manner if not provided by
the user
> diff --git a/ovsdb/ovsdb-tool.1.in b/ovsdb/ovsdb-tool.1.in
> index ec85e14c4..31a918d90 100644
> --- a/ovsdb/ovsdb-tool.1.in
> +++ b/ovsdb/ovsdb-tool.1.in
> @@ -147,6 +147,14 @@ avoid this possibility, specify
\fB\-\-cid=\fIuuid\fR, where
> \fIuuid\fR is the cluster ID of the cluster to join, as printed by
> \fBovsdb\-tool get\-cid\fR.
> .
> +.SS "Database Migration Commands"
> +This commands will convert cluster database to standalone database.
> +.
> +.IP "\fBcluster\-to\-standalone\fI db clusterdb"
> +Use this command to convert to standalone database from clustered
database
> +when the cluster is down and cannot be revived. It creates new standalone
> +\fIdb\fR file from the given cluster \fIdb\fR file.
> +.
> .SS "Version Management Commands"
> .so ovsdb/ovsdb-schemas.man
> .PP
> diff --git a/ovsdb/ovsdb-tool.c b/ovsdb/ovsdb-tool.c
> index 438f97590..3bbf4c8bc 100644
> --- a/ovsdb/ovsdb-tool.c
> +++ b/ovsdb/ovsdb-tool.c
> @@ -173,6 +173,9 @@ usage(void)
> " compare-versions A OP B compare OVSDB schema version
numbers\n"
> " query [DB] TRNS execute read-only transaction on
DB\n"
> " transact [DB] TRNS execute read/write transaction on
DB\n"
> + " cluster-to-standalone DB DB Convert clustered DB to\n"
> + " standalone DB when cluster is down and cannot be\n"
> + " revived\n"
> " [-m]... show-log [DB] print DB's log entries\n"
> "The default DB is %s.\n"
> "The default SCHEMA is %s.\n",
> @@ -942,6 +945,55 @@ print_raft_record(const struct raft_record *r,
> }
> }
>
> +static void
> +raft_header_to_standalone_log(const struct raft_header *h,
> + struct ovsdb_log *db_log_data)
> +{
> + if (h->snap_index) {
> + if (!h->snap.data || json_array(h->snap.data)->n != 2) {
> + ovs_fatal(0, "Incorrect raft header data array length");
> + }
> +
> + struct json *schema_json = json_array(h->snap.data)->elems[0];
> + if (schema_json->type != JSON_NULL) {
> + struct ovsdb_schema *schema;
> + check_ovsdb_error(ovsdb_schema_from_json(schema_json,
&schema));
> + ovsdb_schema_destroy(schema);
> + check_ovsdb_error(ovsdb_log_write_and_free(db_log_data,
> + schema_json));
> + }
> +
> + struct json *data_json = json_array(h->snap.data)->elems[1];
> + if (!data_json || data_json->type != JSON_OBJECT) {
> + ovs_fatal(0, "Invalid raft header data");
> + }
> + if (data_json->type != JSON_NULL) {
> + check_ovsdb_error(ovsdb_log_write_and_free(db_log_data,
> + data_json));
> + }
> + }
> +}
> +
> +static void
> +raft_record_to_standalone_log(const struct raft_record *r,
> + struct ovsdb_log *db_log_data)
> +{
> + if (r->type == RAFT_REC_ENTRY) {
> + if (!r->entry.data) {
> + return;
> + }
> + if (json_array(r->entry.data)->n != 2) {
> + ovs_fatal(0, "Incorrect raft record array length");
> + }
> +
> + struct json *data_json = json_array(r->entry.data)->elems[1];
> + if (data_json->type != JSON_NULL) {
> + check_ovsdb_error(ovsdb_log_write_and_free(db_log_data,
> + data_json));
> + }
> + }
> +}
> +
> static void
> do_show_log_cluster(struct ovsdb_log *log)
> {
> @@ -1511,6 +1563,51 @@ do_compare_versions(struct ovs_cmdl_context *ctx)
> exit(result ? 0 : 2);
> }
>
> +static void
> +do_convert_to_standalone(struct ovsdb_log *log, struct ovsdb_log
*db_log_data)
> +{
> + for (unsigned int i = 0; ; i++) {
> + struct json *json;
> + check_ovsdb_error(ovsdb_log_read(log, &json));
> + if (!json) {
> + break;
> + }
> +
> + if (i == 0) {
> + struct raft_header h;
> + check_ovsdb_error(raft_header_from_json(&h, json));
> + raft_header_to_standalone_log(&h, db_log_data);
> + raft_header_uninit(&h);
> + } else {
> + struct raft_record r;
> + check_ovsdb_error(raft_record_from_json(&r, json));
> + raft_record_to_standalone_log(&r, db_log_data);
> + raft_record_uninit(&r);
> + }
> + }
> +}
> +
> +static void
> +do_cluster_standalone(struct ovs_cmdl_context *ctx)
> +{
> + const char *db_file_name = ctx->argv[1];
> + const char *cluster_db_file_name = ctx->argv[2];
> + struct ovsdb_log *log;
> + struct ovsdb_log *db_log_data;
> +
> + check_ovsdb_error(ovsdb_log_open(cluster_db_file_name,
> + OVSDB_MAGIC"|"RAFT_MAGIC,
> + OVSDB_LOG_READ_ONLY, -1, &log));
> + check_ovsdb_error(ovsdb_log_open(db_file_name, OVSDB_MAGIC,
> + OVSDB_LOG_CREATE_EXCL, -1,
&db_log_data));
> + if (strcmp(ovsdb_log_get_magic(log), RAFT_MAGIC) != 0) {
> + ovs_fatal(0, "Database is not clustered db.\n");
> + }
> + do_convert_to_standalone(log, db_log_data);
> + check_ovsdb_error(ovsdb_log_commit_block(db_log_data));
> + ovsdb_log_close(db_log_data);
> + ovsdb_log_close(log);
> +}
>
> static void
> do_help(struct ovs_cmdl_context *ctx OVS_UNUSED)
> @@ -1550,7 +1647,9 @@ static const struct ovs_cmdl_command all_commands[]
= {
> { "compare-versions", "a op b", 3, 3, do_compare_versions, OVS_RO },
> { "help", NULL, 0, INT_MAX, do_help, OVS_RO },
> { "list-commands", NULL, 0, INT_MAX, do_list_commands, OVS_RO },
> - { NULL, NULL, 0, 0, NULL, OVS_RO },
> + { "cluster-to-standalone", "db clusterdb", 2, 2,
> + do_cluster_standalone, OVS_RW },
> + { NULL, NULL, 2, 2, NULL, OVS_RO },
> };
>
> static const struct ovs_cmdl_command *get_all_commands(void)
> diff --git a/tests/ovsdb-tool.at b/tests/ovsdb-tool.at
> index 69c5d6afa..cf98e4890 100644
> --- a/tests/ovsdb-tool.at
> +++ b/tests/ovsdb-tool.at
> @@ -459,3 +459,49 @@ OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> # Make sure that the clustered data matched the standalone data.
> AT_CHECK([cat dump2], [0], [expout])
> AT_CLEANUP
> +
> +AT_SETUP([ovsdb-tool convert-to-standalone])
> +AT_KEYWORDS([ovsdb file positive])
> +
> +# Create a standalone database and put some data in it.
> +ordinal_schema > schema
> +ovsdb-tool create db1 schema
> +AT_CHECK(
> + [[for pair in 'zero 0' 'one 1' 'two 2' 'three 3' 'four 4' 'five 5'; do
> + set -- $pair
> + ovsdb-tool transact db1 '
> + ["ordinals",
> + {"op": "insert",
> + "table": "ordinals",
> + "row": {"name": "'$1'", "number": '$2'}},
> + {"op": "comment",
> + "comment": "add row for '"$pair"'"}]'
> + done | uuidfilt]], [0],
> +[[[{"uuid":["uuid","<0>"]},{}]
> +[{"uuid":["uuid","<1>"]},{}]
> +[{"uuid":["uuid","<2>"]},{}]
> +[{"uuid":["uuid","<3>"]},{}]
> +[{"uuid":["uuid","<4>"]},{}]
> +[{"uuid":["uuid","<5>"]},{}]
> +]], [ignore])
> +
> +# Dump the data.
> +AT_CHECK([ovsdb-server -vfile -vvlog:off --detach --no-chdir --pidfile
--log-file --remote=punix:db.sock db1])
> +AT_CHECK([ovsdb-client dump > expout])
> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> +
> +# Create a clustered database from the standalone one.
> +ovsdb-tool create-cluster db2 db1 unix:s1.raft
It would be better to execute some transactions in clustered mode before
the conversion test, so that the file will have not only header with
snapshot but also body records, so that the raft record conversion
functions are covered.
> +
> +# Convert to standalone database from clustered database.
> +ovsdb-tool cluster-to-standalone db3 db2
> +
Shall we check if db3 is standalone format by: ovsdb-tool db-is-standalone
db3
Otherwise, would the test pass even if we didn't do conversion at all?
> +# Dump the data.
> +AT_CHECK([ovsdb-server -vconsole:off -vfile -vvlog:off --detach
--no-chdir --pidfile --log-file --remote=punix:db.sock db3])
> +AT_CHECK([ovsdb_client_wait ordinals connected])
> +AT_CHECK([ovsdb-client dump > dump3])
> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> +
> +# Make sure both standalone db data matches.
> +AT_CHECK([cat dump3], [0], [expout])
> +AT_CLEANUP
> --
> 2.20.1 (Apple Git-117)
>
> _______________________________________________
> dev mailing list
> dev at openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
More information about the dev
mailing list