[ovs-discuss] add-port return successfully but ofport is not available in 2 secs

[Joni Lee]

Thank you Ben,

- ovs-vswitchd.log did show that the port was added to br-int:

Jun 30 07:30:14|5422458|bridge|INFO|created port qr-3cc2ad9c-a3 on bridge br-int


- I only found this "No such device" suspicious. Is this logged by the "find Interface" command? (because the command was executed exactly at second 15 and returned empty "ofport")

Jun 30 07:30:15|5422579|netdev_linux|INFO|ioctl(SIOCGIFHWADDR) on qr-3cc2ad9c-a3 device failed: No such device

- In the ovsdb log, the port was added to the db at second 13 successfully. But ovsdb log looks like it never records any entries for "ofport" (i guess this is normal).

record 4399: 2014-06-30 07:30:13 "ovs-vsctl: /usr/bin/ovs-vsctl -- --may-exist add-port br-int qr-3cc2ad9c-a3 -- set Interface qr-3cc2ad9c-a3 type=internal -- set Interface qr-3cc2ad9c-a3 external-ids:iface-id=3cc2ad9c-a3a5-4bd0-b5d9-bbb12ea8a4b0 -- set Interface qr-3cc2ad9c-a3 external-ids:iface-status=active -- set Interface qr-3cc2ad9c-a3 external-ids:attached-mac=fa:16:3e:47:24:0c"
        table Port insert row "qr-3cc2ad9c-a3" (0b207730):
                interfaces=[cf959597-4bd1-4ffd-b883-5bde732054db]
                name="qr-3cc2ad9c-a3"
        table Interface insert row "qr-3cc2ad9c-a3" (cf959597):
                external_ids={attached-mac="fa:16:3e:47:24:0c", iface-id="3cc2ad9c-a3a5-4bd0-b5d9-bbb12ea8a4b0", iface-status=active}
                name="qr-3cc2ad9c-a3"
                type=internal
        table Open_vSwitch row 43e20f98 (43e20f98):
                next_cfg=11097
        table Bridge row "br-int" (aa7c156b):
                ports=[0236a464-e669-4948-afc0-99f832ef7d1a, 03e7a1ae-5b40-4340-859d-50e6eac651a2, 07e25ed7-10fb-45d7-8fc2-fd33540a242e, 0ab22d8e-ebc7-4a2c-9167-cbe412934ad0, 0b207730-d411-4226-864c-86920b608709, 0b768fc7-be4d-4bcd-90b3-71df48390d8f, 0e89fee4-1369-45ed-8e0d-69da6637072e, 113b9e3b-1d58-4f34-914d-5447ce662a46, 137fc2bd-e1c5-44fd-b068-c32ba0b204b3, 16ae96d9-6639-4849-be8f-0684428e0495, 1affd622-f09a-43b3-a846-3d468b62b671, 1b0e2e76-c636-429c-8e8d-3831fdf27834, 1c875b47-b4e5-48fa-b061-72adf2742e16, 2118657f-7a54-4d09-bc2b-72b9ffd98b6f, 2522705c-b81b-4025-abeb-5cb7f6f657a9, 26056d71-9805-4b96-a3e8-6b6ce5f72def, 27dfdf61-b02c-4490-b92a-2922f2f6f784, 28d6d192-591f-44cc-ba31-2522fc7747f0, 2a7ffbff-5961-4273-b179-bcebbf78a386, 3832a07e-0931-4ab9-8ef3-22a74192ffdd, 398634cd-cc95-4762-89fd-0ed63ee1437f, 3e3e0242-c2b9-4e60-a95b-edf7987e3cda, 426041eb-dc33-41e0-bcb4-db351c4d4a86, 47799c1a-725a-466a-88f8-d27ae54a5a48,
 4d92c5bd-d18f-4684-aac5-6779fee5c16a, 4ed27f5f-db42-4161-bbf9-fe1c82841fbb, 57d4ded7-f91a-418d-9791-38334ad45c06, 5b524d6b-e175-4c32-a56d-4c19f611ed36, 5ede2620-4c37-4864-a4ec-c67c97e90c96, 66067589-dc53-4aba-9fa3-dbbf5f1d9d17, 663d7f3f-c147-481f-9d4b-b1a95a4e5f2c, 678a6002-d74b-4908-8111-a3d2243058ae, 6d8fb050-408f-4e95-b36a-ea2f9aed8849, 6e2131dc-3228-4b8b-af2a-b4e837c87aaf, 73d63e8a-169e-4942-9e93-44baea8c1aa6, 74838af4-3839-4565-9e87-4a496df4b306, 74fdc6e6-cc53-4a65-abc1-8e3ff44d48ff, 7c32e0cc-e6cd-4636-9029-c99675f7be42, 7ccb4eff-53c5-4515-9925-91ca2aabab0f, 7fd5cafb-a775-4202-ae46-53ee63beb234, 8b907319-9eba-45ba-ac21-f0713028f7f7, 8ccf9f9a-da3b-4a22-a49a-d6f7e713d01f, 8f4f5842-1418-4f41-a466-2860573abca4, 8fa39df9-e526-4d90-8e71-2c557abcfeaa, 93a6f314-fe07-460c-ace6-e2e2e7c3a048, 9b45b0a5-0250-4115-93fe-1b35c8ef87e6, 9c3adc0f-7515-46a6-b3e5-4cd276ad1bf9, 9cb3d67e-6583-4a8e-ae0d-1510691d7fb5, 9dceb5f4-f909-4421-895f-fc01ff451a4c,
 9e550eeb-1a5f-46aa-82cc-55fee13134d1, a2369f8f-584c-4bdb-9a2a-bfc56883a7b6, a2fbc204-df6b-4b36-8868-c0f6434a338e, a43762aa-05eb-4e49-989b-340fedbd9213, a5377595-17bb-4fd5-b5a9-84e9438e0278, aa2b9722-ebc9-4ab4-a668-c32fb00263fd, abad75f8-b113-4afa-9309-9e16a8859682, ac654863-613b-468d-8ab1-ebf7e8838c1b, b8f06af1-7ee0-4119-8af0-eeec8f85a4b7, bcce72c2-2b71-4c6b-a70d-80724ca4fefc, be37d38d-25bd-4b36-8ff7-d653881e9f07, be647251-cb8e-4e5f-833f-01e19d5497fe, cf940b9e-301c-4371-8f79-f7a4ad01a29c, d2ce10db-68eb-4f1b-ade0-482c380856d0, d4e8ccdb-2bee-4168-9c7c-7c80f848677d, d53f99b7-75c9-4f2f-8f59-e87ed3e7ac2e, d7975141-d0fa-47b3-99a5-5251e3412531, d7d88a29-3cfc-4726-99a3-20fbc37355a0, d846fc3c-9c65-4c70-8107-664119d2d04e, df5a0aa3-d60d-4f65-ae1b-25c95cec0aa4, e0acf44c-4753-4e58-b3ac-cd56a74b04af, e25b87c1-9ac6-4094-953b-72f08382b8c6, e4374e9e-1cf9-48af-bd46-270e1123a2db, e9016599-2349-4b30-80cf-f9c011aacaa8, e9702ed6-b42f-46b9-bad9-8d5afba37840,
 efb00880-4e56-42cd-b94b-b56f5255386b, f5c1a994-0047-4701-96a8-2ea6d5f556b0, f897217f-a91b-47cb-a8c9-8028df20ad60, fae019ea-1fed-475d-8f88-5d733d929cae]


On Tuesday, July 1, 2014 7:48 PM, Ben Pfaff <blp at nicira.com> wrote:
 


On Tue, Jul 01, 2014 at 01:14:16PM +0100, Joni Lee wrote:

> We seem to face a concurrent issue between 2 processes: "add-br" and "find Interface" as follows
> 
> 1. A "l3 agent" process added a port q-router to a bridge (br-int) by calling "ovs-vsctl?add-port"?and the command returned successfully:
> 
> Jun 30 07:30:13 neutron1 ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl -- --may-exist add-port br-int qr-3cc2ad9c-a3 -- set Interface qr-3cc2ad9c-a3 type=internal -- set Interface qr-3cc2ad9c-a3 external-ids:iface-id=3cc2ad9c-a3a5-4bd0-b5d9-bbb12ea8a4b0 -- set Interface qr-3cc2ad9c-a3 external-ids:iface-status=active -- set Interface qr-3cc2ad9c-a3 external-ids:attached-mac=fa:16:3e:47:24:0c
> 
> 2. On the other hand, after 2 secs, another "ovs agent" process finds the interface, but ofport is empty:
> 
> $ ovs-vsctl -- --columns=external_ids,name,ofport find Interface external_ids:iface-id=3cc2ad9c-a3a5-4bd0-b5d9-bbb12ea8a4b0
> external_ids ? ? ? ?: {attached-mac="fa:16:3e:47:24:0c", iface-id="3cc2ad9c-a3a5-4bd0-b5d9-bbb12ea8a4b0", iface-status=active}
> name ? ? ? ? ? ? ? ? ?: "qr-3cc2ad9c-a3"
> ofport ? ? ? ? ? ? ? ? :?[] ? ?<<<<<< empty set
> 
> ofport?the empty set?means that the interface hasn't been created yet,?even the command "add-br" returned successfully???
> 
> 3. After some more seconds,?We checked and see that "ofport" is available.
> 
> Any ideas which causes the latency of "add-br" and how to improve it?

ovs-vsctl, without the --no-wait option, should only exit after the
reconfiguration is complete, so the behavior that you report under
point 2 is unexpected.

I would start by looking in the ovs-vswitchd log to see if it's
reporting some problem that quickly resolves.

If not, then I would look in the ovsdb log (with "ovsdb-tool -mm
show-log") to check out the sequence of database transactions.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20140702/2fdd4314/attachment-0002.html>