[ovs-discuss] OVN Questions

Maurice Qureshi Maurice.Qureshi at calix.com
Thu May 5 21:19:56 UTC 2016 

Hi Justin,

Now that I am able to use the OVN sandbox, I have few more questions. I will start with the simplest one.

I am tracing the packet using the ovn/env1/packet1.sh. Trace seems to show that it is hitting the entry in flow table and eventually action is output port 2. However, The n_packets counters for any of the entry in flow table don't go up. Shouldn't it increase as the packet hits a flow table entry?

Thanks
Maurice 

-----Original Message-----
From: Justin Pettit [mailto:jpettit at ovn.org] 
Sent: Friday, April 29, 2016 9:02 PM
To: Maurice Qureshi <Maurice.Qureshi at calix.com>
Cc: Darrell Ball <dball at vmware.com>; discuss at openvswitch.org
Subject: Re: [ovs-discuss] OVN Questions

That seems odd. It's supposed to set PATH when it's being run from a build directory. (I can even see it using the "-b" option in the output you've provided.) I run it from my build directory and don't have any problem. 

I'm glad you got it working at least in some way, though. 

--Justin


> On Apr 29, 2016, at 5:40 PM, Maurice Qureshi <Maurice.Qureshi at calix.com> wrote:
> 
> Hi Justin,
> 
> This what I did.
> 
> 1) Created a new VM
> 2) Tried to install the sandbox, but ran into the same isuse. PATH wasn't getting updated.
> 3) Installed openvswitch (sudo apt-get install openvswitch-switch)
> 4) Installed sandbox again.
> 5) PATH was correctly updated. Commands started working.
> 
> So to me it looks like openvswitch had to be installed first. Not sure if that is indeed needed, but that seemed to have helped.
> 
> Thanks
> Maurice
> 
> -----Original Message-----
> From: Justin Pettit [mailto:jpettit at ovn.org]
> Sent: Friday, April 29, 2016 4:24 PM
> To: Maurice Qureshi <Maurice.Qureshi at calix.com>
> Cc: Darrell Ball <dball at vmware.com>; discuss at openvswitch.org
> Subject: Re: [ovs-discuss] OVN Questions
> 
> What was the issue?
> 
> --Justin
> 
> 
>> On Apr 29, 2016, at 4:14 PM, Maurice Qureshi <Maurice.Qureshi at calix.com> wrote:
>> 
>> Hi Darrell/Justin,
>> 
>> I got it to work for now. Thanks for your help.
>> 
>> Maurice
>> 
>> -----Original Message-----
>> From: discuss [mailto:discuss-bounces at openvswitch.org] On Behalf Of 
>> Maurice Qureshi
>> Sent: Friday, April 29, 2016 12:43 PM
>> To: Darrell Ball <dball at vmware.com>; Justin Pettit <jpettit at ovn.org>
>> Cc: discuss at openvswitch.org
>> Subject: Re: [ovs-discuss] OVN Questions
>> 
>> Hi Darrell,
>> 
>> I checked and it looks similar to your
>> 
>> + PATH=/home/mqureshi/ovs/ovn/controller:/home/mqureshi/ovs/ovn/contr
>> + o
>> + ll
>> + er-vtep:/home/mqureshi/ovs/ovn/northd:/home/mqureshi/ovs/ovn/utilit
>> + i
>> + es
>> + :/home/mqureshi/ovs/ovsdb:/home/mqureshi/ovs/vswitchd:/home/mquresh
>> + i
>> + /o
>> + vs/utilities:/home/mqureshi/ovs/vtep:/tools/sw/exa_2.0/exa-tools:/u
>> + s
>> + r/
>> + lib64/ccache:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:
>> + /h
>> + ome/mqureshi/bin:/sbin
>> + export PATH
>> 
>> The difference I see is that in your case, the build directory is "openvswitch/ovs/_gcc" whereas in  my case it is simply "ovs". I have attached the full output of " sh -x ../tutorial/ovs-sandbox --ovn" command.
>> 
>> Have two questions.
>> 
>> 1. Do I need to install the openvswitch before installing the 
>> ovn-sandbox. I think the sandbox does that for you as I see all the 
>> necessary daemons running (ovs-vswitchd, ovsdb-server, ovn-northd, 
>> ovn-controller, ovn-controller-vtep)
>> 
>> 2. Looks like this sandbox runs its own shell. But "echo $SHELL" still shows "bash" as shell. Is that correct?
>> 
>> mqureshi at vmexa-mqureshi:~/ovs$ echo $SHELL /bin/bash 
>> mqureshi at vmexa-mqureshi:~/ovs$
>> 
>> 3. What if I just wanted to install OVN directly without sandbox, how can I do that (something similar to "sudo apt-get install openvswitch-switch").  Most of the documentation about OVN on Google is about the sandbox. 
>> 
>> Thanks
>> Maurice
>> 
>> 
>> -----Original Message-----
>> From: Darrell Ball [mailto:dball at vmware.com]
>> Sent: Thursday, April 28, 2016 5:35 PM
>> To: Maurice Qureshi <Maurice.Qureshi at calix.com>; Justin Pettit 
>> <jpettit at ovn.org>
>> Cc: discuss at openvswitch.org
>> Subject: Re: [ovs-discuss] OVN Questions
>> 
>> Since the path is not being updated correctly, from build directory 
>> (in my case ovs/_gcc), check and show the output of
>> 
>> sh -x ../tutorial/ovs-sandbox --ovn
>> 
>> 
>> Output should include something similar to below
>> 
>> .
>> .
>> + PATH=/home/dball/openvswitch/ovs/_gcc/ovn/controller:/home/dball/op
>> + e
>> + nv
>> + switch/ovs/_gcc/ovn/controller-vtep:/home/dball/openvswitch/ovs/_gc
>> + c
>> + /o
>> + vn/northd:/home/dball/openvswitch/ovs/_gcc/ovn/utilities:/home/dbal
>> + l
>> + /o
>> + penvswitch/ovs/_gcc/ovsdb:/home/dball/openvswitch/ovs/_gcc/vswitchd:
>> + /h
>> + ome/dball/openvswitch/ovs/_gcc/utilities:/home/dball/openvswitch/ov
>> + s
>> + /_
>> + gcc/vtep:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/b
>> + i
>> + n
>> + export PATH
>> .
>> .
>> 
>> 
>> 
>> 
>> 
>>> On 4/28/16, 1:01 PM, "discuss on behalf of Maurice Qureshi" <discuss-bounces at openvswitch.org on behalf of Maurice.Qureshi at calix.com> wrote:
>>> 
>>> Hi Justin,
>>> 
>>> As you can see from the logs below,
>>> 
>>> 1. ovn-nbctl doesn't work
>>> 2. The output of all the commands you have asked for is provided 
>>> without any breaks 3. "git log" shows the last commit of Apr. 27th
>>> 
>>> mqureshi at vmexa-mqureshi:~/ovs$ make sandbox SANDBOXFLAGS="--ovn"
>>> make  all-recursive
>>> make[1]: Entering directory `/home/mqureshi/ovs'
>>> Making all in datapath
>>> make[2]: Entering directory `/home/mqureshi/ovs/datapath'
>>> make[3]: Entering directory `/home/mqureshi/ovs/datapath'
>>> make[3]: Leaving directory `/home/mqureshi/ovs/datapath'
>>> make[2]: Leaving directory `/home/mqureshi/ovs/datapath'
>>> make[2]: Entering directory `/home/mqureshi/ovs'
>>> make[3]: Entering directory `/home/mqureshi/ovs/datapath'
>>> make[3]: `distfiles' is up to date.
>>> make[3]: Leaving directory `/home/mqureshi/ovs/datapath'
>>> make[2]: Leaving directory `/home/mqureshi/ovs'
>>> make[1]: Leaving directory `/home/mqureshi/ovs'
>>> cd ./tutorial && MAKE=make ./ovs-sandbox -b /home/mqureshi/ovs --ovn
>>> make[1]: Entering directory `/home/mqureshi/ovs'
>>> /bin/mkdir -p '/home/mqureshi/ovs/tutorial/sandbox/man/man1'
>>> /usr/bin/install -c -m 644 utilities/ovs-pcap.1 utilities/ovs-tcpundump.1 ovsdb/ovsdb-tool.1 ovsdb/ovsdb-client.1 ovsdb/ovsdb-server.1 '/home/mqureshi/ovs/tutorial/sandbox/man/man1'
>>> /bin/mkdir -p '/home/mqureshi/ovs/tutorial/sandbox/man/man5'
>>> /usr/bin/install -c -m 644 vswitchd/ovs-vswitchd.conf.db.5 vtep/vtep.5 ovn/ovn-sb.5 ovn/ovn-nb.5 '/home/mqureshi/ovs/tutorial/sandbox/man/man5'
>>> /bin/mkdir -p '/home/mqureshi/ovs/tutorial/sandbox/man/man7'
>>> /usr/bin/install -c -m 644 ovn/ovn-architecture.7 '/home/mqureshi/ovs/tutorial/sandbox/man/man7'
>>> /bin/mkdir -p '/home/mqureshi/ovs/tutorial/sandbox/man/man8'
>>> /usr/bin/install -c -m 644 utilities/ovs-appctl.8 utilities/ovs-ctl.8 utilities/ovs-testcontroller.8 utilities/ovs-dpctl.8 utilities/ovs-dpctl-top.8 utilities/ovs-l3ping.8 utilities/ovs-ofctl.8 utilities/ovs-parse-backtrace.8 utilities/ovs-pki.8 utilities/ovs-vlan-bug-workaround.8 utilities/ovs-test.8 utilities/ovs-vlan-test.8 utilities/ovs-vsctl.8 utilities/bugtool/ovs-bugtool.8 vswitchd/ovs-vswitchd.8 vtep/vtep-ctl.8 ovn/controller/ovn-controller.8 ovn/controller-vtep/ovn-controller-vtep.8 ovn/northd/ovn-northd.8 ovn/utilities/ovn-ctl.8 ovn/utilities/ovn-nbctl.8 ovn/utilities/ovn-sbctl.8 '/home/mqureshi/ovs/tutorial/sandbox/man/man8'
>>> make[1]: Leaving directory `/home/mqureshi/ovs'
>>> 
>>> 
>>> 
>>> --------------------------------------------------------------------
>>> -
>>> - You are running in a dummy Open vSwitch environment.  You can use 
>>> ovs-vsctl, ovs-ofctl, ovs-appctl, and other tools to work with the 
>>> dummy switch.
>>> 
>>> This environment also has the OVN daemons and databases enabled.
>>> You can use ovn-nbctl and ovn-sbctl to interact with the OVN databases.
>>> 
>>> Log files, pidfiles, and the configuration database are in the 
>>> "sandbox" subdirectory.
>>> 
>>> Exit the shell to kill the running daemons.
>>> mqureshi at vmexa-mqureshi:~/ovs/tutorial$ echo $PATH 
>>> /tools/sw/exa_2.0/exa-tools:/usr/lib64/ccache:/usr/local/bin:/usr/bi
>>> n :/ bin:/usr/local/sbin:/usr/sbin:/home/mqureshi/bin:/sbin
>>> mqureshi at vmexa-mqureshi:~/ovs/tutorial$ ovn-nbctl show
>>> ovn-nbctl: command not found
>>> mqureshi at vmexa-mqureshi:~/ovs/tutorial$ git log commit
>>> a78350d91a67f977608c2f47b4b17d6e045e7930
>>> Author: Pravin B Shelar <pshelar at ovn.org>
>>> Date:   Wed Apr 27 14:57:33 2016 -0700
>>> 
>>>  stt: linearize for CONFIG_SLUB case
>>> 
>>> 
>>> -----Original Message-----
>>> From: Justin Pettit [mailto:jpettit at ovn.org]
>>> Sent: Thursday, April 28, 2016 12:11 PM
>>> To: Maurice Qureshi <Maurice.Qureshi at calix.com>
>>> Cc: discuss at openvswitch.org
>>> Subject: Re: [ovs-discuss] OVN Questions
>>> 
>>> And "ovn-nbctl" doesn't w?ork?  Can you show me all the commands 
>>> I've asked and their output without any breaks
>>> 
>>>   - make sandbox SANDBOXFLAGS="--ovn"
>>>   - echo $PATH
>>>   - ovn-nbctl show
>>> 
>>> Are you pulling the source code from the tip of master?
>>> 
>>> --Justin
>>> 
>>> 
>>>> On Apr 28, 2016, at 1:59 PM, Maurice Qureshi <Maurice.Qureshi at calix.com> wrote:
>>>> 
>>>> Below is the output of "make sandbox" from the time when I typed "make sandbox SANDBOXFLAGS="--ovn". Do I need to turn on some other flag also to make it more verbose.
>>>> 
>>>> mqureshi at vmexa-mqureshi:~/ovs$ make sandbox SANDBOXFLAGS="--ovn"          
>>>> make  all-recursive
>>>> make[1]: Entering directory `/home/mqureshi/ovs'
>>>> Making all in datapath
>>>> make[2]: Entering directory `/home/mqureshi/ovs/datapath'
>>>> make[3]: Entering directory `/home/mqureshi/ovs/datapath'
>>>> make[3]: Leaving directory `/home/mqureshi/ovs/datapath'
>>>> make[2]: Leaving directory `/home/mqureshi/ovs/datapath'
>>>> make[2]: Entering directory `/home/mqureshi/ovs'
>>>> make[3]: Entering directory `/home/mqureshi/ovs/datapath'
>>>> make[3]: `distfiles' is up to date.
>>>> make[3]: Leaving directory `/home/mqureshi/ovs/datapath'
>>>> make[2]: Leaving directory `/home/mqureshi/ovs'
>>>> make[1]: Leaving directory `/home/mqureshi/ovs'
>>>> cd ./tutorial && MAKE=make ./ovs-sandbox -b /home/mqureshi/ovs 
>>>> --ovn
>>>> make[1]: Entering directory `/home/mqureshi/ovs'
>>>> /bin/mkdir -p '/home/mqureshi/ovs/tutorial/sandbox/man/man1'
>>>> /usr/bin/install -c -m 644 utilities/ovs-pcap.1 utilities/ovs-tcpundump.1 ovsdb/ovsdb-tool.1 ovsdb/ovsdb-client.1 ovsdb/ovsdb-server.1 '/home/mqureshi/ovs/tutorial/sandbox/man/man1'
>>>> /bin/mkdir -p '/home/mqureshi/ovs/tutorial/sandbox/man/man5'
>>>> /usr/bin/install -c -m 644 vswitchd/ovs-vswitchd.conf.db.5 vtep/vtep.5 ovn/ovn-sb.5 ovn/ovn-nb.5 '/home/mqureshi/ovs/tutorial/sandbox/man/man5'
>>>> /bin/mkdir -p '/home/mqureshi/ovs/tutorial/sandbox/man/man7'
>>>> /usr/bin/install -c -m 644 ovn/ovn-architecture.7 '/home/mqureshi/ovs/tutorial/sandbox/man/man7'
>>>> /bin/mkdir -p '/home/mqureshi/ovs/tutorial/sandbox/man/man8'
>>>> /usr/bin/install -c -m 644 utilities/ovs-appctl.8 utilities/ovs-ctl.8 utilities/ovs-testcontroller.8 utilities/ovs-dpctl.8 utilities/ovs-dpctl-top.8 utilities/ovs-l3ping.8 utilities/ovs-ofctl.8 utilities/ovs-parse-backtrace.8 utilities/ovs-pki.8 utilities/ovs-vlan-bug-workaround.8 utilities/ovs-test.8 utilities/ovs-vlan-test.8 utilities/ovs-vsctl.8 utilities/bugtool/ovs-bugtool.8 vswitchd/ovs-vswitchd.8 vtep/vtep-ctl.8 ovn/controller/ovn-controller.8 ovn/controller-vtep/ovn-controller-vtep.8 ovn/northd/ovn-northd.8 ovn/utilities/ovn-ctl.8 ovn/utilities/ovn-nbctl.8 ovn/utilities/ovn-sbctl.8 '/home/mqureshi/ovs/tutorial/sandbox/man/man8'
>>>> make[1]: Leaving directory `/home/mqureshi/ovs'
>>>> 
>>>> 
>>>> 
>>>> -------------------------------------------------------------------
>>>> -
>>>> -
>>>> - You are running in a dummy Open vSwitch environment.  You can use 
>>>> ovs-vsctl, ovs-ofctl, ovs-appctl, and other tools to work with the 
>>>> dummy switch.
>>>> 
>>>> This environment also has the OVN daemons and databases enabled.
>>>> You can use ovn-nbctl and ovn-sbctl to interact with the OVN databases.
>>>> 
>>>> Log files, pidfiles, and the configuration database are in the 
>>>> "sandbox" subdirectory.
>>>> 
>>>> Exit the shell to kill the running daemons.
>>>> mqureshi at vmexa-mqureshi:~/ovs/tutorial
>>>> 
>>>> -----Original Message-----
>>>> From: Justin Pettit [mailto:jpettit at ovn.org]
>>>> Sent: Wednesday, April 27, 2016 8:51 PM
>>>> To: Maurice Qureshi <Maurice.Qureshi at calix.com>
>>>> Cc: discuss at openvswitch.org
>>>> Subject: Re: [ovs-discuss] OVN Questions
>>>> 
>>>> You need to run "make sandbox" as shown in the presentation. It doesn't matter if you run just "make" first. 
>>>> 
>>>> I noticed if I copied and pasted from those slides it didn't work properly because the quotes were stylized and not standard. However since you still haven't shown the output that I've requested a couple of times, I can't say for sure. 
>>>> 
>>>> --Justin
>>>> 
>>>> 
>>>>> On Apr 27, 2016, at 8:32 PM, Maurice Qureshi <Maurice.Qureshi at calix.com> wrote:
>>>>> 
>>>>> Hi Justin,
>>>>> 
>>>>> In the link
>>>>> "https://urldefense.proofpoint.com/v2/url?u=http-3A__benpfaff.org_
>>>>> -
>>>>> 7
>>>>> Eblp_ovn-2Dlbnl.pdf&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVM
>>>>> N
>>>>> t
>>>>> Xt-uEs&r=dGZmbKhBG9tJHY4odedsGA&m=h-7ifxFpG2ex-XB1MvTQ7hWetF2L7yBD
>>>>> P w 0hJkPT4E8&s=qkb1Bdud9DEKR0ds7WsyvCJOd_EcXrZSrYKT9ZV15R0&e= ", 
>>>>> there is a slide titled as "Test #1 - ovs-sandbox". Based on this 
>>>>> slide, I followed these steps
>>>>> 
>>>>> 1: git clone
>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__github.com_ope
>>>>> n
>>>>> v
>>>>> switch_ovs.git&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-
>>>>> u
>>>>> E
>>>>> s&r=dGZmbKhBG9tJHY4odedsGA&m=h-7ifxFpG2ex-XB1MvTQ7hWetF2L7yBDPw0hJ
>>>>> k P T4E8&s=vNl5O-6U1y_qyt69zrgI7bInTAHaMDsy0Nk_mc1hymA&e=
>>>>> 2: cd ovs
>>>>> 3a:  ./boot.sh
>>>>> 3b: ./configure
>>>>> 3c: make
>>>>> 4: make sandbox SANDBOXFLAGS="--ovn"
>>>>> 
>>>>> Should I have done "make sandbox", instead of "make" in step "3c" above. These slides may be old, so I may be missing some important step.
>>>>> 
>>>>> Thanks
>>>>> Maurice
>>>>> 
>>>>> -----Original Message-----
>>>>> From: Justin Pettit [mailto:jpettit at ovn.org]
>>>>> Sent: Wednesday, April 27, 2016 3:02 PM
>>>>> To: Maurice Qureshi <Maurice.Qureshi at calix.com>
>>>>> Cc: discuss at openvswitch.org
>>>>> Subject: Re: [ovs-discuss] OVN Questions
>>>>> 
>>>>> It doesn't look like it got started properly. Can you provide all the output starting from when you type "make sandbox"?
>>>>> 
>>>>> --Justin
>>>>> 
>>>>> 
>>>>>> On Apr 27, 2016, at 4:53 PM, Maurice Qureshi <Maurice.Qureshi at calix.com> wrote:
>>>>>> 
>>>>>> Hi Justin,
>>>>>> 
>>>>>> Below is the output of echo $PATH. The output of "make sandbox" is attached.
>>>>>> 
>>>>>> mqureshi at vmexa-mqureshi:~/ovs/tutorial$ echo $PATH
>>>>>> /tools/sw/exa_2.0/exa-tools:/usr/lib64/ccache:/usr/local/bin:/usr/bin:
>>>>>> /bin:/usr/local/sbin:/usr/sbin:/home/mqureshi/bin:/sbin
>>>>>> mqureshi at vmexa-mqureshi:~/ovs/tutorial$
>>>>>> 
>>>>>> Thanks
>>>>>> Maurice
>>>>>> 
>>>>>> -----Original Message-----
>>>>>> From: Justin Pettit [mailto:jpettit at ovn.org]
>>>>>> Sent: Wednesday, April 27, 2016 10:58 AM
>>>>>> To: Maurice Qureshi <Maurice.Qureshi at calix.com>
>>>>>> Cc: discuss at openvswitch.org
>>>>>> Subject: Re: [ovs-discuss] OVN Questions
>>>>>> 
>>>>>> Can you show all the commands and output from when you run "make sandbox"? Also, what does "echo $PATH" show when running in the sanbox shell?
>>>>>> 
>>>>>> --Justin
>>>>>> 
>>>>>> 
>>>>>>> On Apr 27, 2016, at 12:37 PM, Maurice Qureshi <Maurice.Qureshi at calix.com> wrote:
>>>>>>> 
>>>>>>> Hi Justin,
>>>>>>> 
>>>>>>> None of the ovn- commands are available.
>>>>>>> 
>>>>>>> mqureshi at vmexa-mqureshi:~/ovs/tutorial$  ovn-nbctl lswitch-add
>>>>>>> sw0
>>>>>>> ovn-nbctl: command not found
>>>>>>> mqureshi at vmexa-mqureshi:~/ovs/tutorial$ whereis  ovn-nbctl
>>>>>>> ovn-nbctl:
>>>>>>> mqureshi at vmexa-mqureshi:~/ovs/tutorial$ which ovn-nbctl 
>>>>>>> mqureshi at vmexa-mqureshi:~/ovs/tutorial$
>>>>>>> 
>>>>>>> -----Original Message-----
>>>>>>> From: Justin Pettit [mailto:jpettit at ovn.org]
>>>>>>> Sent: Tuesday, April 26, 2016 10:51 PM
>>>>>>> To: Maurice Qureshi <Maurice.Qureshi at calix.com>
>>>>>>> Cc: discuss at openvswitch.org
>>>>>>> Subject: Re: [ovs-discuss] OVN Questions
>>>>>>> 
>>>>>>> 
>>>>>>>> On Apr 26, 2016, at 8:16 PM, Maurice Qureshi <Maurice.Qureshi at calix.com> wrote:
>>>>>>>> 
>>>>>>>> Have following questions on OVN
>>>>>>>> 
>>>>>>>> 1.      Tried to use the ovs sandbox as described in https://urldefense.proofpoint.com/v2/url?u=http-3A__benpfaff.org_-7Eblp_ovn-2Dlbnl.pdf&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=dGZmbKhBG9tJHY4odedsGA&m=h-7ifxFpG2ex-XB1MvTQ7hWetF2L7yBDPw0hJkPT4E8&s=qkb1Bdud9DEKR0ds7WsyvCJOd_EcXrZSrYKT9ZV15R0&e= . Got following message, but the commands like ovn-nbctl are failing. Both OVN daemons (ovn-northd and ovn-controller) are running in addition to ovn-controller-vtep. I am using Ubuntu 14.04 LTS
>>>>>>> 
>>>>>>> What error are you seeing?  I checked out the latest code, and it works for me.
>>>>>>> 
>>>>>>> --Justin
>>>>>> 
>>>>>> <make_sandbox.txt>
>>> 
>>> _______________________________________________
>>> discuss mailing list
>>> discuss at openvswitch.org
>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__openvswitch.org_
>>> m
>>> ai
>>> lman_listinfo_discuss&d=BQIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVM
>>> N
>>> tX
>>> t-uEs&r=dGZmbKhBG9tJHY4odedsGA&m=h-7ifxFpG2ex-XB1MvTQ7hWetF2L7yBDPw0
>>> h Jk PT4E8&s=NHLZJ93vwRCOZ-nh0yGFIv1__R2mcjdUDxm9icqw2MA&e=
>

More information about the discuss mailing list