[ovs-dev] 回复: Re: [ovs-discuss] The encryption problem about vxlan with Ipsec
Mark Gray
mark.d.gray at redhat.com
Tue Jul 6 17:38:48 UTC 2021
On 06/07/2021 03:51, liumk at rc.inesa.com wrote:
>> On 05/07/2021 09:08, liumk at rc.inesa.com wrote:
>>> Dear OVS Team,
>>>
>>> We are seeing an issue occurred in OVS version 2.11.0. Right now we are trying to encrypt our VXLAN packets in communication. The problem is, when the tunnel is already built between host 1 and host 2, if we run the command to let host 1 ping host 2 and at the same time, run tcpdump command to capture the processing packets, we observed that packets from host 1 to host 2 are well encrypted while packets from host 2 to host 1 are in plaintext (it is supposed to be encrypted too).
>>>
>>> Here is the whole processes of how we found this problem:
>>>
>>> Firstly, we install OVS 2.11.0 and ovs-ipsec on both two centos VMs in virtual box.
>>> External IP of host 1 is: 10.200.46.186, host 2 is: 10.200.46.187
>>>
>>> Next, we run the following command in order to build up a VXLAN tunnel: (used https://docs.openvswitch.org/en/latest/tutorials/ipsec/ as a reference)
>>> In host 1:
>>> # ovs-vsctl add-br br-ipsec
>>> # ip addr add 192.0.0.1/24 dev br-ipsec
>>> # ip link set br-ipsec up
>>>
>>> In host 2:
>>> # ovs-vsctl add-br br-ipsec
>>> # ip addr add 192.0.0.2/24 dev br-ipsec
>>> # ip link set br-ipsec up
>>>
>>> And then, we run this command in host 1:
>>> ovs-vsctl add-port br-ipsec tun -- set interface tun type=vxlan options:remote_ip=10.200.46.187 options:psk=swordfish
>>>
>>> And this command in host 2:
>>> ovs-vsctl add-port br-ipsec tun -- set interface tun type=vxlan options:remote_ip=10.200.46.186 options:psk=swordfish
>>>
>>> The network topology looks like this:
>>> Host1:
>>> [root at ovs-ipsec-001 ~]# ovs-vsctl show
>>> 81d4eef4-ebd7-4cb4-b1ab-d2c4a4469496
>>> Bridge br-ipsec
>>> Port tun
>>> Interface tun
>>> type: vxlan
>>> options: {psk=swordfish, remote_ip="10.200.46.187"}
>>> Port br-ipsec
>>> Interface br-ipsec
>>> type: internal
>>> ovs_version: "2.11.0"
>>>
>>>
>>> Host2:
>>> [root at ovs-ipsec-002 ~]# ovs-vsctl show
>>> f2bff28e-4adc-4c8c-941a-602d249e6d6e
>>> Bridge br-ipsec
>>> Port br-ipsec
>>> Interface br-ipsec
>>> type: internal
>>> Port tun
>>> Interface tun
>>> type: vxlan
>>> options: {psk=swordfish, remote_ip="10.200.46.186"}
>>> ovs_version: "2.11.0"
>>>
>>>
>>> The ping connection between host 1 and host 2 is good, however, the tcpdump processes shows the "come back" packets of ping is in plaintext:
>>
>> It looks like the "request" is clear text here (not the reply) but ....
>>
>>> [root at ovs-ipsec-001 ~]# tcpdump -e -ni any net 10.200.46.187
>>
>> .... it may be because you are specifying "-i any" above. You should run
>> tcpdump on the physical interface ( -i <physical interface name ). I
>> think you are seeing the same packet twice.
>
> We've done this experiment again and tried the command you adviced us as this: "tcpdump -ni enp0s3 net 10.200.46.187".
>
> However, the result still shows that the "coming back" packets are in plaintext:
I added ovs-dev to the thread again.
My bad, I was reading the trace incorrectly.
I guess the remote side is not sending out encrypted traffic. Is there
anything suspect in the logs?
>
> Here is the result of tcpdump:
> [root at ovs-ipsec-001 ~]# tcpdump -ni enp0s3 net 10.200.46.187
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on enp0s3, link-type EN10MB (Ethernet), capture size 262144 bytes
> 10:35:51.070218 IP 10.200.46.186 > 10.200.46.187: ESP(spi=0x2ff055b4,seq=0x6), length 148
> 10:35:51.071096 IP 10.200.46.187.52430 > 10.200.46.186.4789: VXLAN, flags [I] (0x08), vni 0
> IP 192.0.0.2 > 192.0.0.1: ICMP echo reply, id 2760, seq 5, length 64
> 10:35:51.645724 IP 10.200.46.186.isakmp > 10.200.46.187.isakmp: isakmp: parent_sa child_sa
> 10:35:51.647024 IP 10.200.46.187.isakmp > 10.200.46.186.isakmp: isakmp: parent_sa child_sa[IR]
> 10:35:52.072198 IP 10.200.46.186 > 10.200.46.187: ESP(spi=0x2ff055b4,seq=0x7), length 148
> 10:35:52.072970 IP 10.200.46.187.52430 > 10.200.46.186.4789: VXLAN, flags [I] (0x08), vni 0
> IP 192.0.0.2 > 192.0.0.1: ICMP echo reply, id 2760, seq 6, length 64
> 10:35:52.079039 ARP, Request who-has 10.200.46.187 tell 10.200.46.186, length 28
> 10:35:52.079520 ARP, Reply 10.200.46.187 is-at 08:00:27:b1:a5:c0, length 46
> 10:35:52.079771 IP 10.200.46.187.35886 > 10.200.46.186.4789: VXLAN, flags [I] (0x08), vni 0
> ARP, Request who-has 192.0.0.1 tell 192.0.0.2, length 28
> 10:35:52.080149 IP 10.200.46.186 > 10.200.46.187: ESP(spi=0x2ff055b4,seq=0x8), length 92
> 10:35:52.146977 IP 10.200.46.186.isakmp > 10.200.46.187.isakmp: isakmp: parent_sa child_sa
> 10:35:52.147701 IP 10.200.46.187.isakmp > 10.200.46.186.isakmp: isakmp: child_sa child_sa[I]
> 10:35:52.648676 IP 10.200.46.186.isakmp > 10.200.46.187.isakmp: isakmp: parent_sa child_sa
> 10:35:52.649707 IP 10.200.46.187.isakmp > 10.200.46.186.isakmp: isakmp: child_sa child_sa[I]
> 10:35:53.073706 IP 10.200.46.186 > 10.200.46.187: ESP(spi=0x2ff055b4,seq=0x9), length 148
> 10:35:53.074533 IP 10.200.46.187.52430 > 10.200.46.186.4789: VXLAN, flags [I] (0x08), vni 0
> IP 192.0.0.2 > 192.0.0.1: ICMP echo reply, id 2760, seq 7, length 64
> 10:35:53.506174 IP 10.200.46.187.isakmp > 10.200.46.186.isakmp: isakmp: child_sa child_sa[I]
> 10:35:53.650223 IP 10.200.46.186.isakmp > 10.200.46.187.isakmp: isakmp: parent_sa child_sa
> 10:35:53.651311 IP 10.200.46.187.isakmp > 10.200.46.186.isakmp: isakmp: child_sa child_sa[I]
> 10:35:54.075673 IP 10.200.46.186 > 10.200.46.187: ESP(spi=0x2ff055b4,seq=0xa), length 148
> 10:35:54.076278 IP 10.200.46.187.52430 > 10.200.46.186.4789: VXLAN, flags [I] (0x08), vni 0
> IP 192.0.0.2 > 192.0.0.1: ICMP echo reply, id 2760, seq 8, length 64
> 10:35:55.075808 IP 10.200.46.186 > 10.200.46.187: ESP(spi=0x2ff055b4,seq=0xb), length 148
> 10:35:55.076479 IP 10.200.46.187.52430 > 10.200.46.186.4789: VXLAN, flags [I] (0x08), vni 0
> IP 192.0.0.2 > 192.0.0.1: ICMP echo reply, id 2760, seq 9, length 64
>
> [root at ovs-ipsec-001 ~]# tcpdump -ni enp0s3 net 10.200.46.187
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on enp0s3, link-type EN10MB (Ethernet), capture size 262144 bytes
> 10:35:51.070218 IP 10.200.46.186 > 10.200.46.187: ESP(spi=0x2ff055b4,seq=0x6), length 148
> 10:35:51.071096 IP 10.200.46.187.52430 > 10.200.46.186.4789: VXLAN, flags [I] (0x08), vni 0
> IP 192.0.0.2 > 192.0.0.1: ICMP echo reply, id 2760, seq 5, length 64
> 10:35:51.645724 IP 10.200.46.186.isakmp > 10.200.46.187.isakmp: isakmp: parent_sa child_sa
> 10:35:51.647024 IP 10.200.46.187.isakmp > 10.200.46.186.isakmp: isakmp: parent_sa child_sa[IR]
> 10:35:52.072198 IP 10.200.46.186 > 10.200.46.187: ESP(spi=0x2ff055b4,seq=0x7), length 148
> 10:35:52.072970 IP 10.200.46.187.52430 > 10.200.46.186.4789: VXLAN, flags [I] (0x08), vni 0
> IP 192.0.0.2 > 192.0.0.1: ICMP echo reply, id 2760, seq 6, length 64
> 10:35:52.079039 ARP, Request who-has 10.200.46.187 tell 10.200.46.186, length 28
> 10:35:52.079520 ARP, Reply 10.200.46.187 is-at 08:00:27:b1:a5:c0, length 46
> 10:35:52.079771 IP 10.200.46.187.35886 > 10.200.46.186.4789: VXLAN, flags [I] (0x08), vni 0
> ARP, Request who-has 192.0.0.1 tell 192.0.0.2, length 28
> 10:35:52.080149 IP 10.200.46.186 > 10.200.46.187: ESP(spi=0x2ff055b4,seq=0x8), length 92
> 10:35:52.146977 IP 10.200.46.186.isakmp > 10.200.46.187.isakmp: isakmp: parent_sa child_sa
> 10:35:52.147701 IP 10.200.46.187.isakmp > 10.200.46.186.isakmp: isakmp: child_sa child_sa[I]
> 10:35:52.648676 IP 10.200.46.186.isakmp > 10.200.46.187.isakmp: isakmp: parent_sa child_sa
> 10:35:52.649707 IP 10.200.46.187.isakmp > 10.200.46.186.isakmp: isakmp: child_sa child_sa[I]
> 10:35:53.073706 IP 10.200.46.186 > 10.200.46.187: ESP(spi=0x2ff055b4,seq=0x9), length 148
> 10:35:53.074533 IP 10.200.46.187.52430 > 10.200.46.186.4789: VXLAN, flags [I] (0x08), vni 0
> IP 192.0.0.2 > 192.0.0.1: ICMP echo reply, id 2760, seq 7, length 64
>>
>>> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
>>> listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
>>> 16:02:53.267379 In 08:00:27:b1:a5:c0 ethertype IPv4 (0x0800), length 150: 10.200.46.187.44471 > 10.200.46.186.4789: VXLAN, flags [I] (0x08), vni 0
>>> 02:39:d5:12:45:4a > 9a:2f:5d:68:cc:4b, ethertype IPv4 (0x0800), length 98: 192.0.0.2 > 192.0.0.1: ICMP echo request, id 2717, seq 1388, length 64
>>> 16:02:53.267549 Out 08:00:27:ca:9b:1a ethertype IPv4 (0x0800), length 184: 10.200.46.186 > 10.200.46.187: ESP(spi=0x6914b456,seq=0x6d7), length 148
>>> 16:02:54.268637 In 08:00:27:b1:a5:c0 ethertype IPv4 (0x0800), length 150: 10.200.46.187.44471 > 10.200.46.186.4789: VXLAN, flags [I] (0x08), vni 0
>>> 02:39:d5:12:45:4a > 9a:2f:5d:68:cc:4b, ethertype IPv4 (0x0800), length 98: 192.0.0.2 > 192.0.0.1: ICMP echo request, id 2717, seq 1389, length 64
>>> 16:02:54.268844 Out 08:00:27:ca:9b:1a ethertype IPv4 (0x0800), length 184: 10.200.46.186 > 10.200.46.187: ESP(spi=0x6914b456,seq=0x6d8), length 148
>>> 16:02:55.270073 In 08:00:27:b1:a5:c0 ethertype IPv4 (0x0800), length 150: 10.200.46.187.44471 > 10.200.46.186.4789: VXLAN, flags [I] (0x08), vni 0
>>> 02:39:d5:12:45:4a > 9a:2f:5d:68:cc:4b, ethertype IPv4 (0x0800), length 98: 192.0.0.2 > 192.0.0.1: ICMP echo request, id 2717, seq 1390, length 64
>>> 16:02:55.270265 Out 08:00:27:ca:9b:1a ethertype IPv4 (0x0800), length 184: 10.200.46.186 > 10.200.46.187: ESP(spi=0x6914b456,seq=0x6d9), length 148
>>> 16:02:56.272043 In 08:00:27:b1:a5:c0 ethertype IPv4 (0x0800), length 150: 10.200.46.187.44471 > 10.200.46.186.4789: VXLAN, flags [I] (0x08), vni 0
>>> 02:39:d5:12:45:4a > 9a:2f:5d:68:cc:4b, ethertype IPv4 (0x0800), length 98: 192.0.0.2 > 192.0.0.1: ICMP echo request, id 2717, seq 1391, length 64
>>> 16:02:56.272231 Out 08:00:27:ca:9b:1a ethertype IPv4 (0x0800), length 184: 10.200.46.186 > 10.200.46.187: ESP(spi=0x6914b456,seq=0x6da), length 148
>>> 16:02:57.274848 In 08:00:27:b1:a5:c0 ethertype IPv4 (0x0800), length 150: 10.200.46.187.44471 > 10.200.46.186.4789: VXLAN, flags [I] (0x08), vni 0
>>> 02:39:d5:12:45:4a > 9a:2f:5d:68:cc:4b, ethertype IPv4 (0x0800), length 98: 192.0.0.2 > 192.0.0.1: ICMP echo request, id 2717, seq 1392, length 64
>>> 16:02:57.275044 Out 08:00:27:ca:9b:1a ethertype IPv4 (0x0800), length 184: 10.200.46.186 > 10.200.46.187: ESP(spi=0x6914b456,seq=0x6db), length 148
>>>
>>> As is shown, when the connection come back from host 2 to host 1, it is not encrypted, and it is obvious that the original VXLAN and ICMP packets are revealed.
>>>
>>> Can you please help us figure out why this happen? Or any other alternative solution of encrypting VXLAN packets?
>>>
>>> Thank you very much!
>>>
>>> Bests,
>>> Yiming Shi
>>>
>>>
>>> _______________________________________________
>>> discuss mailing list
>>> discuss at openvswitch.org
>>> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>>>
>>
>
More information about the dev
mailing list