[ovs-discuss] The encrypted problem about vxlan with Ipsec

[Mark Gray]

On 05/07/2021 09:08, liumk at rc.inesa.com wrote:
> Dear OVS Team,
>  
> We are seeing an issue occurred in OVS version 2.11.0. Right now we are trying to encrypt our VXLAN packets in communication. The problem is, when the tunnel is already built between host 1 and host 2, if we run the command to let host 1 ping host 2 and at the same time, run tcpdump command to capture the processing packets, we observed that packets from host 1 to host 2 are well encrypted while packets from host 2 to host 1 are in plaintext (it is supposed to be encrypted too).
>  
> Here is the whole processes of how we found this problem:
>  
> Firstly, we install OVS 2.11.0 and ovs-ipsec on both two centos VMs in virtual box.
> External IP of host 1 is: 10.200.46.186, host 2 is: 10.200.46.187
>  
> Next, we run the following command in order to build up a VXLAN tunnel: (used https://docs.openvswitch.org/en/latest/tutorials/ipsec/ as a reference)
> In host 1:
> # ovs-vsctl add-br br-ipsec
> # ip addr add 192.0.0.1/24 dev br-ipsec
> # ip link set br-ipsec up
>  
> In host 2:
> # ovs-vsctl add-br br-ipsec
> # ip addr add 192.0.0.2/24 dev br-ipsec
> # ip link set br-ipsec up
>  
> And then, we run this command in host 1:
> ovs-vsctl add-port br-ipsec tun -- set interface tun type=vxlan options:remote_ip=10.200.46.187 options:psk=swordfish
>  
> And this command in host 2:
> ovs-vsctl add-port br-ipsec tun -- set interface tun type=vxlan options:remote_ip=10.200.46.186 options:psk=swordfish
>  
> The network topology looks like this:
> Host1:
> [root at ovs-ipsec-001 ~]# ovs-vsctl show
> 81d4eef4-ebd7-4cb4-b1ab-d2c4a4469496
>     Bridge br-ipsec
>         Port tun
>             Interface tun
>                 type: vxlan
>                 options: {psk=swordfish, remote_ip="10.200.46.187"}
>         Port br-ipsec
>             Interface br-ipsec
>                 type: internal
>     ovs_version: "2.11.0"
>  
>  
> Host2:
> [root at ovs-ipsec-002 ~]# ovs-vsctl show
> f2bff28e-4adc-4c8c-941a-602d249e6d6e
>     Bridge br-ipsec
>         Port br-ipsec
>             Interface br-ipsec
>                 type: internal
>         Port tun
>             Interface tun
>                 type: vxlan
>                 options: {psk=swordfish, remote_ip="10.200.46.186"}
>     ovs_version: "2.11.0"
>  
>  
> The ping connection between host 1 and host 2 is good, however, the tcpdump processes shows the "come back" packets of ping is in plaintext:

It looks like the "request" is clear text here (not the reply) but ....

> [root at ovs-ipsec-001 ~]# tcpdump -e -ni any net 10.200.46.187

.... it may be because you are specifying "-i any" above. You should run
tcpdump on the physical interface ( -i <physical interface name ). I
think you are seeing the same packet twice.

> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
> 16:02:53.267379  In 08:00:27:b1:a5:c0 ethertype IPv4 (0x0800), length 150: 10.200.46.187.44471 > 10.200.46.186.4789: VXLAN, flags [I] (0x08), vni 0
> 02:39:d5:12:45:4a > 9a:2f:5d:68:cc:4b, ethertype IPv4 (0x0800), length 98: 192.0.0.2 > 192.0.0.1: ICMP echo request, id 2717, seq 1388, length 64
> 16:02:53.267549 Out 08:00:27:ca:9b:1a ethertype IPv4 (0x0800), length 184: 10.200.46.186 > 10.200.46.187: ESP(spi=0x6914b456,seq=0x6d7), length 148
> 16:02:54.268637  In 08:00:27:b1:a5:c0 ethertype IPv4 (0x0800), length 150: 10.200.46.187.44471 > 10.200.46.186.4789: VXLAN, flags [I] (0x08), vni 0
> 02:39:d5:12:45:4a > 9a:2f:5d:68:cc:4b, ethertype IPv4 (0x0800), length 98: 192.0.0.2 > 192.0.0.1: ICMP echo request, id 2717, seq 1389, length 64
> 16:02:54.268844 Out 08:00:27:ca:9b:1a ethertype IPv4 (0x0800), length 184: 10.200.46.186 > 10.200.46.187: ESP(spi=0x6914b456,seq=0x6d8), length 148
> 16:02:55.270073  In 08:00:27:b1:a5:c0 ethertype IPv4 (0x0800), length 150: 10.200.46.187.44471 > 10.200.46.186.4789: VXLAN, flags [I] (0x08), vni 0
> 02:39:d5:12:45:4a > 9a:2f:5d:68:cc:4b, ethertype IPv4 (0x0800), length 98: 192.0.0.2 > 192.0.0.1: ICMP echo request, id 2717, seq 1390, length 64
> 16:02:55.270265 Out 08:00:27:ca:9b:1a ethertype IPv4 (0x0800), length 184: 10.200.46.186 > 10.200.46.187: ESP(spi=0x6914b456,seq=0x6d9), length 148
> 16:02:56.272043  In 08:00:27:b1:a5:c0 ethertype IPv4 (0x0800), length 150: 10.200.46.187.44471 > 10.200.46.186.4789: VXLAN, flags [I] (0x08), vni 0
> 02:39:d5:12:45:4a > 9a:2f:5d:68:cc:4b, ethertype IPv4 (0x0800), length 98: 192.0.0.2 > 192.0.0.1: ICMP echo request, id 2717, seq 1391, length 64
> 16:02:56.272231 Out 08:00:27:ca:9b:1a ethertype IPv4 (0x0800), length 184: 10.200.46.186 > 10.200.46.187: ESP(spi=0x6914b456,seq=0x6da), length 148
> 16:02:57.274848  In 08:00:27:b1:a5:c0 ethertype IPv4 (0x0800), length 150: 10.200.46.187.44471 > 10.200.46.186.4789: VXLAN, flags [I] (0x08), vni 0
> 02:39:d5:12:45:4a > 9a:2f:5d:68:cc:4b, ethertype IPv4 (0x0800), length 98: 192.0.0.2 > 192.0.0.1: ICMP echo request, id 2717, seq 1392, length 64
> 16:02:57.275044 Out 08:00:27:ca:9b:1a ethertype IPv4 (0x0800), length 184: 10.200.46.186 > 10.200.46.187: ESP(spi=0x6914b456,seq=0x6db), length 148
>  
> As is shown, when the connection come back from host 2 to host 1, it is not encrypted, and it is obvious that the original VXLAN and ICMP packets are revealed.
>  
> Can you please help us figure out why this happen? Or any other alternative solution of encrypting VXLAN packets?
>  
> Thank you very much!
>  
> Bests,
> Yiming Shi
> 
> 
> _______________________________________________
> discuss mailing list
> discuss at openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>