Source IP being SNATed when using multicast on OpenVZ

Makenai

Hello!

Description

I'm working on setting up a testbed for OSPF routing and currently experiencing abnormalities when running OSPF over GRE inside a OpenVZ container (Quagga + ospfd).
I have OSPF setup in a similar manner in a KVM VPS without any issues.

Servers

The "faulty"

Debian 7 (wheezy)
quagga  - 0.99.22.4-1+wheezy3+deb7u3
kernel  - 2.6.32-042stab127.2
external IP - 185.x.x.x.x
GRE IP - 192.168.254.14

The "working"

Debian 8 (jessie)
quagga 0.99.23.1-1+deb8u5
kernel  3.16.0-4-amd64

The "router"

external IP - 95.x.x.x.x
GRE IP - 192.168.254.13```

##Issue
Something on "faulty" is replacing the packet IP address to external one[1][2] which venet0 is using and as such the OSPF packet on receiving router gets incorrect source IP address over GRE.

##What I've tried so far
  1. Added TRACE to iptables to figure out where the packet could have its IP address SNATed [3]
  2. Tested if ICMP between routers is working correctly - correct source and destination addresses on both sides [4]
  3. Enabled OSPF debugging in Quagga to make sure that the packet is indeed being sent out the correct interface [5]
  4. Googled for the last 4 hours to figure out what could cause this, to no avail

##Questions
  1. What else should I look for/debug?
  2. Have any of you done similar shenanigans in OpenVZ? Were you successful?
##Appendix
### [1] - tcpdump on "faulty" server

04:25:28.480015 IP (tos 0xc0, ttl 1, id 53911, offset 0, flags [none], proto OSPF (89), length 68)
185.x.x.x.x > 224.0.0.5: OSPFv2, Hello, length 48
Router-ID 192.168.254.14, Backbone Area, Authentication Type: none (0)
Options [External]
Hello Timer 10s, Dead Timer 40s, Mask 0.0.0.0, Priority 1
Neighbor List:
192.168.254.1 ```

[2] - Wireshark output from "router"

[3] - iptables trace output - without any indication of SNAT being done

See https://pastebin.com/raw/Pd5G8YDk due to Cloudflare issues

[4] - ICMP between "faulty" and "router" over GRE

04:48:50.605565 IP 192.168.254.13 > 192.168.254.14: ICMP echo request, id 16875, seq 0, length 36
04:48:50.605588 IP 192.168.254.14 > 192.168.254.13: ICMP echo reply, id 16875, seq 0, length 36
04:49:08.501924 IP 192.168.254.14 > 192.168.254.13: ICMP echo request, id 5603, seq 1, length 64
04:49:08.503550 IP 192.168.254.13 > 192.168.254.14: ICMP echo reply, id 5603, seq 1, length 64

[5] - OSPF packet debugging logs from ospfd

See https://pastebin.com/raw/AZzf6iTD due to Cloudflare issues

[6] iptable rules dump

See https://pastebin.com/raw/QHaV9pPQ due to Cloudflare issue

Makenai

Rebooting VPS fixed the issue, haha... Does anyone have idea why?

Clouvider

Why would you run any routing with OVZ container ? It was never made for it.

hzr

I'm honestly surprised anything at all even works with openvz.

Makenai

@Clouvider said:
Why would you run any routing with OVZ container ? It was never made for it.

There are a few reasons

1. It is cheaper, the OVZ VPS is around 2ms away from me and other providers are ridiculously expensive around here
2. I was curious if it would work or there are any OVZ limitations which will stop me
3. A lot of things weren't made for it, but when discovered what they could do, people started using them, just look at microwaves )
4. Well this is just for my own... guilty pleasure.

Clouvider

OpenVZ has limitations that prevent you from proper routing so there's that. It just wasn't made for it ;-).

Makenai

@Clouvider said:
OpenVZ has limitations that prevent you from proper routing so there's that. It just wasn't made for it ;-).

This was a bit ago, but IIRC it was not a routing issue. It was my own fault not dealing with iptables properly and I resolved it by completely redoing iptables rules.

Now I have 2 OVZ servers running Quagga OSPF over GRE without any issues.
For the sake of curiosity, what kind of OVZ routing issues have you experienced?