Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Hetzner subnet with additional IPs - Driving me crazy for days!
New on LowEndTalk? Please Register and read our Community Rules.

Hetzner subnet with additional IPs - Driving me crazy for days!

ShihabSoftShihabSoft Member
edited November 2019 in Help

Hi,

This issue has been driving me crazy for days....

I've a Hetzner Dedi, I've 6 additional IPs for various projects. I needed more IPs so I ordered a /29 subnet from them too. Totalling to 12 IPs...

Here's my /etc/network/interfaces for the Host.

auto lo
iface lo inet loopback

iface enp0s31f6 inet manual
pre-up /usr/sbin/ethtool -K enp0s31f6 tso off gso off
post-up echo 1 > /proc/sys/net/ipv4/ip_forward
post-up echo 1 > /proc/sys/net/ipv4/conf/enp0s31f6/proxy_arp

auto vmbr0
iface vmbr0 inet static
address 195.x.x.194
netmask 255.255.255.248
gateway 195.x.x.193
bridge-ports enp0s31f6
bridge-stp off
bridge-fd 0
up route add -net 195.x.x.192 netmask 255.255.255.248 gw 195.x.x.193 dev vmbr0

auto vmbr1
iface vmbr1 inet static
address 176.x.x.57 (First usable IP in the subnet)
netmask 255.255.255.248
bridge_ports none
bridge_stp off
bridge_fd 0

source /etc/network/interfaces.d/*

My subnet is 176.x.x.56/29 (.56 to .63)

I've followed Hetzner's networking configuration guide and opted the "Network Configuration Bridged" method as all my VMs are KVM.

Bridge vmbr0 is used for VMs with additional IPs and each VM has unique Mac addresses assigned as generated by Hetzner.

VMs with additional single IPs are working perfectly without any issues, that is, it can access the internet and also anyone can access the VMs from the internet.

Bridge vmbr1 is the subnet and used for VMs that comes under the subnet.

Here comes my real problem and the headache of the week.

VMs can ping the host system, also it can access other VMs with additional single IPs.

BUT...

it can't access the internet nor it can't be accessed from the internet.

When I try to ping 1.1.1.1 or any internet IP, the gateway pushes out (Destination host is unreachable)

/etc/network/interface of client VM that comes under the subnet

auto ens18
iface ens18 net static
address 176.9.193.58 (Second usable IP)
netmask 255.255.255.248
gateway 176.9.193.57 (Host subnet IP)

Some command outputs from the host

brctl show
bridge name bridge id STP enabled interfaces
fwbr100i0 8000.4ade6b0f7240 no fwln100i0
tap100i0
fwbr101i0 8000.1e5f25c45715 no fwln101i0
tap101i0
fwbr102i0 8000.c23a9abd20d6 no fwln102i0
tap102i0
fwbr103i0 8000.1e1ef8b0df77 no fwln103i0
tap103i0
fwbr104i0 8000.8e9a5105ab37 no fwln104i0
tap104i0
fwbr105i0 8000.32d707c73ec4 no fwln105i0
tap105i0
vmbr0 8000.901b0ef795e2 no enp0s31f6
fwpr100p0
fwpr101p0
fwpr102p0
fwpr103p0
fwpr104p0
fwpr105p0
vmbr1 8000.8ab69a1f1e36 no tap106i0

sysctl -p
net.ipv4.ip_forward = 1
net.ipv6.conf.all.forwarding = 1

Comments

  • Route -n and paste here

    #!/Bashblog.net | Free Wordpress Hosting | If you can't idle, what's the point?

  • @LTniger said:
    Route -n and paste here

    Host route

    Destination Gateway Genmask Flags Metric Ref Use Iface
    0.0.0.0 195.x.x.193 0.0.0.0 UG 0 0 0 vmbr0
    0.0.0.0 195.x.x.129 0.0.0.0 UG 0 0 0 vmbr0
    0.0.0.0 195.x.x.129 0.0.0.0 UG 0 0 0 enp0s31f6
    176.x.x.56 0.0.0.0 255.255.255.248 U 0 0 0 vmbr1
    195.x.x.129 0.0.0.0 255.255.255.255 UH 0 0 0 vmbr0
    195.x.x.129 0.0.0.0 255.255.255.255 UH 0 0 0 enp0s31f6
    195.x.x.192 195.x.x.193 255.255.255.248 UG 0 0 0 vmbr0
    195.x.x.192 0.0.0.0 255.255.255.248 U 0 0 0 vmbr0

    Guest route

    Destination Gateway Genmask Flags Metric Ref Use Iface
    0.0.0.0 176.x.x.57 0.0.0.0 UG 0 0 0 ens18
    176.x.x.56 0.0.0.0 255.255.255.248 U 0 0 0 ens18

  • It looks to me like you haven't given vmbr1 any access to anything external and it's isolated to the system. Normally for a routed subnet setup, I'd use something like:

    auto lo
    iface lo inet loopback
    
    iface enp0s31f6 inet static
    address 176.x.x.57 (First usable IP in the subnet)
    netmask 255.255.255.248
    gateway <gateway-address>
    
    auto vmbr1
    iface vmbr1 inet static
    address 176.x.x.57 (First usable IP in the subnet)
    netmask 255.255.255.248
    bridge_ports none
    bridge_stp off
    bridge_fd 0
    
    source /etc/network/interfaces.d/*
    

    However, I'm not sure how you would manage to have both a bridged vmbr and a routed one on the same physical interface. If you work it out, let me know. I have a server I could use that on.

  • @Undergrid said:
    It looks to me like you haven't given vmbr1 any access to anything external and it's isolated to the system. Normally for a routed subnet setup, I'd use something like:

    auto lo
    iface lo inet loopback
    
    iface enp0s31f6 inet static
    address 176.x.x.57 (First usable IP in the subnet)
    netmask 255.255.255.248
    gateway <gateway-address>
    
    auto vmbr1
    iface vmbr1 inet static
    address 176.x.x.57 (First usable IP in the subnet)
    netmask 255.255.255.248
    bridge_ports none
    bridge_stp off
    bridge_fd 0
    
    source /etc/network/interfaces.d/*
    

    However, I'm not sure how you would manage to have both a bridged vmbr and a routed one on the same physical interface. If you work it out, let me know. I have a server I could use that on.

    After a lot of fiddling and meddling. Finally I found a solution.

    First and foremost, for this bridged and routed combination to work.

    When you boot up the Proxmox Dedi

    DO NOT START ANY VM.

    Connect to SSH shell of the Proxmox node and put the vmbr0 interface down, which is the bridged interface.

    Now start any one of the VM which is configured to be used in a subnet (routed setup). Once it is booted up, try pinging an Internet address it should work.

    After then you can do ifup vmbr0, bring the bridged interface up.

    Finally now you can start all the VMs and get the network working without any issue.

    I also added a test VM to test if there's a need for reboot when configuring the network in the new VM, but no it worked out right out of the box.

  • Same issue with Virtualizor (OpenVZ7-Virtuzzo KVM) ill give this a try

  • what you do there is not a routed but still a bridged setup. and for that your guest VM simply should use the correct gateway for that subnet, which would be x.56 instead of the hostnodes IP on vmbr1 ;-)
    maybe it's even better to set it up as pointopoint on the guest.

    I use all 8 IPs from that subnet with a routed config which means you can create additional bridges, assign the normal hostnode IP but add one or more of the subnet to it via 'up route add -host ...'

    good read: https://community.hetzner.com/tutorials/install-and-configure-proxmox_ve

    Thanked by 2ITLabs Hetzner_OL

    UltraVPS.eu KVM in US/UK/NL/DE: 15% off first 6 month | Netcup VPS/rootDS - 5€ off: 36nc15279180197 (ref)

  • @Falzo said:
    what you do there is not a routed but still a bridged setup. and for that your guest VM simply should use the correct gateway for that subnet, which would be x.56 instead of the hostnodes IP on vmbr1 ;-)
    maybe it's even better to set it up as pointopoint on the guest.

    I use all 8 IPs from that subnet with a routed config which means you can create additional bridges, assign the normal hostnode IP but add one or more of the subnet to it via 'up route add -host ...'

    good read: https://community.hetzner.com/tutorials/install-and-configure-proxmox_ve

    The thing is I already tried that. But it looks like there's some conflict in the routing in the host system. The above mentioned steps in my previous reply worked, but it failed after the systems being online for a while.

    I guess I've to opt the complete routed setup for it to work.

Sign In or Register to comment.