proxmox on hetzner

smicroz

Hi,

I'm trying configure proxmox on hetzner, but for some reason I can't setup the network for vps.

Based in the guide I setup the interfaces:

source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback
iface lo inet6 loopback

auto enp0s31f6

auto vmbr0
iface vmbr0 inet static
      address   main ip
       netmask   main ip netmask
       pointopoint   main ip gateway
       gateway   main ip gateway
       bridge_ports enp0s31f6
       bridge_stp off
       bridge_fd 1
       bridge_hello 2
       bridge_maxage 12

auto vmbr1
iface vmbr1 inet static
       address  ip of subnet /29
       netmask   netmask of ip of subnet /29
       bridge_ports none
       bridge_stp off
       bridge_fd 0

Anyone trying setup this before? any suggestion?

Thanks!

graphic

https://pve.proxmox.com/wiki/Network_Model

After this you can use the virtual mac addresses while vm config.

Use default config (bridged).

Falzo

so you have an additional subnet and want to use all of the IPs? that's a 'routed' setup then.

please use < pre > tags to put in those text as it is really hard to read otherwise.

over all it should look like:

auto lo 
iface lo inet loopback 
iface lo inet6 loopback

iface enp0s31f6 inet manual

auto vmbr0 
iface vmbr0 inet static 
    address main-ip 
    netmask 255.255.255.255 
    pointopoint main-ip-gateway 
    gateway main-ip-gateway 
    bridge_ports enp0s31f6 
    bridge_stp off 
    bridge_fd 1 
    bridge_hello 2 
    bridge_maxage 12

auto vmbr1
iface vmbr1 inet static 
    address main-ip
    netmask 255.255.255.255
    bridge_ports none 
    bridge_stp off 
    bridge_fd 0

    up route add -host subnet-ip1 dev vmbr1
    up route add -host subnet-ip2 dev vmbr1
    up route add -host subnet-ip3 dev vmbr1
    up route add -host subnet-ip4 dev vmbr1
    up route add -host subnet-ip5 dev vmbr1
    up route add -host subnet-ip6 dev vmbr1
    up route add -host subnet-ip7 dev vmbr1
    up route add -host subnet-ip8 dev vmbr1

on creation of your VM use the bridge vmbr1 of course. be adviced that without mac-filtering hijacking of those IPs inside of the guests will be possible. if you want to avoid that you could set up multiple bridges and always only add one of those IPs per bridge.

Falzo

@graphic said:
https://pve.proxmox.com/wiki/Network_Model

After this you can use the virtual mac addresses while vm config.

Use default config (bridged).

looks like he has an additional subnet, there are no vmacs for subnets with hetzner.

graphic

@Falzo said:

@graphic said:
https://pve.proxmox.com/wiki/Network_Model

After this you can use the virtual mac addresses while vm config.

Use default config (bridged).

looks like he has an additional subnet, there are no vmacs for subnets with hetzner.

Ok, that's another config for sure.

Mr_Tom

I had a similar thing.

Ended up with vmbr0 as main ip and vmbr1 with first ip in additional subnet.

Still didn't work... until I did

sysctl -w net.ipv4.ip_forward=1

and it all magically started working lol.

Took me many attempts at getting the network setup (including making it unreachable) - but that's what it's for, for me anyway, learning to setup VMs.

Hetzner have a good wiki page on it, that and the proxmox page on setting up routed network helped me out.

smicroz

I > @Mr_Tom said:

I had a similar thing.

Ended up with vmbr0 as main ip and vmbr1 with first ip in additional subnet.

Still didn't work... until I did

sysctl -w net.ipv4.ip_forward=1

and it all magically started working lol.

Took me many attempts at getting the network setup (including making it unreachable) - but that's what it's for, for me anyway, learning to setup VMs.

Hetzner have a good wiki page on it, that and the proxmox page on setting up routed network helped me out.

Do you have a subnet? I have a block /29

But on guest vps I haven't network. I'm setup this on centos

ip: subnet/29 ip usable
gateway: main ip
netmask: 255.255.255.248


ping x.x.x.x

destination host unreachable

Mr_Tom

Yeah I've got an additional subnet. I'm not bridging vmbr0 to enp6s0 though.

I've got the following in /etc/network/interfaces

auto enp6s0
iface enp6s0 inet static
        address  a.a.a.a
        netmask  b.b.b.b
        gateway  c.c.c.c
        post-up echo 1 > /proc/sys/net/ipv4/conf/enp6s0/proxy_arp

auto vmbr0
iface vmbr0 inet static
        address a.a.a.a
        netmask b.b.b.b
        bridge_ports none
        bridge_stp off
        bridge_fd 0
        up route add y.y.y.y/32 dev vmbr0
        up route add z.z.z.z/32 dev vmbr0
        etc

auto vmbr1
iface vmbr1 inet static
        address  x.x.x.x
        netmask w.w.w.w
        bridge_ports none
        bridge_stp off
        bridge_fd 0

y.y.y.y and z.z.z.z are subnet IPs which will be used on VMs

Note: this works but I don't think it's correct. You need the sysctl command for it to work.

I'm going to retry the network config as above but vmbr0 as first_subnet_ip and no vmbr1. I had it this way before and it didn't work but the above didn't work until I enabled forwarding (doh).

I'm sure there's someone more knowledgable who might know the correct way lol.

I used proxmox wiki and hetzners wiki page on proxmox to get to here.

svmo

@Mr_Tom said:
Yeah I've got an additional subnet. I'm not bridging vmbr0 to enp6s0 though.

I've got the following in /etc/network/interfaces

> auto enp6s0
> iface enp6s0 inet static
>         address  a.a.a.a
>         netmask  b.b.b.b
>         gateway  c.c.c.c
>         post-up echo 1 > /proc/sys/net/ipv4/conf/enp6s0/proxy_arp
> 
> auto vmbr0
> iface vmbr0 inet static
>         address a.a.a.a
>         netmask b.b.b.b
>         bridge_ports none
>         bridge_stp off
>         bridge_fd 0
>         up route add y.y.y.y/32 dev vmbr0
>         up route add z.z.z.z/32 dev vmbr0
>         etc
> 
> auto vmbr1
> iface vmbr1 inet static
>         address  x.x.x.x
>         netmask w.w.w.w
>         bridge_ports none
>         bridge_stp off
>         bridge_fd 0
> 
> 

y.y.y.y and z.z.z.z are subnet IPs which will be used on VMs

Note: this works but I don't think it's correct. You need the sysctl command for it to work.

I'm going to retry the network config as above but vmbr0 as first_subnet_ip and no vmbr1. I had it this way before and it didn't work but the above didn't work until I enabled forwarding (doh).

I'm sure there's someone more knowledgable who might know the correct way lol.

I used proxmox wiki and hetzners wiki page on proxmox to get to here.

• forgetting to turn on ip forwarding is a classic ...
• configure your /xx subnet on the vmbridge with first IP in subnet assigned to the bridge

• VM's should have similar config (/xx subnet on interface ) - but gateway MUST be the ip assigned to the bridge - NOT the IP of your main interface.

• from vm check ping bridge address - should always work

• check ping box main address a.a.a.a - ( should not work if forwarding is off )
• notice iptables FORWARDING chain applies to traffic to and from your VM's

AFAIK your subnet is routed into your box so there would be no ARP requests for those IP's they are simply sent to your interface

• tcpdump -n -i enp6s0 arp or icmp
• try ping one of your subnet ip's from outside - you will se pings arriving but no prior arp

Falzo

that has been covered a lot here. simply use a proper pointopoint setup and make sure to use netmask 255.255.255.255

Mr_Tom

Just another note, I believe proxmox adds rpcbind - make sure you disable/firewall it or you may get an abuse notification from Hetzner (via BSI)

smicroz

@Mr_Tom said:
Just another note, I believe proxmox adds rpcbind - make sure you disable/firewall it or you may get an abuse notification from Hetzner (via BSI)

Thanks! I receive today this notification.