Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Help Proxmox bridged container no internet
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Help Proxmox bridged container no internet

kmmmkmmm Member

Hi.

What I have:

  • A VPS with 5 IPv4 xxx.xxx.20.93-xxx.xxx.20.97, same subnet and 1 /64 IPv6.
  • Proxmox latest, 6.0 VE: Debian 10. Container used Ubuntu 16 template.
  • I can make IPv4 and IPv6 work on Proxmox host.
  • Provider doesn't restrict IP's to virtual MAC ID's, so bridged mode is recommended.
  • No Nest Virtualization so only linux container.

What I want:

  • 1 IPv4 for Proxmox host.
  • Assign 4 IPv4 and 4 IPv6 for 4 Containers.
  • Fancy stuff: private LAN network for containers like 10.14.43.xxx so containers can talk to each other if needed.

What is my situation now:

  • IPv4 and v6 work on Proxmox host.
  • Try routed mode, assign IP for container: Can ping host from container but cannot ping internet, like 8.8.8.8. There is packet arrived vmbr0, but no tx packet from vmbr.
  • Try bridged mode, assign IP for container: Can not ping anything.

What I have tried:

  • blog.no-panic.at/2016/08/09/proxmox-on-debian-at-hetzner-with-multiple-ip-addresses/
  • sysorchestra.com/proxmox-5-on-hetzner-root-server-with-ipv4/
  • And countless other. Both route and bridge.

Since my provider told me bridged mode is recommended I will pursuit this direction. Here is my config:

Proxmox /etc/network/interfaces:

auto lo
iface lo inet loopback
iface ens3 inet manual
auto vmbr0
iface vmbr0 inet static
        address  xxx.xxx.20.93
        netmask  255.255.254.0
        gateway  xxx.xxx.20.1
        bridge-ports ens3
        bridge-stp off
        bridge-fd 0

My config for a container /etc/network/interfaces: (nothing in /etc/network/interfaces.d)

auto lo
iface lo inet loopback
iface lo inet6 loopback
# interfaces(5) file used by ifup(8) and ifdown(8)
# Include files from /etc/network/interfaces.d:
source-directory /etc/network/interfaces.d
auto eth0
iface eth0 inet static
        address xxx.xxx.20.94
        netmask 255.255.254.0
        gateway xxx.xxx.20.1

The whole weekend was spent, no hope ;(. I am really depressed now.

Comments

  • LeviLevi Member

    Probably there is no routes for vmbr0.

  • kmmmkmmm Member

    @LTniger said:
    Probably there is no routes for vmbr0.

    Can you explain? The config above is for bridged mode so I think it has nothing to do with route in /etc/network/interfaces file. Or you mentioned ip route?

  • AlwaysSkintAlwaysSkint Member
    edited July 2019

    https://www.kiloroot.com/proxmox-kimsufi-ovh-soyoustart-ipv6-host-multiple-containers-and-virtual-machines-on-a-single-kimsufi-server-using-ipv6-and-proxmox/

    I use CSF so things are a bit different but I do setup manual routing.

    Here's my host setup, from a live environment with IPs setup like yours..

    auto lo
    iface lo inet loopback

    iface enp1s0 inet manual

    allow-vmbr1 int1
    iface int1 inet static
    address 192.168.xxx.2
    netmask 255.255.255.0
    ovs_type OVSIntPort
    ovs_bridge vmbr1

    auto vmbr0
    iface vmbr0 inet static
    address 66.xxx.xxx.18
    netmask 255.255.255.248
    gateway 66.xxx.xxx.17
    bridge-ports enp1s0
    bridge-stp off
    bridge-fd 0

    iface vmbr0 inet6 static
    address 26xx:xxx0:12xx:1::2
    netmask 64
    gateway 26xx:xxx0:12xx:1::1

    auto vmbr1
    iface vmbr1 inet static
    address 192.168.xxx.1
    netmask 255.255.255.0
    ovs_type OVSBridge
    ovs_ports int1

    ip r

    default via 66.xxx.xxx.17 dev vmbr0 onlink
    66.xxx.xxx.16/29 dev vmbr0 proto kernel scope link src 66.xxx.xxx.18
    192.168.xxx.0/24 dev int1 proto kernel scope link src 192.168.xxx.2
    192.168.xxx.0/24 dev vmbr1 proto kernel scope link src 192.168.xxx.1

  • I'm assuming you already set up forwarding, however, looks like you're missing an SNAT.

  • kmmmkmmm Member

    @AlwaysSkint said:
    https://www.kiloroot.com/proxmox-kimsufi-ovh-soyoustart-ipv6-host-multiple-containers-and-virtual-machines-on-a-single-kimsufi-server-using-ipv6-and-proxmox/

    I use CSF so things are a bit different but I do setup manual routing.

    Here's my host setup, from a live environment with IPs setup like yours..

    auto lo
    iface lo inet loopback

    auto vmbr1
    iface vmbr1 inet static
    address 192.168.xxx.1
    netmask 255.255.255.0
    ovs_type OVSBridge
    ovs_ports int1

    ip r

    default via 66.xxx.xxx.17 dev vmbr0 onlink
    66.xxx.xxx.16/29 dev vmbr0 proto kernel scope link src 66.xxx.xxx.18
    192.168.xxx.0/24 dev int1 proto kernel scope link src 192.168.xxx.2
    192.168.xxx.0/24 dev vmbr1 proto kernel scope link src 192.168.xxx.1

    I assume you have a typo in your post?

    @roliverio said:
    I'm assuming you already set up forwarding, however, looks like you're missing an SNAT.

    I understand that SNAT is necessary only for routed mode, am I correct?

  • AlwaysSkintAlwaysSkint Member
    edited July 2019

    @kmmm said:
    I assume you have a typo in your post?

    Usually, but where do you assume the typo?
    I originally just copied some snippets from my notes but I changed them for the interfaces and routing table which are obscured live ones.

  • roliverioroliverio Member
    edited July 2019

    Indeed, however, i'm seeing that your container example config is pointing to the provider's gateway instead of the proxmox public host ip, if you want to set up this as per your config, you'll need an open bridge which interfaces to all container interfaces, so it works as a proper bridge.

    In my configs i avoid trying to set up public interfaces for anything behind the host, and instead use private IP's directly assigned with a minimal and restricted DHCP server, and then setup DNATs/SNATs correspondingly for any additional public IPs, this reduces management complexity as the public IP setup is done on one place only, and i can control firewall rules on a single place, and i benefit from the fact that i know that there is only one place where the network failures can be traced.

    The method i use is to set up a bridge only for the local lan, and leave virtual interfaces for any additional public IPs i want to use properly restricted with firewall / forwarding rules. (this has minimal overhead, and, doesn't cause total network connectivity loss on interface de(re)-activation if i need to flush rules or reload network.)

  • I have the VM (KVM in this case) gateway the same as the host gateway.

  • kmmmkmmm Member

    @AlwaysSkint said:

    @kmmm said:
    I assume you have a typo in your post?

    Usually, but where do you assume the typo?
    I originally just copied some snippets from my notes but I changed them for the interfaces and routing table which are obscured live ones.

    That is what I see:
    ip r

    @roliverio said:
    Indeed, however, i'm seeing that your container example config is pointing to the provider's gateway instead of the proxmox public host ip, if you want to set up this as per your config, you'll need an open bridge which interfaces to all container interfaces, so it works as a proper bridge.

    In my configs i avoid trying to set up public interfaces for anything behind the host, and instead use private IP's directly assigned with a minimal and restricted DHCP server, and then setup DNATs/SNATs correspondingly for any additional public IPs, this reduces management complexity as the public IP setup is done on one place only, and i can control firewall rules on a single place, and i benefit from the fact that i know that there is only one place where the network failures can be traced.

    The method i use is to set up a bridge only for the local lan, and leave virtual interfaces for any additional public IPs i want to use properly restricted with firewall / forwarding rules. (this has minimal overhead, and, doesn't cause total network connectivity loss on interface de(re)-activation if i need to flush rules or reload network.)

    But then do you need to deal with the problem such that app in container never knows the true IP where the packet come from (i.e in Wordpress's log all request come from route IP like 10.14.43.xxx).

    Update: It suddenly works. :)

  • Sounds of a Leonard Cohen classic ring out from the galleries. :-p

  • kmmmkmmm Member

    No shit man. The same procedure, same host. It suddenly works after a few hour.

  • Oh, if i'm using specific services (like a web server) i'll use mod_rpaf for apache, or real_ip with nginx.

    If i don't care about that, i'll simply DNAT/SNAT the whole IP address.

    Thanked by 1kmmm
Sign In or Register to comment.