Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Problem with KVM setup (no internet access on guests)
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Problem with KVM setup (no internet access on guests)

pechspilzpechspilz Member
edited November 2012 in Help

So I have this lovely low-end dedicated server on its own subnet (the server is part of same subnet) and I want to use it for KVM virtualization. I'm using a dead simple bridged network setup (like I did many times before on other servers) so my guests are able to connect to the internet without any routing configuration in the host.

Now here's the problem: my guest doesn't get any inbound traffic from the internet.

Some more facts:

  • All netfilter chains are set to ACCEPT per default, no rules are present (guest and host)
  • Even though I don't think it's needed in bridged setups, ip forwarding is allowed on the KVM host
  • The KVM host has full internet connectivity
  • The KVM host and the guest are able to ping each other
  • The guest uses the switch's gateway address as its gateway
  • The guest gets ARP broadcasts from the switch's gateway address
  • The guest puts the switch's MAC address into its ARP table (and tells its virtual MAC address to the switch)
  • The guest doesn't get an ICMP echo back from the switch's gateway address even though the KVM host sees the outgoing request (but neither see a reply from the switch for the guest's IP)
  • A remote host under my control gets the guest's ICMP request as well but the guest never gets the reply, nor does the KVM host see the reply
  • Rewriting the guests MAC address on the host replacing it with the hosts eth0 MAC address using fancy ebtables rules leads to full internet connectivity on the guest
  • I initially had a smaller subnet from the same provider. Using the same bridged libvirt setup, everything was working nicely. It stopped working after they assigned the new subnet.

I asked my provider if there is a MAC filter active for my switch port and they denied this 3 (!) times. However, the guest gets full internet connectivity once the provider manually adds the guest's MAC address to the switch's static ARP table.

At this time I'm a little bit out of ideas. Does anyone have an idea what could possibly cause this problem?

Comments

  • Does this happen with other providers? tried changing distros?

  • What distro are you using anyways for the main OS?

  • MaouniqueMaounique Host Rep, Veteran

    I fear this is a router problem. For some reason drops packets destined to your guests.
    It might not be under your control. You can test this a bit risky by changing MAC on the main interface (host) and if you cannot access internet afterwards, I think the problem is clear.

  • This is a brilliant and simple idea. Makes debugging the problem a whole lot easier, thanks! I can always gain access to the server since it's a blade with ILO.

  • rskrsk Member, Patron Provider

    @pechspilz

    @getkvm_ash can help i guess? He's the KVM expert :P

    regards

  • Could be the switch does not have a pre-comfigured filter per se but once it learns/caches a Mac for a given IP it refuses to accept packets with the same MAC going to a different IP. In the guest, try changing the MAC 3 times in a row...doing your ping test each time while keeping the same guest IP... If you don't get a reply then after a MAC change then wait....and retry in an hour and see if the cache entry has timed out or not...

  • It just occurred to me that they might have disabled MAC autolearning on my server's switch port in order to add virtual routing interfaces. That must be it.

  • This certainly sounds like something to do with MAC addresses, i have faced this problem before (By the sounds of it). Some networks just straight up wont allow you to generate and use custom MAC addresses.

    What DC/company is this with?

    I feel for you, i know how frustrating it is when your trying to fix something that cant be fixed on your end, happened to me yesterday with some IPv4 allocation that just wouldn't route. Then i get a message later on that day "Sorry it was a misconfiguration on our end".

    Good luck!

  • pechspilzpechspilz Member
    edited November 2012

    It's Datashack (I discovered them on LET btw.).

    Since my KVM guests were able to connect just fine on my initial subnet, I suppose there must be some sort of SNAFU on their end. The case has been unresolved for quite a while now (I'm talking about weeks). I spent countless hours debugging the problem but on the positive side, it made me a badass wannabe networking expert. Providers, FEAR ME :-)

  • @pechspilz said: - I initially had a smaller subnet from the same provider. Using the same bridged libvirt setup, everything was working nicely. It stopped working after they assigned the new subnet.

    @pechspilz said: Since my KVM guests were able to connect just fine on my initial subnet, I suppose there must be some sort of SNAFU on their end.

    Did you assign a static route in your Guest OS? Specify gateway address of new subnet in your static route. Gateway address in ifcfg-eth0 file won't work if new subnet has a different gateway address.

    You can refer CentOS Static Route for details.

  • @biplap: There's just one network left. Everything is on the same network now (server + guests).

    But even with a default OS installation (I'm using Ubuntu Server 12.04 LTS) and a modified eth0 MAC, I'm not getting any inbound traffic (except ARP) from their switch on the server. Outbound traffic passes through without interruption, no matter what MAC I'm using.

    The switch only operates normally if I'm using the NIC's eth0 MAC.

    @Datashack customers: Since it's been another 5 days without any update/progress from Datashack I'm wondering: is there any way to escalate a ticket? Or is there no one to escalate to? What are your experiences with their (network) support?

  • cybjecybje Member
    edited November 2012

    I have seen similar problems with another server brand, where the management network interfaced (in your case ILO) was shared with the normal network interface.

    Incoming traffic to the management MAC was going to the management card, incoming traffic to the eth0 MAC was going to eth0, but any other incoming traffic wasn't going through, eventhough a bridge was configured in Linux. Sometimes after a few network resets it worked fine though, that was the strange part.

    What finally solved it, was putting the management on it's own interface, instead of sharing it with the eth0 uplink.

  • DataShack had been unable to solve the problem. However they offered a full refund and I took it. Their network support is beyond crap but general support, sales...very nice people. I loved their HP Proliant equipment, I haven't seen anything even remotely comparable in that price range. Kinda sad they had to "let me go" ;-/

Sign In or Register to comment.