Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Slabbing 4 Dummies (really semi-experts, but who's counting?)
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Slabbing 4 Dummies (really semi-experts, but who's counting?)

Please note: I did not write this tutorial to be an ethical discussion on slabbing, merely as a how to on slabbing. Similar comparison is the ethics of hunting versus a firearm class.

I also wrote this guide so that there are some expertise barriers, so that dum dums can't exactly just start a business and load up 600-700 VPS's on a single hardware node (lets call it the mistake with slabbing).

First off, you want to install your host node. For this, you'll need a Ubuntu 13.10+ install with 2gb on /boot, 8gb on /, and 2gb on swap. Make the rest of the drive an unformatted partition. You may want to argue about that 8gb on root. Don't. It's a bitch to upgrade if you skimp.

Once you have your base OS installed, make sure to log in as root. This can be pulled off by doing sudo -i from the user you cretaed during the installation process. From this point, use apt-get to install xen-hypervisor-amd64, openvswitch-datapath-dkms, and openvswitch-switch.

After this step, make SURE you have remote access to the physical node. Sometimes network cards don't play nice, and it's a royal pain in the ass to correct without access to the node (read: reinstall and try again, many DC's charge $25 a reinstall). If you're in ColoCrossing, you can use ipmicfg to shimmy yourself an IPMI user and use a VPN to a VPS in CC-NET to connect to the IP it gives you (I am unashamed to admit I've done this on MULTIPLE occasions, the VPN part). Beyond that, I don't know other DC's procedures.

Now that you've verified that you have remote access, edit your network config so that you have a standard bridge in Ubuntu (you don't have the bridge_ports line, OpenVSwitch handles that bit for you).

After that, run the following command (replacing p4p1 with your external network interface, along with 10.0.0.1 with your actual network gateway):

ifconfig p4p1 down && ovs-vsctl add-br br0 && ifconfig p4p1 0.0.0.0 up && ovs-vsctl add-port br0 p4p1 && ifup br0 && ip route add default 10.0.0.1 && ping -c 3 8.8.8.8

(yes, the ping -c 3 8.8.8.8 is necessary, I've had OpenVSwitch fail on me a few times without it)

Once you've got that setup, you need to setup xen.

mv /etc/grub.d/20_linux-xen /etc/grub.d/09_linux-xen

update-grub

sed 's/DEFAULT=.*/DEFAULT=xl/' -i /etc/default/xen

reboot

Sometimes Ubuntu will interpret that reboot command as a shutdown command. Boot it externally if this happens.

Now that you've rebooted successfully, you're on what I like to call step 2.

What you need to do is first set your default network script in /etc/xen/xl.conf to vif-openvswitch. After this, you need to pull the netboot images from CentOS. They can be found in $CENTOSMIRROR/6.5/os/x86_64/images/pxeboot/ (you need both vmlinuz and initrd.img).

Now, you need to install lvm2 via apt-get. After you've done that, create a volume group on your target volume, and create a logical volume for each slab you intend to load.

The basic installation xen configuration is like follows.

vmlinuz = "/root/vmlinuz"

ramdisk = "/root/initrd.img"

disk = [ '/dev/mapper/VolGroup-LogVol,raw,xvda,rw' ]

vif = [ 'mac=00:16:3E:01:23:45,bridge=br0' ]

name = "slab1"

memory = 10000

Run thru the centos installer, then reconfigure xen to full boot:

bootloader = "/usr/lib/xen-4.3/bin/pygrub"

disk = [ '/dev/mapper/VolGroup-LogVol,raw,xvda,rw' ]

vif = [ 'mac=00:16:3E:01:23:45,bridge=br0' ]

name = "slab1"

memory = 10000

Install OpenVZ like you normally would, then voila.

A couple things you may want to keep in mind when using Xen slabbing:

  • Set your Dom0's maximum memory.

  • Set your CPU's on your DomU's with vcpus = and cpu = ['1','2','3','4','5'] (remember, count starts at 0, reserve one on each physical CPU for Dom0's networking et al).

  • elevator=noop is a good kernel option to make disk IO less fucky.

  • Remove most your kernel command line, leaving just the ones you need.

Comments

  • Mmmm, yummy slabbing. :P

  • MannDudeMannDude Host Rep, Veteran
    edited March 2014

    Good write up. Have you done this in production environments for companies that have hired you? I don't think slabbing is inherently wrong if done properly or for a good reason, so it's nice to the write-up.

    EDIT: Also your resume is missing some employers. ;)

  • MannDude said: Have you done this in production environments for companies that have hired you?

    All too much, it's a tad boring doing the same thing over and over.

    Thanked by 1MannDude
  • Here's a tip for you young one. Don't include employers in your resume if you didn't necessarily leave on good terms with them, as future/potential employers WILL contact the past employers to verify the quality of your work.

    After all, you did initiate a public debate while being employed with your latest previous employer (Green Value Host) on WHT. No wonder GVH-Jon felt the need to let you go :P

  • said: Similar comparison is the ethics of hunting versus a firearm class.

    Like the analogy.

  • RalliasRallias Member
    edited March 2014

    What... so nobody has any questions related to the thread?

    macecis said: Here's a tip for you young one. Don't include employers in your resume if you didn't necessarily leave on good terms with them, as future/potential employers WILL contact the past employers to verify the quality of your work.

    Think logically. I posted it publicly on LET, where I was publicly a contractor used by a company. I was "let go" as you put it because the contract ran out.

  • Besides what Xen reports in /proc/cpuinfo as opposed to what KVM will report in /proc/cpuinfo, are there any other advantages to using Xen instead of KVM for slabbing your OpenVZ containers?

  • Rallias said: What... so nobody has any questions related to the thread?

    Well, I'm curios what the advantage of using Xen PV for slabbing is? I've read that VMware and KVM enable provider to do some sort of memory dedup. Does Xen do that as well?

  • VirtovoVirtovo Member
    edited March 2014

    @mpkossen said:
    Well, I'm curios what the advantage of using Xen PV for slabbing is? I've read that VMware and KVM enable provider to do some sort of memory dedup. Does Xen do that as well?

    Xen has Memory CoW which is similar to VMWare TPS, although in a slabbed environment you're unlikely to see much benefit of KSM/COW/TPS. Merging only becomes a benefit when you are running a large number of similar guests.

    @imchandave said:
    Besides what Xen reports in /proc/cpuinfo as opposed to what KVM will report in /proc/cpuinfo, are there any other advantages to using Xen instead of KVM for slabbing your OpenVZ containers?

    KVM can pass correct CPU info across also.

  • RalliasRallias Member
    edited March 2014

    mpkossen said: Well, I'm curios what the advantage of using Xen PV for slabbing is?

    It allows for "slabbing" without having a full virtualized environment. This means reduced overhead in terms of network and disk read's. It does come at the low cost of having a slightly higher system call overhead (Somewhere in the area of 7 additional instructions, 2 additional context switches). It also allows you to rate limit bandwidth outbound so that if you have an outbound DDoS, you don't get completely removed from being able to access the server.

    imchandave said: Besides what Xen reports in /proc/cpuinfo as opposed to what KVM will report in /proc/cpuinfo, are there any other advantages to using Xen instead of KVM for slabbing your OpenVZ containers?

    Well, actually being able to use the additional instructions is nice.

    Thanked by 2mpkossen Mark_R
  • perennateperennate Member, Host Rep
    edited March 2014

    What is the point of slabbing though? In the end isn't there going to be more virtualization overhead for every resource (CPU, memory, disk)?

    Edit: nevermind, I see from https://vpsboard.com/topic/2973-vps-slabbing-who-does-it-and-admits-to-it/ (although no really good answers -- like if you're running Windows you can put OpenVZ in KVM or something, or @Virtovo says "build your base infrastructure with a single virtualisation type", KuJoe points out mobility)

  • perennate said: What is the point of slabbing though?

    It allows you to segregate your resources so that one abuser cannot gimp the whole node.

    Thanked by 1perennate
  • BruceBruce Member

    can you buy xen vps from someone, then slab it with ovz and then sell those vps? presumably there's a way to stop a vps being used for slabbing

  • Bruce said: can you buy xen vps from someone, then slab it with ovz and then sell those vps? presumably there's a way to stop a vps being used for slabbing

    You can do that, but it's really fucking stupid to comingle your virtualization types. The only way to stop slabbing is to force a kernel on a VPS (sorta like Digital Ocean does).

  • DerekDerek Member

    So, I can oversell heavily with this? like 300gb?

    Thanked by 2Pwner perennate
  • perennateperennate Member, Host Rep
    edited March 2014

    Derek said: So, I can oversell heavily with this? like 300gb?

    400gb, maybe 500gb if you optimize

  • Ugh.

  • DerekDerek Member

    @perennate said:
    400gb, maybe 500gb if you optimize

    So, what do you guys do to optimize the node? Like limit each VPS to like 10% CPU or something?

  • Derek said: So, what do you guys do to optimize the node? Like limit each VPS to like 10% CPU or something?

    No need. They'll only affect a couple hundred customers at once.

    Thanked by 1MannDude
  • perennateperennate Member, Host Rep
    edited March 2014

    Derek said: So, what do you guys do to optimize the node? Like limit each VPS to like 10% CPU or something?

    No, we limit each VM to 1gb CPU so we can put 500gb CPU on each node.

    Edit: to optimize we put 5 GB on top of 4 GB so you can 9 GB, then you can put the 9 GB inside 14 GB to get 300 GB.

  • mikhomikho Member, Host Rep

    @perennate said:

    500gb CPU?

  • perennateperennate Member, Host Rep
    edited March 2014

    MikHo said: 500gb CPU?

    Maybe 600 gb

    Edit: TBH I have no idea what "So, I can oversell heavily with this? like 300gb?" means

Sign In or Register to comment.