Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Use Lobster to manage VMs across multiple providers
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Use Lobster to manage VMs across multiple providers

perennateperennate Member, Host Rep
edited January 2016 in General

Lobster is designed as an hourly billing panel, but today I made some changes that make it easier to use as just a personal panel for VM management.

Lobster is an open source web application that uses APIs provided by VM backends to allow users to provision and manage virtual machines. Supported VM backends include software like OpenStack/SolusVM/CloudStack (on SolusVM, need admin API; CloudStack is still experimental; EC2 is planned) and also various hourly billing providers that have their own API (Luna Node, Digital Ocean, Linode, and Vultr are currently supported). Lobster also exposes an API and supports using its own API as a backend, so if you happen to have an account on a Lobster platform (probably the demo, which is not useful), then you can set that up as well.

So, for personal usage, basically you can link your API keys across these providers to get a single panel where you can control your virtual machines. "Control" includes basic things like booting, shutting down, and re-imaging. But, you can also take snapshots (if supported by the backend interface) and provision new VMs from those snapshots; or fetch images and create a VM from that image.

There is a setup guide for using Lobster this way; in a nutshell, you setup a MySQL database, configure your API keys, autopopulate plans/images on all the platforms that you're using, and manually associate any existing VMs that you want to manage.

Feedback is welcome! Especially if you have ideas on how to autopopulate backend regions (so you can just configure a backend without configuring it for each backend region separately).

Comments

  • Thanks mate! :)

    Thanked by 1perennate
  • jarjar Patron Provider, Top Host, Veteran

    Every time you post I realize that I need to be using LunaNode more.

  • GM2015GM2015 Member
    edited January 2016

    He's got some mailing stuff on vpsboard that sounds interesting.

    I wish I knew as much, I'll be glad one day when I learn how to turn on my computer.

    jarland said:

    Every time you post I realize that I need to be using LunaNode more.

    image

    Thanked by 1perennate
  • jarjar Patron Provider, Top Host, Veteran
    edited January 2016

    GM2015 said: He's got some mailing stuff on vpsboard

    Lunanode actually does email hosting now too. I swear it's a one stop shop for everything you need ;)

    I gotta brag on them constantly just because I feel like they're so underrated and they have so many features it's insane.

    Thanked by 1perennate
  • GM2015GM2015 Member
    edited January 2016

    Yeah, they look nice.

    jarland said: I gotta brag on them constantly just because I feel like they're so underrated and they have so many features it's insane.

  • perennateperennate Member, Host Rep

    jarland said: I swear it's a one stop shop for everything you need ;)

    Jason keeps telling me to add domain registration through namecheap API, but I don't think anyone would use it.

    Also if anyone runs into problems setting up Lobster, let me know (reply here or PM; better to reply since someone else may have same problem), it's still a bit rough.

    Thanked by 1jar
  • hbjlee17hbjlee17 Member, Host Rep

    @perennate said:
    Also if anyone runs into problems setting up Lobster, let me know (reply here or PM; better to reply since someone else may have same problem), it's still a bit rough.

    have a problem. doesnt allow me to register domains, like on lunanode

  • perennateperennate Member, Host Rep
    edited January 2016

    hbjlee17 said: have a problem. doesnt allow me to register domains, like on lunanode

    x.x

    also for some reason I was reminded of this awesome post -- http://www.lowendtalk.com/discussion/comment/319382/#Comment_319382

    those were good times

  • @jarland said:
    Every time you post I realize that I need to be using LunaNode more.

    ^^This. Bearmon is also one of the most accurate distributed monitors I've tried.

    Good job all around @perennate

    Thanked by 1perennate
  • Any plans for Proxmox support?

    Thanked by 2perennate ItsChrisG
  • perennateperennate Member, Host Rep

    doughmanes said: Any plans for Proxmox support?

    I set up a Proxmox VM a few months ago but haven't gotten around to adding it yet. Will look into it again.

  • @perennate

    Congratulations on publishing your project, nice work!

    It is refreshing to see some such work in a good systems language such as golang, vs the usual yet another sloppy PHP nightmare..

    Minor criticism, use go fmt, in golang it's not optional, there is only one true way. I believe this will automatically take care of those >80 column sql statements you have too.. If not then https://golang.org/ref/spec#String_literals In vim-go its automatic,some have it as part of their editor tooling and some do it pre-commit hooks

    There's also parts where you could leverage channels for example instead of handling mail with an anonymous go block, if you pump into a channel you could then have centralized retry and backoff logic, you could then remove the strong coupling of ReportError to email https://github.com/LunaNode/lobster/blob/ca2306d0c37037f13cac69b9620bd22a4706aa00/email.go#L86 I'm sure you have many ideas to explore as always with freshly minted projects so just take with a pinch of salt.

    Great start, keep going!

    Thanked by 1perennate
  • perennateperennate Member, Host Rep
    edited January 2016

    nowprovision said: There's also parts where you could leverage channels for example instead of handling mail with an anonymous go block, if you pump into a channel you could then have centralized retry and backoff logic, you could then remove the strong coupling of ReportError to email

    Ah that sounds great, so maybe email will subscribe to an error logging module via channel. Thanks for the idea! There's also some improvements I want to make with general modularity, like the payment interfaces should definitely be in separate packages, and several VM things are also a mess.

    nowprovision said: Minor criticism, use go fmt, in golang it's not optional, there is only one true way. I believe this will automatically take care of those >80 column sql statements you have too.. If not then https://golang.org/ref/spec#String_literals In vim-go its automatic,some have it as part of their editor tooling and some do it pre-commit hooks

    Yeah I should start doing that :/ will see what's best to integrate gofmt in my setup.

  • Have any screenshots?

  • perennateperennate Member, Host Rep

    rmlhhd said: Have any screenshots?

    You can check the demo at http://lobster.lunanode.com

    It is still running version from several months ago, but the UI hasn't changed, mainly easier to set up plans/images and some architectural things.

  • @doughmanes said:
    Any plans for Proxmox support?

    I was just about to post asking for plans on ProxMox VE4.0 support! :)

    Thanked by 2perennate doughmanes
  • @perennate said:
    It is still running version from several months ago, but the UI hasn't changed, mainly easier to set up plans/images and some architectural things.

    Maybe I'm blind, but I cant find the demo?

  • perennateperennate Member, Host Rep
    edited January 2016

    ItsChrisG said: Maybe I'm blind, but I cant find the demo?

    http://lobster.lunanode.com/

    You can create an account and then login with it. "Virtual VM" = fake vm for demo, lol.

    Thanked by 1vimalware
  • perennateperennate Member, Host Rep

    ItsChrisG said: I was just about to post asking for plans on ProxMox VE4.0 support! :)

    Well, I got basic Proxmox functionality working (see this commit), but I'm a bit confused about how to provision a Proxmox VM from a template (or at least snapshot), either from the API or even from the control panel. Any tips on that?

    Only does for qemu calls currently but eventually there should be virtualization type option.

  • I usually go the Create CT route via the web UI.

  • perennateperennate Member, Host Rep
    edited January 2016

    doughmanes said: I usually go the Create CT route via the web UI.

    Yeah, I think I see how to use template with CT from API. But I can't find anything similar with VM. Yet it seems like you can create templates from VMs.

  • I've used the Backup tab and created 'a template' that way

    Thanked by 1perennate
  • @perennate said:
    Yeah, I think I see how to use template with CT from API. But I can't find anything similar with VM. Yet it seems like you can create templates from VMs.

    The template feature seems to be for ovz only? Not sure why there's a template method under qemu.. Have you tried cloning a disk and creating a new VM from a cloned disk? That could work sorta like templating.

    Thanked by 1perennate
  • perennateperennate Member, Host Rep

    @doughmanes @Jonchun alright thanks, I'll look at those two things.

    Thanked by 2doughmanes Jonchun
Sign In or Register to comment.