Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


BlueVM Illinois server hacked, data lost
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

BlueVM Illinois server hacked, data lost

lbftlbft Member
edited May 2012 in Providers

I searched but I can't see any other thread discussing this. I received this email from BlueVM (about four hours ago, according to the timestamp).

Hello,

>

Yesterday around 4:00 PM MST one of the servers in our Illinois datacenter was compromised. This compromise lead to the loss of data on that server along with all backups and data associated with it. That data was unrecoverable and thus roughly 30 of our clients VPS are currently offline. We have no reason to believe that the data on that server was stolen by a third party. We have determined the cause of the incident to be due to an exploit carried out against our VPN services on that node. We have shut down all of our VPN services on our other nodes to prevent this incident from happening again and have changed all of our root passwords, control panel passwords and administrator logins. Clients who run a VPN on their VPS will not be affected by our shutdown of those services.

>

We are offering those clients who were/are affected by this compromise. As such we are offering our clients affected by this data loss their choice of:

>

  1. Three (3) months of services on a different node of their choice with the plan they currently have now.
    or
  2. A full refund for this month's services.

>

To request either of these two options simply open a new ticket with us and we will help you come up with a solution to get you back online as quickly and easily as possible.

>

In addition we are taking steps to prevent our clients from experiencing data loss again, thus starting immediately we will be offering 10 GB of free backup space with FTP, control panel and mysql access to all of our clients upon request. We feel that this change will prevent our clients from experiencing a loss of data due to hacking in the future and we are committed to helping to resolve any issues caused by this outage to the best of our ability.

>

We would like to thank each and every one of you for choosing BlueVM and we would like to assure you we are doing everything in our power to prevent an incident like this from occuring again. If you have any suggestions, comments, questions or general feedback please feel free to open a ticket with us.

>

Best Regards,
BlueVM Staff
https://bluevm.com

I'm luckily not on the affected node, and I'm sure it's little comfort to those who've lost their data, but it's really nice to see a company be upfront about what happened and the steps they've taken to ensure it doesn't happen again. Unfortunately part of the reality of the internet is that compromises do happen sometimes :( The compensation for the affected customers is a nice touch.

«1

Comments

  • vps6netvps6net Member
    edited May 2012

    Seems like a strange incident, but it's good to see that BlueVM handled it in a reasonably professional manner. I especially like the offer of "10 GB of free backup space with FTP, control panel and mysql access to all of [their] clients upon request." Best of luck to them!

  • subigosubigo Member
    edited May 2012

    Why were the backups stored on the same server and why was there a VPN server running on the core node?

    Thanked by 1NanoG6
  • ZeroZero Member

    @subigo said: Why were the backups stored on the same server and why was there a VPN server running on the core node?

    I was thinking the same.

  • joepie91joepie91 Member, Patron Provider

    @subigo said: Why were the backups stored on the same server and why was there a VPN server running on the core node?

    Core node?

  • subigosubigo Member

    @joepie91 said: Core node?

    The core of the node. Not a container. Main root.

  • NightNight Member

    Were they running OpenVPN as root or an unprivileged user?

  • About the VPN offer, seems like someone already remind them about the hacking possibilities. But I don't remember where it ( WHT or LET.)

  • jcalebjcaleb Member

    i have a vps with them, not sure if compromised. luckily i only use it for testing.

  • @jcaleb said: i have a vps with them, not sure if compromised. luckily i only use it for testing.

    I have one too, the $1,5. But I don't know what to use that vps for. Since many os templates running update/upgrade is not working :(

  • KairusKairus Member

    The BlueVM guy doesn't seem too active on LET anymore. He used to post a lot.

  • JeffreyJeffrey Member

    @Kairus I think he has gotten enough clients to satisfy himself. ;)

  • AldryicAldryic Member

    Just a guess, but at the moment I'd wager that he's too busy doing damage control to pop in for a spell.

  • SpiritSpirit Member
    edited May 2012

    @subigo said: Why were the backups stored on the same server and why was there a VPN server running on the core node?

    Backups? Of $1.50/month VPSs? :) It's most likely just standard notification like "We have no reason to believe that the data on that server was stolen by a third party" part. If hack really happened there's of course always big reason to believe that atleast some datas are also stolen and saying anything else to calm down clients seems sort of unresponsible.

    Anyway, I wish to BlueVM host all the best and hope that this will be fixed soon without too big damage for clients and company.

    Thanked by 1marrco
  • BlueVMBlueVM Member
    edited May 2012

    @Spirit said: If hack really happened there's of course always big reason to believe that atleast some datas are also stolen and saying anything else to calm down clients seems sort of unresponsible.

    We had network monitoring to tell us that... No big spikes in network usage and considering there was 200 GB of data on the node between all the users there would have been some indication of a spike.

    As @Aldryic said I've been preforming damage control all day... I'm at the 36 hour mark without sleep.

    The VPN clients were under an unprivileged user.

  • KairusKairus Member

    @BlueVM said: I'm at the 36 hour mark without sleep.

    Damn, best of luck.

  • SpencerSpencer Member

    @Kairus said: I'm at the 36 hour mark without sleep.

    If you get really tired I can sing to you ALL night if you want to help keep you up.

  • jcalebjcaleb Member

    @ErawanArifNugroho what do you mean?

  • @jcaleb said: @ErawanArifNugroho what do you mean?

    I can't perform #apt-get update or #apt-get upgrade, or #yum update

  • yomeroyomero Member

    @ErawanArifNugroho said: I can't perform #apt-get update or #apt-get upgrade, or #yum update

    sudo????
    LOL, nah just kidding :P

    When I had the vz one I used Debian 6 without issues o_O

    Thanked by 1ErawanArifNugroho
  • @ErewanArifNugroho: maybe you should have tried emerge world instead ;) (gentoo joke)

  • [@yomero] Yes, with Debian6 it's working, but I don't understand to run with Debian6 :D

    @quirkyquark said: @ErewanArifNugroho: maybe you should have tried emerge world instead ;) (gentoo joke)

    hehe...

  • @BlueVM said: The VPN clients were under an unprivileged user.

    How did he cause you to lose data? Did he somehow (exploit, etc) get root?

  • yomeroyomero Member
    edited May 2012

    VPN clients doesn't need data o_O
    Or the node had containers too?
    Edit: nvm, I read again xD

  • FRCoreyFRCorey Member

    This is why we send backups to Amazon S3 for safekeeping. Blow up the node, worset my customers who are not following TOS will at least have something that's 24 hours old.

    Another reminder that people need to keep backups for themselves even if the provider provides it.

  • BassHostBassHost Member
    edited May 2012

    @FRCorey said: This is why we send backups to Amazon S3 for safekeeping. Blow up the node, worset my customers who are not following TOS will at least have something that's 24 hours old.

    Self marketing at its best :)

  • SpencerSpencer Member

    @FRCorey said: worset my customers who are not following TOS will at least have something that's 24 hours old.

    Hopefully the content isnt so terrible and that amazon finds out and suspends you!

  • BlueVMBlueVM Member
    edited May 2012

    @FRCorey said: This is why we send backups to Amazon S3 for safekeeping. Blow up the node, worset my customers who are not following TOS will at least have something that's 24 hours old.

    So you spend $1395.00 a month to backup just 1,000 clients or 37% of your income without paying for any S3 bandwidth? It's great your so dedicated :)

    What part of Colorado are you from?

    --
    That being said I took a nap and now I feel much better.

  • @BlueVM said: So you spend $2075.00 a month to backup just 1,000 clients or 66% of your income without paying for S3 bandwidth?

    This is what I was thinking ... Amazon does not make any sense as a back up solution for someone providing hosting solutions.

  • BlueVMBlueVM Member

    @BassHost said: This is what I was thinking ... Amazon does not make any sense as a back up solution for someone providing hosting solutions.

    I re-did my math and assumed he only kept 3 backups on file: a daily, a weekly and a monthly... If you add in bandwidth his customers are actually saving money over choosing S3.

  • FreekFreek Member
    edited May 2012

    First thing that came to mind:
    image

Sign In or Register to comment.