Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In with OpenID
Advertise on LowEndTalk.com

In this Discussion

NodeDeploy

NodeDeploy

birdie25birdie25 Member
edited July 2012 in General

Shit's hit the fan, yo.

http://nodedeploy.com/

Proud member of the VPS Collectors Club

«1

Comments

  • Looks like it @birdie25

  • gsrdgrdghdgsrdgrdghd Member without signature

    Maybe tag @PhilND but i'm sure hes busy right now fixing stuff.

  • JackJack Member
    edited July 2012

    @gsrdgrdghd said: Maybe tag @PhilND but i'm sure hes busy right now fixing stuff.

    There LEB offer has a lot of info.

    image image image image

  • PhilNDPhilND Member

    @Everyone

    The hardware raid controller on aura completely nuked it's self and the entire raid array, taking the disks with it, hetzner are pending a shipment of raid controllers in the morning. Fortunately no data was lost as we have measures in place to recover from such instances, it's just a pain to have to shuffle 1.5tb of data around overnight on a very very tight time scale.

    Rest assured we're getting things ironed out and new procedures put in place to stop un announced things like this happening again, e.g complete downtime of our services, websites etc. Phoenix and Zeus are unaffected however, we realise this is unacceptable and we're willing to offer compensation and discuss things with clients who feel mistreated, if you need direct contact, ding an email to [email protected]

    Thanks for your patience.

    Phil

    https://nodedeploy.com | Premium VPS Solutions | Managed

  • @PhilND said: The hardware raid controller on aura completely nuked it's self and the entire raid array, taking the disks with it, hetzner are pending a shipment of raid controllers in the morning. Fortunately no data was lost as we have measures in place to recover from such instances, it's just a pain to have to shuffle 1.5tb of data around overnight on a very very tight time scale.

    Rest assured we're getting things ironed out and new procedures put in place to stop un announced things like this happening again, e.g complete downtime of our services, websites etc. Phoenix and Zeus are unaffected however, we realise this is unacceptable and we're willing to offer compensation and discuss things with clients who feel mistreated, if you need direct contact, ding an email to [email protected]

    Thanks for your patience.

    Phil

    All the best :)

  • PhilNDPhilND Member

    @ShardHost Thanks! Teething problems.. always do suck the most! Things were running smooth as aswell. We'll get it sorted :)

    https://nodedeploy.com | Premium VPS Solutions | Managed

  • Did the disks die during a rebuild or did the controller dump trash onto the disks?

    BBU/No BBU?

  • JackJack Member

    .@PhilND Good Luck hopefully no data is lost!

  • PhilNDPhilND Member

    @ShardHost

    Dumped aload of crap onto the disks, was then getting I/O errors from 2/4 drives in the array, thought it would be best for them to pull them all and stick some more in.

    This particular model doesn't have BBU on it (though it does support it, PITA :()

    @Jack We're verifying everything we have on our backup servers, everything looks good to go, waiting on raid controller before we push it all over. :)

    https://nodedeploy.com | Premium VPS Solutions | Managed

  • JackJack Member

    @PhilND what are you using to make the backups? Just SolusVM or ?

  • JacobJacob Member

    Thats just a disaster waiting to happen, Controller with no BBU, Oh well....

  • PhilNDPhilND Member

    @Jack Rsync to an external server :-)

    https://nodedeploy.com | Premium VPS Solutions | Managed

  • JarJar Member

    It happens. Hindsight is always 20/20. Good luck and hope you guys get everything straightened out.

  • Everyone can have problem, so, good luck, and wish you all the best :)

    My blog | Server Uptime | I'm not working for any providers in here, all my comments just my own opinion.image
  • This is why you keep your website on a different server, amateur mistake :P

  • JarJar Member

    @Satellite No one wants to think about hardware failure, it hurts ;)

    For real though I have my website on my shared hosting server. That's in my list for July. It was a dumb thing to do, but at the time it didn't even occur to me.

  • PhilNDPhilND Member

    @jarland said: but at the time it didn't even occur to me.

    This. It'll all be resolved in due course!

    https://nodedeploy.com | Premium VPS Solutions | Managed

  • Yep, school boy error. But best of luck @PhilND getting stuff sorted.

    For reference, we host our main site with @prometeus and <3 it so far.

    Thanked by 1ErawanArifNugroho
  • JackJack Member

    @GetKVM_Ash said: For reference, we host our main site with @prometeus and <3 it so far.

    @GetKVM_Ash I got a KVM with them to do the same thing haha.

  • @prometeus you're the host of hosting companies :P

    Catalyst Host - Pie Approved!
  • PADPAD Member

    Never host your WHMCS on your own shared hosting node, so easy to hack its unbelivable. So many vulnerabilities. Lesson learnt though right? :D GL getting everything back up.

  • @HalfEatenPie said: @prometeus you're the host of hosting companies :P

    Well, when I realized that there are more hosts than clients I adapted our business :P

    IperWeb & Prometeus, Hosting Provider since 1997. iwStack cloud infrastructure
  • @prometeus said: Well, when I realized that there are more hosts than clients I adapted our business :P

    Hahah :P Well, looks like most hosts are customers too for things like DNS, monitoring, even main site host :) M

    Who's General Failure, and why is he reading my drive A: ?

  • PhilNDPhilND Member

    @PAD It was actually on a VPS, not our shared node.

    https://nodedeploy.com | Premium VPS Solutions | Managed

  • Ash_HawkridgeAsh_Hawkridge Member
    edited July 2012

    @PAD said: Never host your WHMCS on your own shared hosting node, so easy to hack its unbelivable. So many vulnerabilities. Lesson learnt though right? :D GL getting everything back up.

    What lol. A shared hosting server is no more vulnerable than a VPS or dedicated server :S

    Well, at least if your with a reputable company.

    Thanked by 1Jar
  • CoreyCorey Member

    I think it's funny that 'providers' here will rent a VPS from another provider to host their main website and talk to someone else about making a 'school boy error'. I could see renting a VPS from another provider for redundancy on your own site - but why the hell would you put your main site in the hands of another VPS provider here?

    BitAccel - OpenVZ VPS / IRC,VPN,Anything Legal & Unrivaled Support!
  • @Corey said: I think it's funny that 'providers' here will rent a VPS from another provider to host their main website and talk to someone else about making a 'school boy error'. I could see renting a VPS from another provider for redundancy on your own site - but why the hell would you put your main site in the hands of another VPS provider here?

    Dude, you could have just tagged me if you wanted to bitch.

  • PADPAD Member

    @GetKVM_Ash

    Yes it is, huge difference - you have configuration, control, and privacy.

    I would never put my own site hosting confidential information alongside my customers.

    @Corey exactly, same aspect.

  • PADPAD Member

    @GetKVM_Ash I don't think he was trying to start an argument, but he has a valid point. Its idiotic to me and would put me off hosting with you. I just can't see it making any sense.

  • Ash_HawkridgeAsh_Hawkridge Member
    edited July 2012

    @Corey said: I think it's funny that 'providers' here will rent a VPS from another provider to host their main website and talk to someone else about making a 'school boy error'. I could see renting a VPS from another provider for redundancy on your own site - but why the hell would you put your main site in the hands of another VPS provider here?

    Okay, let me elaborate.

    Its safer because its outside my own network and thus our site will remain online in cases where our other services may be down.

    And i don't see why i wouldn't put my site in the hands of another reputable provider, even more so one that advertises here. Obviously if anything shady went off, it would all come out here?

    @PAD said: I would never put my own site hosting confidential information alongside my customers.

    Its not alongside my customers.

  • rds100rds100 Member

    I don't know how is it in the UK, but here we have laws about how we should handle customer personal data (names, addresses, etc.). I am sure it would be illegal to host this information somewhere else or even worse - in a different contry.

  • @rds100 said: I don't know how is it in the UK, but here we have laws about how we should handle customer personal data (names, addresses, etc.). I am sure it would be illegal to host this information somewhere else or even worse - in a different contry.

    This has been explained far too many times. Moving data from its original place to an outside country is illegal (But only in some cases, depending on what info is stored) but regardless, the WHMCS install is in Italy, which is in Europe. So where kosher.

  • JarJar Member

    @PAD I encourage any of my customers to make their way to my HostBill database via any method that is significantly accelerated by their having a shared hosting account on my server. If they manage an exploit, I then encourage them to enjoy their ban before they get there ;)

    Thanked by 1Ash_Hawkridge
  • CoreyCorey Member

    @GetKVM_Ash said: @Corey said: I think it's funny that 'providers' here will rent a VPS from another provider to host their main website and talk to someone else about making a 'school boy error'. I could see renting a VPS from another provider for redundancy on your own site - but why the hell would you put your main site in the hands of another VPS provider here?

    Dude, you could have just tagged me if you wanted to bitch.

    You weren't the only one in this thread that keeps your site on another vps provider's services.

    BitAccel - OpenVZ VPS / IRC,VPN,Anything Legal & Unrivaled Support!
  • @Corey said: You weren't the only one in this thread that keeps your site on another vps provider's services.

    You quoted my text :S

  • Although I wouldn't host our corporate sites on a VPS from a provider, ultimately unless you own the location where you are housing your websites you are putting your trust in a third party somewhere along the line.

    Thanked by 1Ash_Hawkridge
  • PADPAD Member

    @jarland ignorance is bliss.

  • JarJar Member
    edited July 2012

    @PAD Properly configured and monitored server is even better. LiquidWeb has one of the best monitoring systems around, extremely quick to react staff members 24/7/365, and I always keep up to date with the latest exploits and take appropriate measures. Anyone ahead of me there, well they don't need to be a customer of mine to break in and I won't help myself by being hosted outside of the system. A determined and knowledgeable hacker is going to find a way in.

    The reason I'm moving it outside of the server this month is for communication purposes in the event of downtime.

  • MaouniqueMaounique Member
    edited July 2012

    For me it would make sense to host the site some other place if I put there only the front-end, the general static data about the company that is available in many other places and to advertise the products. Regarding trust, well, we have to draw the line somewhere. If we go that path to the end, everyone should make their own DC. This is why there are clouds out there, so ppl will host their stuff, even corporations, I think it is less likely to leak your database in a managed cloud some place than in your own DC where the janitor has the keys, not to mention the risk to lose the data in case of an earthquake, fire or flood. That being said, I still feel bad about depending on someone else, maybe because the industry is not mature enough and I get replies like "no, you only have one IP, not 6" when I ask about IPv6. I simply can't trust those ppl. M

    Who's General Failure, and why is he reading my drive A: ?

  • PhilNDPhilND Member

    Hi all,

    We're SYNCing data over to the new array shortly.

    Thanks for your continued paitence.

    Phil

    https://nodedeploy.com | Premium VPS Solutions | Managed

  • @Maounique said: That being said, I still feel bad about depending on someone else, maybe because the industry is not mature enough and I get replies like "no, you only have one IP, not 6" when I ask about IPv6.

    LMFAO Who/where said that... if you don't mind disclosing.

  • JacobJacob Member

    Three Locations and DNS Failover, You got yourself a possible 100% Uptime every month. :-)

    @Corey said: I think it's funny that 'providers' here will rent a VPS from another provider to host their main website and talk to someone else about making a 'school boy error'. I could see renting a VPS from another provider for redundancy on your own site - but why the hell would you put your main site in the hands of another VPS provider here?

    Thanked by 1Ash_Hawkridge
  • @Jacob said: Three Locations and DNS Failover, You got yourself a possible 100% Uptime every month. :-)

    +1 on that brother.

  • @Jacob said: Three Locations and DNS Failover, You got yourself a possible 100% Uptime every month. :-)

    I got a similar setup, however with a load balancer, not DNS failover. How do you manage to keep your mysql DB in a sync? Remote DB or somehow you sync everything in real time?

    Disclosure: I work for Query Foundry LLC.
    I own DA International Group Ltd.
  • LAKidLAKid Member

    how do you sync mysql db in real time? would you share?

  • PhilNDPhilND Member

    @LAKid

    We don't sync 'realtime' we sync every night with an LVM Snapshot of individual VM's Then rysnc the differences, no point in syncing 100GB of unchanged data, we only want what's changed.

    Thanks, Phil

    https://nodedeploy.com | Premium VPS Solutions | Managed

  • PhilNDPhilND Member

    All VM's are online an data migrated, we're upgrading everyone from 60GB -> 80GB and adding addtiona days (4) to your services.

    Thanks, Phil

    https://nodedeploy.com | Premium VPS Solutions | Managed

  • PhilNDPhilND Member

    We appear to be fully functional again, if you experience any issues, just drop us a ticket. There is a chance we have missed a thing or two, had like 0 sleep in the past 48 hours.

    https://nodedeploy.com | Premium VPS Solutions | Managed

    Thanked by 1Infinity
  • MaouniqueMaounique Member
    edited July 2012

    @GetKVM_Ash said: LMFAO Who/where said that... if you don't mind disclosing.

    One of my home ISPs, RDS here, offered free ipv6 for home a /64 and naturally, i wanted to have the same at work, even if I was to pay something for it so I called the Business helpdesk and asked about that. I leave aside the fact they were bound by contract to offer us /29 of v4 at least which they are postponing for more than a year. I got that answer and had to explain what IPv6 is, but the guy was totally lost. He said he will call back and he did with anegative answer, no, they dont offer IPv6 for businesses... M

    Who's General Failure, and why is he reading my drive A: ?

  • @LAKid said: how do you sync mysql db in real time? would you share?

    what you ask is for replication

    http://dev.mysql.com/doc/refman/5.0/en/replication.html

Sign In or Register to comment.