New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
Did last | reboot, instead of last reboot on a production server.
This is perhaps the most embarrassing on helpdesk..
Not me, but someone supposedly re-ran my install script, wiping about 600 users data.
1) When upgrading debian and grub I didn't selected any of the raid disks and rebooted.
2) Trying to setup IPv6 I stopped the network interface locking myself out without any IPMI/KVM.
Luckily both servers were not for production.
Needed libstdc++.so.5 on Squeeze and Wheezy. Typed:
Should work, right? After that, every command typed into the OS caused Segfaults. Had to recreate the instance and do it this way.
THIS
After 25+ years of managing servers, I've done almost all of these... And don't forget the occasional fdisking of the wrong drive.
My favorite however was when I worked for a computer chip design company back in the early 90's and they had compute jobs that ran for days. All the computers were on UPSs to protect against power flickers.
One guy, we'll call him "George" had a critical job running for >40 hours. And of course, we lost power. So I'm frantically trying to keep his workstation alive since if it dropped, his job dropped.
He's freaking out.
I'm running around the office pulling UPSs off any computer I can. I set one down in his office and am getting ready to nest the UPSs.
Did I mention how he's freaking out?
He, trying to help, unplugged his computer from his UPS and then plugged it into the 'new' UPS. My jaw dropped.
Oops.
On the plus side, he gave me $20 to not tell anyone.
Probably shutdown -h now on the wrong node.
Erasing the / portition but I recovered in 1 hour with idera bare metal restore
buying a service from a company that doesn't care about their users or nodes. I have spent SO much time patching and putting in tickets because they couldn't manage their nodes.
nothing too bad on a production server.
worst is probably apt-get remove --purge lighttpd while I was in the wrong PuTTY window (for a production server), but that only resulted in a couple minutes downtime until I realised what I'd done.
I've done rm -rf / too just to see what happens
edit: I mean I knew what it does, but I wasn't sure if it would let you do it. now you need to use --no-preserve-root
No worry, we all did that!
rm -rf httpd.conf
Did that on Host1Free to see what it does. Nothing important lost but good experience.
chown www-data:www-data / -Ris the worst. You feel you're so close that you spend hours trying to recover rather than restoring from backups.had a sight split brain issue with drbd.... replaced faulty drive... then commissioned the newly formatted block as master and told the live to sync to it
Back in around 2003, I was running a VPS business using UserModeLinux (terrible visualization technology, but that isn't hte mistake). It ran all the vm's in screen sessions. I was working ona new base image and ran rm -rf /home . The problem I wasn't in the screen session of the vm I thought, I was on the host node and all my users vms were in the /home ...
That was a sleepless night, but we managed to recover them all buy dumping the running vm's ram state to disk . Was a very stressful night!
Two years ago I rebooted a Xen VPS about 10 times when it was resizing it's disk and corrupted all of it's inodes.
disable root login and install fail2ban, and I forgot the password..
On that topic, I've done
pkill -v tincforgetting that v does not mean verbose.rm -rf /*instead of
rm -rf *Didn't have enough coffee...
4 - 5 years ago:
rm -fr /instead ofrm -fr ./Hell, any killing of tinc for me would be an issue because that's what my management interface is on.
@DalComp: Yeah, we are colleagues (handshare)
1) Disabled root login on SSH and without adding alternate user - Locked out
2) Changed the SSH Port without enabling it on iptables - Locked out
Let her idle
rm -rf /var/lib/mysql/* instead of /var/lib/mysql/dbPrefix_* to clear stale .frm tables / dirs.
FML
my biggest mistake was: I miss the cheap VPS offer on LET
removed yum