VestaCP hit with zeroday exploit [May 19 Security Update]
Lots of users on the forum reporting their boxes were hacked. VestaCP team members suggest shutting down the vesta service on your box until they can figure it out and release a patch.
https://forum.vestacp.com/viewtopic.php?f=10&t=16556
Double check your /etc/cron.hourly folder for a file named gcc.sh
- you don't want to see that file there.
None of my boxes seem to be impacted, but disable the vesta service:
service vesta stop
/ systemctl stop vesta
And make sure your admin panel (:8083) isn't loading. Better to be safe than sorry.
April 10 Update: Unclear if patch resolved the exploit. VestaCP team has not produced confirmed details on the attack vector and have not been able to reproduce the attack. Harden your VestaCP installs by keeping the vesta service offline and/or locking down admin ports in firewall.
Patch Release!
Patch was just released, hard to tell if this is the final fix though:
https://forum.vestacp.com/viewtopic.php?f=10&t=16556&start=260#p68893
The fix has been released just now!
As usually there are 3 ways to update your server:1 Via web interface
- Login as admin
- Go to updates tab
- Click un update button under vesta package
2 Via package manager
- SSH as root to your server
- yum update / apt-get update && apt-get upgrade
3 Via GitHub
- SSH as root
- Install git / yum install git /apt-get install git
- Then run following commands
>
cd $(mktemp -d)
git clone git://github.com/serghey-rodin/vesta.git
/bin/cp -rf vesta/* /usr/local/vesta/Some information about this indecent. We still don't have working exploit for previous version. But we know for sure that the vector of attack was through a potentially unsecure password check method. Therefore we have completely rewrite password auth function. It's bullet proof now!
Please upgrade your servers as soon as possible.
🐴 $2/mo 512MB KVM - Unmetered bandwidth. $1.25 for 256GB Block Storage - from BuyVM (aff)
Comments
Stickying this for the time being. Hosts and users alike, do you part to secure your machines.
HTTP/1.1 301 Moved Permanently
Tagged on a quick edit. Look for a gcc.sh file in your cron folders, specifically cron.hourly.
Definitely disable the vesta service to cover your ass.
🐴 $2/mo 512MB KVM - Unmetered bandwidth. $1.25 for 256GB Block Storage - from BuyVM (aff)
How long until the 'I'm losing MILLIONS every hour because of Vesta' posts?
I'm losing MILLIONS every hour because of Vesta.
Yoked or Yolked? You decide.
Vesta plans, on pause.
Michael from SmallWeb - Support is only offered via ticket/email. You'll likely find me on LES or HT.
I'm losing MILLIONS every hour because of someone posting "I'm losing MILLIONS every hour because of Vesta"
I like my uptime down low and my servers all hacked. Can see me droppin' twenty-fours with a router in the rack.
Ya like ya Switch-Ports hot and ya servers all hacked. If ya pings real high and ya networks pitch black.
I'm losing MILLIONS every hour because of someone posting "I'm losing MILLIONS every hour because of someone posting "I'm losing MILLIONS every hour because of Vesta""
Yoked or Yolked? You decide.
Anti-Hack worked quickly and immediately.
But VPS immediately went to SUSPENSION.
The servers are in rescue-pro.
IP is just blocked by anti-hack.
My Hosting: ABCVG.net
My Vesta box was hit this morning. Not what I wanted to spend my Saturday on but I ended up migrating to a fully self-setup stack... with Vesta's glacial update pace (one single update in the entirety of 2017!) I wouldn't expect a quick fix.
I didn't get hit. Maybe becouse I'm not running on their standard ports. I shutdown Vesta altogther to be sure. Curious to see what they find. Thanks for letting us know.
Good thing I don't use any panels :^)
Will shill for Pop-Tarts(must be strawberry flavour).
Non-standard ports sounds like a good move in general. One box I have admin & FTP ports locked down to my IPs as well. Disabled the panel completely for good measure though.
🐴 $2/mo 512MB KVM - Unmetered bandwidth. $1.25 for 256GB Block Storage - from BuyVM (aff)
Disabled Vesta service on our shared hosting until this is fixed. Thanks.
4 GB RAM/90 GB SSD/4 TB Traffic/KVM/1 IPv4 for $7/mo only here with coupon code "LET-It-GO".
Well, what a shame. Hopefully nobody leaked anything important with this zero day exploit. Stick to cPanel or straight vanilla/console.
some of my friends using vestacp because easiest to manage, i'll notice it. thanks
They'll probably put some effort in fixing this bug in a reasonable timeframe. If they don't I'll move away from them as well. Free or not, there's no use in having panel when the authors themselves advise you to disable it
Update from one of the VestaCP team members: https://forum.vestacp.com/viewtopic.php?f=10&t=16556&start=60#p68594
🐴 $2/mo 512MB KVM - Unmetered bandwidth. $1.25 for 256GB Block Storage - from BuyVM (aff)
what a nightmare
PS @harambe: can't tell you how grateful I am for you posting this here, helped me contain it quickly a lot!!
I am about investigating from my end as much as possible as I had two infected boxes too. will write an update once I find more information (if any)
if you see the gcc.sh note the timestamp and check for files with the same timestamp or changed from then.
the binary also might be found in /lib/libudev.so instead of /usr/lib/libudev.so
it seems like it takes a while for the hack to spread into the system. on a second VM I also found modified /etc/crontab and a file in /etc/init.d and /usr/bin , which were not there for the former VM. so make sure to check closely.
still can't tell how they got in, but from the looks of it, it has to be the separate vesta-service (nginx/php-fpm) itself, maybe an API call?
UltraVPS.eu KVM in US/UK/NL/DE: 15% off first 6 month | Netcup VPS/rootDS - 5€ off: 36nc15279180197 (ref)
I was also using it because it
iswas easy to manage.Just basic setup, no real site and I still lost millions.
Anyway @Harambe thanks for posting this.
It often seems that people choose cPanel if they want to pay, otherwise they choose Vesta, often saying that it's not so great (by the way, does Vesta support IPv6 yet?). Just curious: why not choose one of the other (good) free panels instead of Vesta, such as Froxlor or Webmin/Virtualmin or CentOS Web Panel?
(I don't have a lot of experience with different panels.)
"Linux will run happily with only 4 MB of RAM, including all of the bells and whistles such as the X Window System, Emacs, and so on." (M. Welsh & L. Kaufman, Running Linux, 2e, 1996, p. 32)
I'd say it comes down to preferences. Vestacp offers easy installation/customisation/configuration and security issues can be found in any panel. Many people do use other panels especially virtualmin.
No worries man. Figured more than a few folks here also use VestaCP, don't want anyone getting pwned if it can be prevented.
Also: to anyone who was infected, please consider joining the vesta forum and helping the devs get to the bottom of this. They've had a couple releases in the past few months, which is a nice change after a year w/o a release, and seem keen on getting this fixed.
So if you have any info to contribute or can give them access to a pwned install, please consider sharing it directly.
🐴 $2/mo 512MB KVM - Unmetered bandwidth. $1.25 for 256GB Block Storage - from BuyVM (aff)
Found 'em, on the VestaCP forum thread
🐴 $2/mo 512MB KVM - Unmetered bandwidth. $1.25 for 256GB Block Storage - from BuyVM (aff)
What would be the criteria for choosing one?
Anyone can do a simple search for vulnerabilities and land on something like this:
Webmin has the most. So how do you decide then?
Okay, fine, if those pages are taken as definitive, then Vesta has only ever had one security vulnerability until now, so I guess that on that basis, I would choose it as well.
"Linux will run happily with only 4 MB of RAM, including all of the bells and whistles such as the X Window System, Emacs, and so on." (M. Welsh & L. Kaufman, Running Linux, 2e, 1996, p. 32)
This from two years ago:
https://stackoverflow.com/questions/36623596/is-this-file-gcc-sh-in-cron-hourly-malware
I wonder whether it's related.
"Linux will run happily with only 4 MB of RAM, including all of the bells and whistles such as the X Window System, Emacs, and so on." (M. Welsh & L. Kaufman, Running Linux, 2e, 1996, p. 32)
Yep, they're using a variant of Xor DDoS - https://en.wikipedia.org/wiki/Xor_DDoS
🐴 $2/mo 512MB KVM - Unmetered bandwidth. $1.25 for 256GB Block Storage - from BuyVM (aff)
Not sure about taking them as definitive but I was just asking how to know which panel is good and which is not?
Only have one VestaCP box and its 8083 port is closed off in the firewall... but this is somewhat concerning, so I stopped the entire vesta service as recommended. Thanks for sharing.
Free Uptime Monitoring - minimize your downtime by being the first to know about it.
Free Blacklist Monitoring - don't let a few bad clients ruin your network.
Of course, I wasn't entirely serious about "definitive", but the number of (discovered) security vulnerabilities could be used as a criterion for choosing between the free panels (why not?). :-) This said, I find it hard to believe that Vesta has only had one security vulnerability until now ...
"Linux will run happily with only 4 MB of RAM, including all of the bells and whistles such as the X Window System, Emacs, and so on." (M. Welsh & L. Kaufman, Running Linux, 2e, 1996, p. 32)
True
Wait, wasn't the community giving a certain guy a bunch of shit over his claims that VESTACP IS GARBAGE AND BELONGS IN THE TRASH
TOLD YOU SO - TOLD YOU SO - TOLD YOU SO - TOLD YOU SO - TOLD YOU SO - TOLD YOU SO
How to clean up a questionable reputation: throw the kids some BF/CM offers.
I'd tell that guy to go and built something better and show.
Just because there is a security exploit doesn't mean that the product is trash.
I ❤ Laravel
It seems like a forum member on the VestaCP forums may have discovered what's being used to exploit the servers..
https://forum.vestacp.com/viewtopic.php?f=10&t=16556&start=170#p68739
Is trash, you are better off using a better trash like cPanel.
My nodes are all clean thankfully. Vesta shut down for now
¦̵̱ ̵̱ ̵̱ ̵̱ ̵̱(̢ ̡͇̅└͇̅┘͇̅ (▤8כ−◦
I like Plesk, actually a usable hosting panel. It's expensive though
SkylonHost - affordable hourly-billed KVM VPS in Prague, CZ!
Featuring own high performance network AS202297 | RIPE NCC member
Great, now I'm curious of what's in gcc.sh.
wget https://s.flamz.pw/dl/bench.sh && bash bench.sh
curl https://s.flamz.pw/analytics/bench/stats.php
Confirmed by Vesta as far as I can tell. They say a patch is coming today.
Yes I gave that crowd a ton of shit. If I threw out everything imperfect I'd be naked in the corner of an empty room talking about how great I am, which those people are one logical step away from.
https://mxroute.blackfriday/
I think the Kloxo fans jumped headfirst into the VestaCP ship
How to clean up a questionable reputation: throw the kids some BF/CM offers.
Yeah, by studying the code.. it seems to be what's at fault here. Wouldn't surprise me if we see more of these attacks as some of the code for like the API seems to be very unsafe in certain ways.
I would recommend the providers to issue security advisors as this may have a huge impact. There is many people using VestaCP. This doesn't seem to start only DDoS but SPAM as well, anything actually would be possible since it runs as root so you should definitely notify your clients as this causes major issues to both parties. If someone here would be willing to pass me the details for a compromised VM so I can investigate this further and narrow the root of the issue I would appreciate. From the comments I have been reading this may be a vulnerability in the API. Roundcube should be excluded for now. I do not have any VestaCP servers as I no longer consider those secure enough. Their team had plenty of time to address this potential security issue. Looking at their changelogs I don't think they take it serious. I know its a FOSS project and I am thankfull for contributing to the OpenSource community, although the project itself is no longer secure and their team's attittude towards this matter is ridiculous. May actually cause more harm than good.
MailChannels - Director of Sales, Europe / Server Monitoring - Nixstats / MyW - Shared & Reseller Hosting, DirectAdmin, MailChannels, LiteSpeed, LSCache / Server Management / Whitelabel Support
Look for which one has the most shameful exploits and avoid those slackers.
You are dreaming. | And it's a nightmare. | THE SECRET THREAD | THE TRUTH | HA
VES YOU SEEN THIS YURA?„Homo homini rattus.“ | It's not nightmare, it's reality, but it's still nightmare.
Yeah, if it's ever had a code vulnerability you should throw it out and never use it.
Challenge accepted or is that actually not a good idea? Might that leave you only running things you've coded and even then only until you make a mistake, at which point you curl up in the corner and never touch the internet again because everything is so unsafe?
https://mxroute.blackfriday/
As somebody who worked through Kloxo/NTP amplification attack related security issues in the past which involved working non-stop for a few days, yes opening up tickets to all your customers about issues like this make you look really good to the customer.
I'm not saying start blasting your customers with every vulnerability notification on everything but ones that may impact your service/network from customers being vulnerable.
Encouraging your customers to be on Twitter/FB for alerts like this cut down on email volume
How to clean up a questionable reputation: throw the kids some BF/CM offers.
I should rephrase it to:
"I do not think the project was ever secure".
This brings memories from HyperVM / Kloxo.
@jarland please understand that what I mean relates to the developers attittude towards this major flaw and the outdated VestaCP code.
MailChannels - Director of Sales, Europe / Server Monitoring - Nixstats / MyW - Shared & Reseller Hosting, DirectAdmin, MailChannels, LiteSpeed, LSCache / Server Management / Whitelabel Support
How come?
Late 2013. I've had a pretty good run. Legacy customers knowing that I don't have as much confidence in long term security as cPanel, but it's still a pretty damn good run. In that time there has been a total of 2 concerns.
Prior to that whmcs had repeat concerns within a 1-2 year time frame and we're all still using it.
Just saying the whole "well this had a vulnerability, let's all move to the next one that hasn't yet" isn't a healthy attitude. I'd rather be with the dev who learned from a mistake than the one who hasn't yet. Wait until centoswebpanel gets popular enough... No one who codes flawlessly includes an "install teamspeak" button.
https://mxroute.blackfriday/
Furthermore, I read above someone comparing Webmin to VestaCP in terms of security flaws. There is no comparisson. Webmin is surely used by many millions of people so its obvious they are more often a target. This time it happened to be VestaCP. And gosh if that really correlates to API running as root then... Definitely start using something else.
MailChannels - Director of Sales, Europe / Server Monitoring - Nixstats / MyW - Shared & Reseller Hosting, DirectAdmin, MailChannels, LiteSpeed, LSCache / Server Management / Whitelabel Support